Gitleaks — Secret Scanning

Gitleaks is the first stage of the DevSecOps pipeline. It scans the entire git history of a repository to detect accidentally committed secrets before any build or deployment activity begins.

---

What Gitleaks Detects

Gitleaks uses a configurable rule engine with over 140 built-in patterns covering:

Category	Examples
API keys	AWS access keys, Azure SAS tokens, GitHub PATs
Passwords	Hardcoded credentials in connection strings
Tokens	JWT secrets, OAuth tokens, webhook secrets
Certificates	Private keys (RSA, EC, PGP) embedded in code
Connection strings	SQL Server, Redis, RabbitMQ strings with credentials
Cloud secrets	Azure storage account keys, Cosmos DB keys
Microtec-specific	XApiKey values, internal service tokens

---

Pipeline Integration

Gitleaks runs as Stage 1 using the official Docker image:

- stage: SecretScan
  displayName: 'Stage 1 - Secret Scan (Gitleaks)'
  jobs:
    - job: Gitleaks
      pool:
        vmImage: 'ubuntu-latest'
      steps:
        - checkout: self
          fetchDepth: 0            # full git history required

        - task: Bash@3
          displayName: 'Run Gitleaks'
          inputs:
            script: |
              docker run --rm \
                -v $(Build.SourcesDirectory):/repo \
                zricethezav/gitleaks:latest \
                detect \
                  --source /repo \
                  --verbose \
                  --redact \
                  --config /repo/.gitleaks.toml \
                  --report-format sarif \
                  --report-path $(Build.ArtifactStagingDirectory)/gitleaks-report.sarif \
                  --exit-code 1

        - task: PublishBuildArtifacts@1
          condition: always()
          inputs:
            pathToPublish: '$(Build.ArtifactStagingDirectory)/gitleaks-report.sarif'
            artifactName: 'security-gitleaks'

Important: fetchDepth: 0 is mandatory. Without full history, Gitleaks only scans the working tree and will miss secrets that were added and then "deleted" in later commits — they still exist in git history.

---

Configuration File (.gitleaks.toml)

Place this file in the repository root to customize detection behavior.

# .gitleaks.toml
title = "Microtec ERP Gitleaks Configuration"

[extend]
# Extend the default ruleset rather than replacing it
useDefault = true

# ─────────────────────────────────────────────
# Microtec-specific custom rules
# ─────────────────────────────────────────────

[[rules]]
id = "microtec-xapikey"
description = "Microtec internal XApiKey hardcoded"
regex = '''['"](3bb564df-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})['"]'''
tags = ["api-key", "microtec"]
severity = "critical"
keywords = ["XApiKey", "x-api-key", "xapikey"]

[[rules]]
id = "microtec-connection-string"
description = "Microtec SQL Server connection string with password"
regex = '''(?i)(Server|Data Source)=[^;]+;.*Password=[^;'"\s]+'''
tags = ["connection-string", "database"]
severity = "critical"
keywords = ["Password=", "Pwd="]

[[rules]]
id = "microtec-keycloak-secret"
description = "Keycloak client secret"
regex = '''(?i)keycloak[_-]?client[_-]?secret['":\s=]+[0-9a-zA-Z\-_]{20,}'''
tags = ["keycloak", "secret"]
severity = "critical"

[[rules]]
id = "azure-storage-account-key"
description = "Azure Storage Account access key"
regex = '''(?i)AccountKey=[A-Za-z0-9+/]{86}=='''
tags = ["azure", "storage"]
severity = "critical"

# ─────────────────────────────────────────────
# Allowlist — paths excluded from all rules
# ─────────────────────────────────────────────

[allowlist]
description = "Global allowlist for test and doc files"
paths = [
  '''\.gitleaksignore''',
  '''tests?/fixtures?/''',
  '''(?i)test[_-]?data''',
  '''\.md$''',
]

---

.gitleaksignore — False Positive Management

For findings that are confirmed non-secrets (test fixtures, documentation examples, hashed values), add the finding fingerprint to .gitleaksignore:

# .gitleaksignore
# Format: <sha256-of-secret>:<rule-id>
#
# Entry added 2025-01-15 by M.Araby
# Justification: Test connection string uses placeholder password from documentation
a1b2c3d4e5f6...:<rule-id>

To get a fingerprint from a Gitleaks report:

gitleaks detect --source . --report-format json | \
  jq '.[] | select(.RuleID == "microtec-connection-string") | .Fingerprint'

Policy: Every .gitleaksignore entry must include a comment with the date, author, and justification. Entries are reviewed quarterly by the security team.

---

Pre-Commit Hook Option

Developers can run Gitleaks locally before every commit to catch secrets before they reach Azure DevOps:

# Install Gitleaks locally (macOS)
brew install gitleaks

# Install the pre-commit hook into the repo
cat > .git/hooks/pre-commit << 'EOF'
#!/bin/bash
gitleaks protect --staged --redact --config .gitleaks.toml
if [ $? -ne 0 ]; then
  echo "❌ Gitleaks detected secrets in staged files. Commit blocked."
  echo "   Review the findings above and remove all secrets before committing."
  exit 1
fi
EOF
chmod +x .git/hooks/pre-commit

Alternatively, use the pre-commit framework:

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.18.4
    hooks:
      - id: gitleaks
        args: ['--config', '.gitleaks.toml']

Install hooks for all team members:

pip install pre-commit
pre-commit install

---

What To Do When a Secret Is Detected

Critical: A secret detected in git history means it may already be compromised, even if the pipeline blocked deployment.

Immediate Response

1. Rotate the secret immediately — do not wait. Assume it is compromised.
2. Update the secret in Azure Key Vault.
3. Update all services that use the secret.
4. Notify the security team via the #security-incidents Teams channel.

Removing the Secret from Git History

# Option A: git filter-repo (recommended)
pip install git-filter-repo
git filter-repo --path-glob '**/*.cs' --replace-text secrets-to-remove.txt

# secrets-to-remove.txt format:
# literal:ActualSecretValue==>REMOVED

# Option B: BFG Repo Cleaner
java -jar bfg.jar --replace-text secrets-to-remove.txt

# After either option — force push requires security team approval
git push --force-with-lease

Warning: Rewriting git history requires coordination with all team members. All clones must be re-created after a force push.

---

Never Commit Secrets — Use KV References

All application secrets must be stored in Azure Key Vault and referenced via the keyvaultref: pattern in services-config.json:

// WRONG — never do this
"ConnectionStrings__DefaultConnection": "Server=...;Password=MyActualPassword"

// CORRECT — KV reference
"ConnectionStrings__DefaultConnection": "keyvaultref:https://mic-erp-be-dev-skv.vault.azure.net/secrets/ConnectionStrings--DefaultConnection"

See config-management.md for the full KV reference pattern.