Pipeline Stages

All 16 DevSecOps stages are defined in DevSecOps/templates/security-pipeline-template.yml. This page describes each stage in detail: the tool used, its purpose, and the conditions that block the pipeline.

Stage Summary Table

Stage	Name	Tool	Purpose	Blocks on
1	Secret Scan	Gitleaks	Detect committed secrets	Any secret found
2	SAST	SonarCloud	Static code analysis	Quality gate fail
3	Dependency CVE	OWASP DC	Known CVE in packages	Critical / High CVE (CVSS ≥ 7)
4	Dependency Vuln	Trivy (deps)	Vulnerability scan on lockfiles	Critical
5	IaC Scan	Trivy (IaC)	Bicep / YAML misconfiguration	High
6	Dockerfile Lint	Hadolint	Dockerfile best-practice check	Error level
7	Build + Unit Tests	.NET / Node / Flutter	Compile and run unit tests	Any test failure
8	Docker Build	Docker	Container image build	Build failure
9	Image Scan	Trivy (image)	Scan built container image	Critical CVE
10	SBOM	Syft	Generate software bill of materials	None (informational)
11	Integration Tests	Newman / REST client	End-to-end API test suite	Any test failure
12	AI Code Review	Azure OpenAI GPT-4o	Automated code suggestions	None (advisory)
13	DAST	OWASP ZAP	Dynamic attack surface test	High findings
14	Report	HTML/PDF generator	Publish security report	N/A
15	Notify	Teams webhook	Alert team on results	N/A
16	Archive	Azure Blob Storage	Store artifacts and reports	N/A

Stage 1 — Secret Scanning (Gitleaks)

Tool: GitleaksWhen: First stage; runs before any build activity.

Gitleaks scans the entire git history of the repository for secrets: API keys, passwords, connection strings, tokens, certificates, and Microtec-specific patterns (e.g., XApiKey).

yaml

- task: Bash@3
  displayName: 'Stage 1 - Secret Scan (Gitleaks)'
  inputs:
    script: |
      docker run --rm \
        -v $(Build.SourcesDirectory):/repo \
        zricethezav/gitleaks:latest \
        detect --source /repo --verbose \
        --config /repo/.gitleaks.toml \
        --redact

Blocks on: Any secret detected. Exit code non-zero fails the stage.

See gitleaks.md for configuration details.

Stage 2 — SAST (SonarCloud)

Tool: SonarCloudWhen: After secret scan, before build.

SonarCloud performs static application security testing across C#, TypeScript, Java, Python, and Dart. It evaluates code quality gates covering coverage, duplication, security hotspots, and code smells.

yaml

- task: SonarCloudPrepare@1
  displayName: 'Stage 2 - SAST Prepare (SonarCloud)'
  inputs:
    SonarCloud: 'SonarCloud-ServiceConnection'
    organization: 'microtec'
    scannerMode: 'MSBuild'    # or CLI for non-.NET
    projectKey: '$(sonarProjectKey)'
    extraProperties: |
      sonar.exclusions=**/obj/**,**/bin/**,**/*.spec.ts
      sonar.coverage.exclusions=**/*Tests*/**

- task: SonarCloudAnalyze@1
  displayName: 'Stage 2 - SAST Analyze'

- task: SonarCloudPublish@1
  displayName: 'Stage 2 - SAST Publish'
  inputs:
    pollingTimeoutSec: '300'

Blocks on: Quality gate FAILED status returned from SonarCloud API.

See sonarcloud.md for quality gate configuration.

Stage 3 — Dependency CVE Check (OWASP Dependency-Check)

Tool: OWASP Dependency-CheckWhen: After SAST.

OWASP DC downloads the NVD (National Vulnerability Database) and matches all project dependencies against known CVEs. Covers NuGet, npm, and Maven package graphs.

yaml

- task: dependency-check-build-task@6
  displayName: 'Stage 3 - Dependency CVE (OWASP DC)'
  inputs:
    projectName: '$(Build.Repository.Name)'
    scanPath: '$(Build.SourcesDirectory)'
    format: 'HTML,JSON'
    failOnCVSS: '7'           # Block on High (7) and above
    suppressionPath: '$(Build.SourcesDirectory)/.owasp-suppression.xml'

Blocks on: Any dependency with CVSS score ≥ 7 (High or Critical).

Tip: False positives can be suppressed in .owasp-suppression.xml with justification and expiry date.

Stage 4 — Dependency Vulnerabilities (Trivy deps)

Tool: Trivy — filesystem mode When: Parallel with Stage 3 (if pipeline allows parallelism).

Trivy scans dependency lockfiles (packages.lock.json, package-lock.json, pubspec.lock) directly without downloading the NVD. Faster than OWASP DC and useful as a second opinion.

yaml

- task: Bash@3
  displayName: 'Stage 4 - Trivy Dependency Scan'
  inputs:
    script: |
      trivy fs \
        --scanners vuln \
        --severity CRITICAL \
        --exit-code 1 \
        --format table \
        $(Build.SourcesDirectory)

Blocks on: Critical severity vulnerabilities only.

Stage 5 — IaC Scanning (Trivy IaC)

Tool: Trivy — config mode When: After dependency scans.

Scans Bicep templates and Kubernetes/Docker Compose YAML files for misconfigurations: open ports, missing resource limits, insecure defaults.

yaml

- task: Bash@3
  displayName: 'Stage 5 - IaC Scan (Trivy)'
  inputs:
    script: |
      trivy config \
        --severity HIGH,CRITICAL \
        --exit-code 1 \
        --format table \
        $(Build.SourcesDirectory)/Devops/azure/infrastructure

Blocks on: High or Critical misconfigurations in Bicep / YAML.

See trivy.md for .trivyignore usage.

Stage 6 — Dockerfile Linting (Hadolint)

Tool: HadolintWhen: Before Docker build.

Hadolint validates Dockerfiles against best practices: COPY vs ADD, apt-get patterns, USER declarations, and security rules (e.g., avoid latest tag, avoid --no-check-certificate).

yaml

- task: Bash@3
  displayName: 'Stage 6 - Dockerfile Lint (Hadolint)'
  inputs:
    script: |
      hadolint \
        --failure-threshold error \
        --config .hadolint.yaml \
        $(dockerfilePath)

Blocks on: Any finding at error severity or above.

Stage 7 — Build + Unit Tests

Tool: dotnet test, npm test, flutter testWhen: After all pre-build scans pass.

Compiles the application and runs the full unit test suite. Test results are published as JUnit XML artifacts.

yaml

# .NET example
- task: DotNetCoreCLI@2
  displayName: 'Stage 7 - .NET Test'
  inputs:
    command: 'test'
    projects: '$(testProject)'
    arguments: >
      --configuration Release
      --collect:"XPlat Code Coverage"
      --results-directory $(Agent.TempDirectory)/TestResults
    publishTestResults: true

Blocks on: Any unit test failure; code coverage below the SonarCloud quality gate threshold.

Stage 8 — Docker Build

Tool: Docker When: After tests pass.

Builds the container image using the service Dockerfile. Uses --mount=type=secret,id=nuget_pat for NuGet authentication (see NuGet Setup docs).

yaml

- task: Docker@2
  displayName: 'Stage 8 - Docker Build'
  inputs:
    containerRegistry: '$(acrServiceConnection)'
    repository: '$(imageName)'
    command: 'build'
    Dockerfile: '$(dockerfilePath)'
    buildContext: '$(Build.SourcesDirectory)'
    tags: '$(Build.BuildId)'
    arguments: '--secret id=nuget_pat,src=$(NUGET_PAT_FILE)'

Blocks on: Docker build exit code non-zero.

Stage 9 — Container Image Scan (Trivy image)

Tool: Trivy — image mode When: Immediately after Docker build; before push.

Scans the freshly built image for OS-level CVEs and application-layer vulnerabilities in the base image layers.

yaml

- task: Bash@3
  displayName: 'Stage 9 - Image Scan (Trivy)'
  inputs:
    script: |
      trivy image \
        --severity CRITICAL \
        --exit-code 1 \
        --ignore-unfixed \
        --format table \
        $(imageName):$(Build.BuildId)

Blocks on: Critical CVEs in the container image.

Stage 10 — SBOM Generation (Syft)

Tool: SyftWhen: After image scan.

Generates a Software Bill of Materials in SPDX and CycloneDX formats. The SBOM is published as a pipeline artifact and archived in the security storage account. This stage is informational and never blocks the pipeline.

yaml

- task: Bash@3
  displayName: 'Stage 10 - SBOM (Syft)'
  inputs:
    script: |
      syft $(imageName):$(Build.BuildId) \
        -o spdx-json=$(Build.ArtifactStagingDirectory)/sbom.spdx.json \
        -o cyclonedx-json=$(Build.ArtifactStagingDirectory)/sbom.cyclonedx.json

Blocks on: Nothing. Always informational.

Stage 11 — Integration Tests

Tool: Newman (Postman CLI) or REST Assured When: After image is pushed to ACR and deployed to the ephemeral dev environment.

Runs the full API test collection against the deployed service. Covers happy-path, error-path, authentication, and authorization scenarios.

yaml

- task: Bash@3
  displayName: 'Stage 11 - Integration Tests (Newman)'
  inputs:
    script: |
      newman run $(Build.SourcesDirectory)/tests/postman-collection.json \
        --environment $(Build.SourcesDirectory)/tests/env-dev.json \
        --reporters cli,junit \
        --reporter-junit-export $(Agent.TempDirectory)/integration-results.xml

Blocks on: Any test failure in the collection.

Stage 12 — AI Code Review (Azure OpenAI)

Tool: Azure OpenAI GPT-4o When: After integration tests.

The pipeline sends the diff (git diff origin/main...HEAD) to Azure OpenAI for a security-focused code review. The response is published as a PR comment and as a pipeline artifact. This stage is advisory only and does not block the pipeline.

yaml

- task: Bash@3
  displayName: 'Stage 12 - AI Code Review'
  inputs:
    script: |
      DIFF=$(git diff origin/main...HEAD -- '*.cs' '*.ts' '*.dart')
      REVIEW=$(python scripts/ai-review.py "$DIFF")
      echo "##vso[task.setvariable variable=aiReview]$REVIEW"

Blocks on: Nothing. Advisory output only.

Stage 13 — DAST (OWASP ZAP)

Tool: OWASP ZAPWhen: Against the deployed dev environment after integration tests pass.

ZAP performs an active scan against the running application: authentication bypass attempts, injection attacks, XSS probes, CSRF checks, and API fuzzing.

yaml

- task: Bash@3
  displayName: 'Stage 13 - DAST (OWASP ZAP)'
  inputs:
    script: |
      docker run --rm \
        -v $(Agent.TempDirectory)/zap:/zap/wrk \
        ghcr.io/zaproxy/zaproxy:stable \
        zap-api-scan.py \
          -t $(appBaseUrl)/swagger/v1/swagger.json \
          -f openapi \
          -r zap-report.html \
          -x zap-report.xml \
          -z "-config replacer.full_list(0).description=auth \
              -config replacer.full_list(0).enabled=true \
              -config replacer.full_list(0).matchtype=REQ_HEADER \
              -config replacer.full_list(0).matchstr=Authorization \
              -config replacer.full_list(0).replacement=Bearer\ $(testJwt)"
      
      # Fail on high-risk findings (riskCode 3)
      python scripts/zap-check.py --report zap-report.xml --max-risk 3

Blocks on: Any finding with risk code 3 (High) or above.

Stage 14 — Security Report

Tool: Custom HTML/PDF generator (Python + WeasyPrint) When: After DAST, regardless of previous stage outcomes.

Aggregates results from all previous stages into a single HTML security report and converts it to PDF. Published as a pipeline artifact and linked in the PR description.

Blocks on: Nothing (runs in always() condition).

Stage 15 — Notifications

Tool: Azure DevOps Teams webhook / service hook When: After report generation.

Sends a summary card to the designated Microsoft Teams channel:

Pass/fail status per stage
Links to detailed reports
Assignee for any blocking findings

Blocks on: Nothing.

Stage 16 — Archive

Tool: Azure Blob Storage (az storage blob upload-batch) When: Last stage, always runs.

Uploads all security artifacts to the long-term security archive storage account:

SBOM files (SPDX + CycloneDX)
ZAP HTML and XML reports
OWASP DC HTML and JSON reports
SonarCloud quality gate JSON
AI review output

Artifacts are stored under {year}/{month}/{repoName}/{buildId}/.

Blocks on: Nothing.

Stage Dependencies (Simplified DAG)

1 (Gitleaks)
    └── 2 (SonarCloud)
    └── 3 (OWASP DC) ─┐
    └── 4 (Trivy deps) ┤
    └── 5 (Trivy IaC)  ┤
    └── 6 (Hadolint) ──┘
                        └── 7 (Build + Tests)
                                └── 8 (Docker build)
                                        └── 9 (Trivy image)
                                                └── 10 (SBOM)
                                                └── 11 (Integration tests)
                                                        └── 12 (AI review)
                                                        └── 13 (ZAP)
                                                                └── 14 (Report)
                                                                └── 15 (Notify)
                                                                └── 16 (Archive)

Pipeline Stages ​

Stage Summary Table ​

Stage 1 — Secret Scanning (Gitleaks) ​

Stage 2 — SAST (SonarCloud) ​

Stage 3 — Dependency CVE Check (OWASP Dependency-Check) ​

Stage 4 — Dependency Vulnerabilities (Trivy deps) ​

Stage 5 — IaC Scanning (Trivy IaC) ​

Stage 6 — Dockerfile Linting (Hadolint) ​

Stage 7 — Build + Unit Tests ​

Stage 8 — Docker Build ​

Stage 9 — Container Image Scan (Trivy image) ​

Stage 10 — SBOM Generation (Syft) ​

Stage 11 — Integration Tests ​

Stage 12 — AI Code Review (Azure OpenAI) ​

Stage 13 — DAST (OWASP ZAP) ​

Stage 14 — Security Report ​

Stage 15 — Notifications ​

Stage 16 — Archive ​

Stage Dependencies (Simplified DAG) ​

Pipeline Stages

Stage Summary Table

Stage 1 — Secret Scanning (Gitleaks)

Stage 2 — SAST (SonarCloud)

Stage 3 — Dependency CVE Check (OWASP Dependency-Check)

Stage 4 — Dependency Vulnerabilities (Trivy deps)

Stage 5 — IaC Scanning (Trivy IaC)

Stage 6 — Dockerfile Linting (Hadolint)

Stage 7 — Build + Unit Tests

Stage 8 — Docker Build

Stage 9 — Container Image Scan (Trivy image)

Stage 10 — SBOM Generation (Syft)

Stage 11 — Integration Tests

Stage 12 — AI Code Review (Azure OpenAI)

Stage 13 — DAST (OWASP ZAP)

Stage 14 — Security Report

Stage 15 — Notifications

Stage 16 — Archive

Stage Dependencies (Simplified DAG)