ADR-007: Centralized DevSecOps Pipeline

Status

Accepted

Date

2023-Q3

---

Context

As the Microtec ERP platform grew to 14+ microservices and 10 frontend applications, each team maintained its own CI/CD pipeline YAML. This created:

• Security inconsistency: Some pipelines ran SAST scans; others did not. Secret scanning was optional.
• Duplication: 80% of every pipeline YAML was boilerplate (checkout, NuGet restore, Docker build, ACA deploy). Changes had to be replicated across 20+ pipeline files manually.
• Compliance drift: Security tooling (Trivy, SonarCloud, Gitleaks) was at different versions across pipelines.
• Audit gaps: There was no single place to verify that every deployment went through a mandatory security gate.

The team evaluated two approaches:

Option	Description	Concerns
Template per team	Each team owns their pipeline; central templates for common steps	Teams may skip steps; no enforcement mechanism
Centralized 16-stage pipeline	One master pipeline template; all repos must use it	More upfront work; single point of change

---

Decision

Build and mandate a centralized 16-stage DevSecOps pipeline template in Azure DevOps.

Every service (backend and frontend) that is deployed to any Microtec environment must use this template. Teams cannot opt out of any stage.

The 16 Stages

Stage	Tool	Blocks on failure?
1. Secret scan	Gitleaks	Yes — hard block
2. Dependency audit	OWASP Dependency-Check	Yes (CVSS ≥ 7.0)
3. SAST	SonarCloud (Quality Gate)	Yes
4. License check	FOSSA / manual allowlist	Yes (GPL detected)
5. Unit tests	dotnet test / ng test	Yes
6. Coverage gate	80% minimum	Yes
7. Build artifact	dotnet publish / ng build	Yes
8. Docker build	docker build	Yes
9. Container scan	Trivy	Yes (CRITICAL CVE)
10. Push to ACR	docker push	Yes
11. Deploy dev	az containerapp update	Yes
12. Integration tests	Postman / Newman	Warning only
13. DAST scan	OWASP ZAP (baseline)	Warning only
14. Deploy stage	az containerapp update	Yes
15. Smoke tests	curl / health checks	Yes
16. Approval gate	Azure DevOps manual approval	Required for prod

Toolchain Selection Rationale

Tool	Selected For	Alternatives Considered
Gitleaks	OSS, fast, regex-based, 800+ built-in rules	TruffleHog (slower), git-secrets (fewer rules)
SonarCloud	Deep .NET + Angular integration, PR decoration	SonarQube self-hosted (infra overhead), Semgrep (weaker .NET)
Trivy	Multi-target (image + IaC + filesystem), fast, free	Anchore Grype (similar), Snyk (cost at scale)
OWASP ZAP	Industry standard DAST, Azure DevOps task available	Burp Suite (cost), Nikto (limited)
OWASP Dependency-Check	Java/NuGet/npm support, integrates with ADO	Dependabot (GitHub-only at time of decision)

---

Consequences

Positive

• Audit trail: Every deployment to every environment passes through all 16 stages. The pipeline run log is the audit record.
• Consistency: All 24+ services use identical security tooling at the same version. Tooling updates propagate to all services by updating one template file.
• Secret leak prevention: Gitleaks blocks the pipeline at Stage 1 before any code is built or deployed — secrets cannot reach any environment.
• Compliance evidence: The Stage 3 SonarCloud gate and Stage 9 Trivy scan produce exportable reports for audit purposes.
• Reduced pipeline YAML per service: Each service's pipeline YAML is ~30 lines (trigger + template reference); the template handles everything else.

Negative

• Longer pipeline runtime: The full 16 stages take 25–40 minutes end-to-end. Fast feedback requires running only the first 6 stages for PRs.
• Template ownership: The central template is a single point of failure. A breaking change in the template affects all 24+ services simultaneously. Mitigated by the template being versioned and tested before rollout.
• False positives from DAST: ZAP baseline scan generates noise on endpoints that intentionally return 4xx. Custom suppressions must be maintained per service.
• Coverage gate friction: New services bootstrapped during early development cannot realistically reach 80% coverage immediately. A coverageOverride parameter exists for bootstrapping but requires team lead approval.

Neutral

• Stage 12 (integration tests) and Stage 13 (DAST) are Warning rather than hard failures because their false positive rates make them unsuitable for hard gates at this time. This is reviewed quarterly.

---

Implementation Notes

Template Location

Devops/azure/templates/devsecops/
├── pipeline-template.yml      ← Master 16-stage template
├── stages/
│   ├── 01-secret-scan.yml
│   ├── 02-dependency-audit.yml
│   ├── 03-sonarcloud.yml
│   ├── 04-license-check.yml
│   ├── 05-unit-tests.yml
│   ├── 06-coverage-gate.yml
│   ├── 07-build-artifact.yml
│   ├── 08-docker-build.yml
│   ├── 09-trivy-scan.yml
│   ├── 10-push-acr.yml
│   ├── 11-deploy-dev.yml
│   ├── 12-integration-tests.yml
│   ├── 13-dast-zap.yml
│   ├── 14-deploy-stage.yml
│   ├── 15-smoke-tests.yml
│   └── 16-approval-gate.yml
└── configs/
    ├── gitleaks.toml
    ├── trivy.yaml
    └── zap-baseline.conf

Service Pipeline Entrypoint

Each service references the template with minimal configuration:

# Devops/azure/pipelines/my-service/deploy-my-service.yml
trigger:
  branches: { include: [main, stage, PreProd, production] }
  paths: { include: [Platforms/Src/AppsPortal/MyService/**] }

extends:
  template: ../../templates/devsecops/pipeline-template.yml
  parameters:
    serviceName: my-service
    dockerfilePath: Platforms/Src/AppsPortal/MyService/MyService.Apis/Dockerfile
    testProject: Platforms/Src/AppsPortal/MyService/MyService.Tests/MyService.Tests.csproj
    sonarProjectKey: microtec_my-service

Onboarding a New Repository

See the Runbook: Onboard Repo to DevSecOps for the step-by-step process.

---

Related ADRs

• ADR-001: Microservices Architecture (the pipeline operates per-service)
• ADR-002: Azure Container Apps (Stage 11 and Stage 14 deploy targets)