DevOps Technology Reference

Detailed reference for every infrastructure, CI/CD, and DevOps tool used in the Microtec ERP platform.

---

CI/CD Platform

Azure DevOps

Role: Central hub for source control, CI/CD pipelines, work tracking, and NuGet packaging
Organisation: https://dev.azure.com/microtec
Projects: ERP

Key capabilities used:

Feature	Usage
Repos	All source code (Git)
Pipelines	Build and deploy automation
Artifacts	Private NuGet feed (Microtec)
Boards	Sprint planning, work items
Test Plans	Test execution tracking

Pipeline structure:

Devops/azure/
├── pipelines/                    # Per-service and per-app pipeline YAML files
│   ├── containerBackend/         # Backend service pipelines
│   └── frontApps/                # Frontend app pipelines
├── templates/                    # Reusable pipeline templates
│   ├── containerBackend/
│   │   ├── build/build-docker.yml
│   │   └── deploy/deploy-container.yml
│   └── frontApps/
│       └── deploy/unified-frontend-pipeline.yml
└── config/
    └── container-backend/
        └── services-config.json  # Master service configuration

Branch → environment mapping:

Branch pattern	Environment
main, master, production	Production
stage, staging	Stage
PreProd, preprod	Pre-production
Any other branch	Dev

---

Infrastructure as Code

Azure Bicep

Role: Declarative IaC DSL for all Azure infrastructure provisioning
Scope: Subscription-scoped (main.bicep) — creates all resource groups and resources
Location: Devops/azure/infrastructure/

Key templates:

File	Purpose
main.bicep	Subscription entrypoint — orchestrates all modules
modules/containerEnv.bicep	Container Apps Environment (CAE)
modules/containerApp.bicep	Individual Container App
modules/keyVault.bicep	Key Vault with RBAC
modules/serviceBus.bicep	Service Bus namespace and queues
modules/redis.bicep	Azure Managed Redis
modules/staticWebApp.bicep	Frontend Static Web App

Deploy command:

# Deploy full infrastructure for dev environment
az deployment sub create \
  --location "uksouth" \
  --template-file Devops/azure/infrastructure/main.bicep \
  --parameters @Devops/azure/infrastructure/parameters/dev.bicepparam

Parameters are generated from services-config.json via:

# Build-BicepParams.ps1 reads services-config.json and outputs .bicepparam files
Devops/azure/scripts/infra/Build-BicepParams.ps1 -Environment dev

---

Containerisation

Docker

Version: 25.x
Role: Package every backend service as a portable container image
Key Dockerfile pattern — multi-stage with NuGet secret:

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS restore
# Stage 1: Restore with private NuGet feed (secret never baked into image)
RUN --mount=type=secret,id=nuget_pat \
    NUGET_PAT=$(cat /run/secrets/nuget_pat) \
    dotnet restore ...

FROM restore AS build
RUN dotnet build ...

FROM build AS publish
RUN dotnet publish -o /app/publish ...

FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS final
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "MyService.dll"]

Base images used:

Image	Used For
mcr.microsoft.com/dotnet/aspnet:8.0	Backend service runtime
mcr.microsoft.com/dotnet/sdk:8.0	Build stage (multi-stage Dockerfile)
nginx:1.25-alpine	Frontend static file serving
bitnami/keycloak:22	Keycloak identity provider

---

Container Runtime

Azure Container Apps (ACA)

Role: Managed serverless container runtime for all services
Features used:

Feature	Configuration
KEDA autoscaling	HTTP + CPU + cron triggers
Managed identity	ACR image pull, Key Vault access
VNet integration	Private CAE on customer VNet
mTLS	Enabled on Private CAE
Dapr	Not used
Revisions	Traffic splitting for blue/green deployments

CAE topology:

Public CAE  (internet-accessible):
  ├── Gateway.API (YARP)
  └── Keycloak SSO

Private CAE (VNet-internal, mTLS):
  ├── AppsPortal.Apis
  ├── Inventory.Apis
  ├── BusinessOwners.Apis
  ├── HR.Apis
  ├── Attachment.Apis
  ├── Notification.Apis
  ├── Workflows.Apis
  ├── Template.Apis
  ├── Integration.Apis
  └── Platforms.Worker

VNet CIDR allocation:

Environment	VNet CIDR	CAE Subnet
dev	10.0.0.0/16	10.0.0.0/23
stage	10.1.0.0/16	10.1.0.0/23
preprod	10.6.0.0/16	10.6.0.0/23
uat	10.5.0.0/16	10.5.0.0/23
production	10.2.0.0/16	10.2.0.0/23
shared-sql	10.100.0.0/16	—

---

Container Registry

Azure Container Registry (ACR)

Role: Private Docker image registry
Authentication: Managed identity (pipeline pulls via $(System.AccessToken))
Naming formula: replace('mic-erp-be-{env}acr', '-', '') → e.g. micerpbedevacr

ACR names per environment:

Environment	Backend ACR	Frontend ACR
dev	micerpbedevacr	micerpfrdevacr
stage	micerpbestageacr	micerpfrstageacr
preprod	micerpbepreprodacr	micerpfrpreprodacr
uat	micerpbeuatacr	micerpfruatacr
production	micerpbeproductionacr	micerpfrproductionacr

Image tag strategy:

Tag	Usage
$(Build.BuildId)	Every pipeline build (unique, traceable)
latest	Overwritten on every successful dev build
stable	Set manually after prod validation

---

Global Load Balancing

Azure Front Door (Standard)

Role: Global CDN, WAF, and load balancer in front of all internet-facing endpoints
Features used:

• Custom domain routing per environment
• TLS termination (auto-managed certificates)
• WAF policies (OWASP Core Rule Set)
• Origin groups for backend (Gateway CAE) and frontend (Static Web Apps)
• Health probe every 30 seconds

Traffic flow:

User → Azure Front Door
  ├── /api/*          → Public CAE → Gateway.API → Private CAE services
  ├── /auth/*         → Public CAE → Keycloak SSO
  └── /* (all other)  → Azure Static Web Apps → Angular apps

---

Secrets Management

Azure Key Vault

Role: Central secrets store — all credentials, connection strings, API keys
Authentication: Managed identity (Container Apps use system-assigned MI)
KV reference pattern: keyvaultref:{KV_URI}/secrets/{secret-name}

Secret naming convention:

Nested config keys use double-dash (--) instead of colon (:)
Example: ConnectionStrings:DefaultConnection → ConnectionStrings--DefaultConnection

Key Vault names:

Environment	Key Vault
dev	mic-erp-be-dev-skv
stage	mic-erp-stg-kv
preprod	mic-erp-be-preprod-skv
uat	mic-erp-uat-kv

[WARNING] KV names are inconsistent across environments — use the exact names above, not a formula.

---

Async Messaging

Azure Service Bus (Standard)

Role: Production message broker for all async inter-service communication
Tier: Standard (supports topics, subscriptions, dead-letter queues)
MassTransit transport: Configured automatically in cloud environments

Queue/topic naming:

Queue/Topic	Purpose
erp-events	General domain events
notification-requests	Email/SMS/push notifications
zatca-submissions	Saudi e-invoicing submissions
import-jobs	Data import processing

RabbitMQ (Local Dev)

Version: 3.13.x (Docker)
Role: MassTransit transport for local development
Management UI: http://localhost:15672 (guest/guest)
Seamless swap: Same MassTransit code works with both RabbitMQ and ASB transports

---

Caching

Azure Managed Redis

Role: Distributed cache, session storage, rate limiting
Tier: C1 Standard (1 GB, non-prod) / C2 Standard (6 GB, prod)
TLS: Enabled (port 10000)
Auth: Password in Key Vault RedisConfiguration--Password

Usage patterns:

// Output caching (per-tenant response cache)
[OutputCache(Duration = 300, VaryByQueryKeys = new[] { "tenantId" })]
public async Task<IActionResult> GetDropdowns() { ... }

// Distributed cache (manual)
var cached = await cache.GetStringAsync($"tenant:{tenantId}:config");
if (cached is null)
{
    var config = await LoadConfig(tenantId);
    await cache.SetStringAsync($"tenant:{tenantId}:config",
        JsonSerializer.Serialize(config),
        new DistributedCacheEntryOptions { AbsoluteExpirationRelativeToNow = TimeSpan.FromHours(1) });
}

---

Autoscaling

KEDA (Kubernetes Event-Driven Autoscaling)

Role: Event-driven autoscaling built into Azure Container Apps
Trigger types used:

Trigger	Metric	Typical config
http	Concurrent requests per replica	concurrentRequests: 100
cpu	CPU utilisation %	value: 70
memory	Memory utilisation %	value: 75
cron	Time-based warm replicas	Business hours pre-warm

Configuration lives in services-config.json → triggers[].

See Runbook: Scale a Service for full configuration reference.

---

Monitoring

Application Insights

Role: APM — request traces, exceptions, performance metrics, dependency tracking
Integration: Auto-configured via OpenTelemetry → Azure Monitor exporter
Correlation: All requests tagged with TraceId for distributed tracing across services

Seq

Role: Structured log viewer (dev and stage)
On-prem: eg-sv-ai at port 8095 equivalent / Docker at port 1234 locally
Integration: Serilog writes to Seq via OTLP sink

---

Code Quality

SonarQube

Version: 10.x
Location: eg-sv-ai (192.168.120.254:8095) — admin/Admin@2024
Integration: ADO pipeline sonar-analysis template runs on every PR
Quality gate: Must pass before merge to main

---

On-Prem Infrastructure

NGINX (eg-sv-vip)

Role: Reverse proxy for on-prem environment; handles SSL termination and routing
Admin: NPM (Nginx Proxy Manager) at http://192.168.120.12:8100
Access: mahmoudaraby36@gmail.com / Admin@1234

Docker Registry (eg-sv-01)

Role: Self-hosted Docker registry for on-prem container images
Port: 5000
External domain: registery.microtec-develop.com
Management: Portainer CE at http://192.168.120.233:9000 or https://192.168.120.233:9443

Build Agents

Three build agent VMs that run Azure DevOps pipeline jobs:

Agent	IP	OS
eg-build-01	192.168.120.88	Ubuntu 22.04
eg-build-02	192.168.120.122	Ubuntu 22.04
eg-build-05	192.168.120.44	Ubuntu 22.04

All agents have Azure CLI 2.85.0, Docker, .NET 8 SDK, and Node.js 18 installed.

---

Related Documentation

• Backend Technology Reference
• Technology Index
• CI/CD Pipelines
• Environments Overview
• Networking
• Runbook: Scale a Service
• Runbook: Key Rotation