CI/CD Guidelines

This document defines CI/CD guidelines and practices for ConnectSoft systems. It is written for engineers and DevOps teams understanding how we build, test, and deploy software.

ConnectSoft uses Azure DevOps pipelines for CI/CD. All services have automated pipelines that build, test, and deploy code. The Factory generates pipelines automatically, but humans review and customize them as needed.

Tip

Pipelines are code—version them, review them, and test them. Don't treat pipelines as afterthoughts. They're critical infrastructure that ensures quality and enables fast, safe deployments.

Pipeline Structure

Standard Pipeline Stages

Stages:

1. Build - Compile code, restore packages
2. Test - Run unit and integration tests
3. Package - Create deployment artifacts
4. Deploy (Dev) - Deploy to dev environment
5. Deploy (Staging) - Deploy to staging (after approval)
6. Deploy (Production) - Deploy to production (after approval)

Pipeline Flow:

flowchart LR
    BUILD[Build] --> TEST[Test]
    TEST -->|Pass| PACKAGE[Package]
    TEST -->|Fail| STOP[Stop]
    PACKAGE --> DEV[Deploy Dev]
    DEV -->|Auto| STAGE[Deploy Staging<br/>Manual Approval]
    STAGE -->|Approve| PROD[Deploy Production<br/>Manual Approval]

    style BUILD fill:#2563EB,color:#fff
    style TEST fill:#10B981,color:#fff
    style PROD fill:#EF4444,color:#fff

Hold "Alt" / "Option" to enable pan & zoom

Pipeline Configuration

Location: pipelines/azure-pipelines.yml

Structure:

trigger:
  branches:
    include:
      - main
      - develop
  paths:
    exclude:
      - docs/**

stages:
  - stage: Build
    jobs:
      - job: Build
        steps:
          - task: DotNetCoreCLI@2
            inputs:
              command: build

  - stage: Test
    jobs:
      - job: UnitTests
        steps:
          - task: DotNetCoreCLI@2
            inputs:
              command: test

Build, Test, and Quality Gates

Build Stage

Steps:

1. Restore Packages - Restore NuGet packages
2. Build - Compile code
3. Publish Artifacts - Publish build artifacts

Quality Gates:

• Build succeeds - Code must compile without errors
• No warnings - Treat warnings as errors (optional, recommended)

Test Stage

Steps:

1. Run Unit Tests - Execute unit tests
2. Run Integration Tests - Execute integration tests
3. Code Coverage - Measure and report code coverage

Quality Gates:

• All tests pass - No failing tests
• Code coverage ≥ 80% - Minimum coverage threshold
• No flaky tests - Tests must be reliable

Static Analysis

Tools:

• Security scanning - Check for security vulnerabilities
• Code quality - Linting, complexity analysis
• Dependency scanning - Check for vulnerable dependencies

Quality Gates:

• No critical issues - Critical security issues block deployment
• No high-severity issues - High-severity issues require review

Deployment Patterns

Blue/Green Deployment

How It Works:

1. Deploy new version to "green" environment (parallel to "blue")
2. Test green environment
3. Switch traffic from blue to green
4. Keep blue as rollback option

When to Use:

• Zero-downtime deployments
• Easy rollback
• Production deployments

Canary Deployment

How It Works:

1. Deploy new version to small percentage of traffic (e.g., 10%)
2. Monitor metrics and errors
3. Gradually increase traffic (10% → 50% → 100%)
4. Rollback if issues detected

When to Use:

• Risk mitigation
• Gradual rollout
• A/B testing

Staged Environments

How It Works:

1. Deploy to dev (automatic)
2. Deploy to staging (manual approval)
3. Deploy to production (manual approval)

When to Use:

• Most common pattern
• Simple and reliable
• Good for most services

Example:

stages:
  - stage: DeployDev
    jobs:
      - deployment: Deploy
        environment: dev
        strategy:
          runOnce:
            deploy:
              steps:
                - script: deploy.sh dev

  - stage: DeployStaging
    dependsOn: DeployDev
    condition: succeeded()
    jobs:
      - deployment: Deploy
        environment: staging
        strategy:
          runOnce:
            deploy:
              steps:
                - script: deploy.sh staging

  - stage: DeployProduction
    dependsOn: DeployStaging
    condition: succeeded()
    jobs:
      - deployment: Deploy
        environment: production
        strategy:
          runOnce:
            deploy:
              steps:
                - script: deploy.sh production

Rollback and Hotfixes

Rollback Process

Automatic Rollback:

• Health check failures - Automatic rollback if health checks fail
• Error rate threshold - Rollback if error rate exceeds threshold
• Latency threshold - Rollback if latency exceeds threshold

Manual Rollback:

1. Identify issue - Detect problem in production
2. Decide rollback - Determine if rollback is needed
3. Execute rollback - Revert to previous version
4. Verify - Confirm rollback resolved issue
5. Postmortem - Document what went wrong

Rollback Methods:

• Redeploy previous version - Deploy previous artifact
• Database rollback - Rollback database migrations (if applicable)
• Feature flags - Disable feature via feature flags (if applicable)

Hotfix Process

When to Use:

• Critical production bugs
• Security vulnerabilities
• Data corruption risks

Process:

1. Create hotfix branch - Branch from main
2. Fix issue - Fix the bug
3. Add test - Add test to prevent regression
4. Fast-track review - Expedited code review
5. Deploy - Deploy directly to production (bypass staging if urgent)
6. Merge to main - Merge hotfix back to main

Example:

# Create hotfix branch
git checkout main
git pull
git checkout -b hotfix/security-patch

# Fix and commit
git commit -m "Fix security vulnerability"

# Create PR and fast-track
# Deploy after approval

# Merge to main
git checkout main
git merge hotfix/security-patch

Factory and Pipelines

How Factory Generates Pipelines

Factory Process:

1. Analyze service - Understand service structure and dependencies
2. Generate pipeline - Create Azure DevOps YAML pipeline
3. Configure stages - Set up build, test, deploy stages
4. Add quality gates - Include tests, coverage, security checks
5. Create pipeline file - Commit pipeline to repository

Generated Pipeline Includes:

• Build stage (compile, restore packages)
• Test stage (unit, integration tests)
• Code coverage reporting
• Deployment stages (dev, staging, production)
• Quality gates (tests, coverage, security)

Customizing Factory-Generated Pipelines

When to Customize:

• Custom deployment steps - Add service-specific deployment logic
• Additional tests - Add performance or E2E tests
• Custom quality gates - Add service-specific checks
• Environment-specific config - Customize per environment

How to Customize:

• Edit pipeline file - Modify azure-pipelines.yml directly
• Add pipeline templates - Use Azure DevOps pipeline templates
• Custom scripts - Add custom deployment scripts

Best Practices:

• Don't break structure - Keep Factory-generated structure
• Document customizations - Document why customizations were needed
• Version pipeline changes - Treat pipeline as code

Tip

Factory generates standard pipelines that work for most services. Customize only when necessary. Document customizations so future Factory regenerations don't overwrite them.

Related Documents

• Development Workflow - How CI/CD fits into workflow
• Testing Strategy - Testing practices in pipelines
• Branching Strategy - How branches trigger pipelines
• Factory Overview - How Factory generates pipelines