Skip to content

CI/CD Pipeline

Continuous Integration and Continuous Deployment pipeline design and implementation for the RCIIS DevOps platform.

Overview

The RCIIS platform uses GitHub Actions for CI/CD, integrated with ArgoCD for GitOps-based deployments, ensuring automated testing, building, and deployment workflows.

CI/CD Architecture

Pipeline Components

  1. Source Control: GitHub repositories
  2. CI Platform: GitHub Actions
  3. Container Registry: Harbor registry
  4. GitOps: ArgoCD for deployment
  5. Testing: Automated test suites
  6. Security: SAST/DAST scanning
  7. Monitoring: Deployment tracking

Workflow Stages

  1. Code Commit: Developer pushes to feature branch
  2. CI Pipeline: Build, test, and security scanning
  3. Pull Request: Code review and validation
  4. Merge: Automated chart version bump
  5. CD Pipeline: Image build and registry push
  6. GitOps Sync: ArgoCD deploys to environments
  7. Validation: Post-deployment testing

GitHub Actions Workflows

Release Workflow

# .github/workflows/release.yaml
name: Release Pipeline

on:
  push:
    branches: [master]
  pull_request:
    branches: [master]

env:
  REGISTRY: harbor.devops.africa
  IMAGE_NAME: rciis/nucleus

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4

    - name: Setup .NET
      uses: actions/setup-dotnet@v3
      with:
        dotnet-version: '8.0.x'

    - name: Restore dependencies
      run: dotnet restore

    - name: Build
      run: dotnet build --no-restore

    - name: Test
      run: dotnet test --no-build --verbosity normal --collect:"XPlat Code Coverage"

    - name: Upload coverage
      uses: codecov/codecov-action@v3

  security-scan:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4

    - name: Run Trivy vulnerability scanner
      uses: aquasecurity/trivy-action@master
      with:
        scan-type: 'fs'
        scan-ref: '.'
        format: 'sarif'
        output: 'trivy-results.sarif'

    - name: Upload Trivy scan results
      uses: github/codeql-action/upload-sarif@v2
      with:
        sarif_file: 'trivy-results.sarif'

  build-and-push:
    needs: [test, security-scan]
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/master'

    steps:
    - uses: actions/checkout@v4

    - name: Set up Docker Buildx
      uses: docker/setup-buildx-action@v3

    - name: Login to Harbor
      uses: docker/login-action@v3
      with:
        registry: ${{ env.REGISTRY }}
        username: ${{ secrets.HARBOR_USERNAME }}
        password: ${{ secrets.HARBOR_PASSWORD }}

    - name: Extract metadata
      id: meta
      uses: docker/metadata-action@v5
      with:
        images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
        tags: |
          type=ref,event=branch
          type=ref,event=pr
          type=sha,prefix={{branch}}-
          type=raw,value=latest,enable={{is_default_branch}}

    - name: Build and push
      uses: docker/build-push-action@v5
      with:
        context: .
        platforms: linux/amd64,linux/arm64
        push: true
        tags: ${{ steps.meta.outputs.tags }}
        labels: ${{ steps.meta.outputs.labels }}
        cache-from: type=gha
        cache-to: type=gha,mode=max

  update-chart:
    needs: build-and-push
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/master'

    steps:
    - uses: actions/checkout@v4
      with:
        token: ${{ secrets.MAGNABOT_GH_TOKEN }}

    - name: Bump chart version
      run: |
        NEW_VERSION="0.1.${{ github.run_number }}"
        yq ".version = \"$NEW_VERSION\"" -i charts/rciis/Chart.yaml
        echo "CHART_VERSION=$NEW_VERSION" >> $GITHUB_ENV

    - name: Package Helm chart
      run: |
        helm package charts/rciis/

    - name: Push chart to Harbor
      run: |
        helm registry login ${{ env.REGISTRY }} -u ${{ secrets.HARBOR_USERNAME }} -p ${{ secrets.HARBOR_PASSWORD }}
        helm push rciis-${{ env.CHART_VERSION }}.tgz oci://${{ env.REGISTRY }}/rciis

    - name: Commit version bump
      run: |
        git config --local user.email "[email protected]"
        git config --local user.name "GitHub Action"
        git add charts/rciis/Chart.yaml
        git commit -m "Bump Chart to ${{ env.CHART_VERSION }} [skip ci]"
        git push

  automated-api-tests:
    needs: update-chart
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/master'

    steps:
    - uses: actions/checkout@v4

    - name: Setup Node.js
      uses: actions/setup-node@v4
      with:
        node-version: '18'

    - name: Install Newman
      run: npm install -g newman newman-reporter-htmlextra

    - name: Wait for deployment
      run: sleep 300  # Wait for ArgoCD to sync

    - name: Run API tests
      run: |
        newman run tests/api/nucleus-api.postman_collection.json \
          --environment tests/api/staging.postman_environment.json \
          --reporters cli,htmlextra \
          --reporter-htmlextra-export test-results.html

    - name: Upload test results
      uses: actions/upload-artifact@v3
      with:
        name: api-test-results
        path: test-results.html

Feature Branch Workflow

# .github/workflows/feature.yaml
name: Feature Branch Pipeline

on:
  push:
    branches-ignore: [master]
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4

    - name: Setup .NET
      uses: actions/setup-dotnet@v3
      with:
        dotnet-version: '8.0.x'

    - name: Restore dependencies
      run: dotnet restore

    - name: Build
      run: dotnet build --no-restore

    - name: Test
      run: dotnet test --no-build --verbosity normal

    - name: Lint code
      run: dotnet format --verify-no-changes --verbosity diagnostic

    - name: Security scan
      uses: security-code-scan/security-code-scan-action@v1

  docker-build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4

    - name: Build Docker image
      run: |
        docker build -t test-image:${{ github.sha }} .
        docker run --rm test-image:${{ github.sha }} --version

Container Build Strategy

Multi-Stage Dockerfile

# Dockerfile
FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build
WORKDIR /src

# Copy csproj files and restore dependencies
COPY ["src/Nucleus.API/Nucleus.API.csproj", "src/Nucleus.API/"]
COPY ["src/Nucleus.Core/Nucleus.Core.csproj", "src/Nucleus.Core/"]
COPY ["src/Nucleus.Data/Nucleus.Data.csproj", "src/Nucleus.Data/"]
RUN dotnet restore "src/Nucleus.API/Nucleus.API.csproj"

# Copy source code and build
COPY . .
WORKDIR "/src/src/Nucleus.API"
RUN dotnet build "Nucleus.API.csproj" -c Release -o /app/build

# Publish application
FROM build AS publish
RUN dotnet publish "Nucleus.API.csproj" -c Release -o /app/publish /p:UseAppHost=false

# Runtime image
FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS final
WORKDIR /app

# Create non-root user
RUN adduser --disabled-password --gecos '' appuser && chown -R appuser /app
USER appuser

# Copy published application
COPY --from=publish /app/publish .

# Health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
  CMD curl -f http://localhost:8080/health || exit 1

EXPOSE 8080
ENTRYPOINT ["dotnet", "Nucleus.API.dll"]

Build Optimization

# Optimized build step
- name: Build and push with cache
  uses: docker/build-push-action@v5
  with:
    context: .
    platforms: linux/amd64,linux/arm64
    push: true
    tags: ${{ steps.meta.outputs.tags }}
    cache-from: type=gha
    cache-to: type=gha,mode=max
    build-args: |
      BUILDKIT_INLINE_CACHE=1

Testing Pipeline

Unit Testing

test-unit:
  runs-on: ubuntu-latest
  steps:
  - uses: actions/checkout@v4

  - name: Setup .NET
    uses: actions/setup-dotnet@v3
    with:
      dotnet-version: '8.0.x'

  - name: Run unit tests
    run: |
      dotnet test src/Nucleus.Tests/ \
        --logger trx \
        --results-directory TestResults/ \
        --collect:"XPlat Code Coverage" \
        --settings coverlet.runsettings

  - name: Upload test results
    uses: actions/upload-artifact@v3
    with:
      name: test-results
      path: TestResults/

Integration Testing

test-integration:
  runs-on: ubuntu-latest
  services:
    mssql:
      image: mcr.microsoft.com/mssql/server:2019-latest
      env:
        SA_PASSWORD: Test123!
        ACCEPT_EULA: Y
      options: >-
        --health-cmd "/opt/mssql-tools/bin/sqlcmd -S localhost -U sa -P Test123! -Q 'SELECT 1'"
        --health-interval 10s
        --health-timeout 5s
        --health-retries 3

    kafka:
      image: confluentinc/cp-kafka:7.4.0
      env:
        KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
        KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092

  steps:
  - uses: actions/checkout@v4

  - name: Run integration tests
    run: |
      dotnet test src/Nucleus.IntegrationTests/ \
        --logger trx \
        --results-directory TestResults/
    env:
      ConnectionStrings__DefaultConnection: "Server=localhost;Database=TestDB;User Id=sa;Password=Test123!;TrustServerCertificate=true;"
      Kafka__BootstrapServers: "localhost:9092"

End-to-End Testing

test-e2e:
  runs-on: ubuntu-latest
  if: github.ref == 'refs/heads/master'

  steps:
  - uses: actions/checkout@v4

  - name: Setup Node.js
    uses: actions/setup-node@v4
    with:
      node-version: '18'

  - name: Install dependencies
    run: npm install -g newman

  - name: Run E2E tests
    run: |
      newman run tests/e2e/api-tests.postman_collection.json \
        --environment tests/e2e/staging.postman_environment.json \
        --timeout 300000

Security Integration

Vulnerability Scanning

security-scan:
  runs-on: ubuntu-latest
  steps:
  - uses: actions/checkout@v4

  - name: Run Trivy vulnerability scanner
    uses: aquasecurity/trivy-action@master
    with:
      scan-type: 'fs'
      scan-ref: '.'
      format: 'table'
      exit-code: '1'
      ignore-unfixed: true
      severity: 'CRITICAL,HIGH'

  - name: Run CodeQL analysis
    uses: github/codeql-action/init@v2
    with:
      languages: csharp

  - name: Autobuild
    uses: github/codeql-action/autobuild@v2

  - name: Perform CodeQL Analysis
    uses: github/codeql-action/analyze@v2

Container Security

container-scan:
  runs-on: ubuntu-latest
  steps:
  - name: Build image
    run: docker build -t scan-target .

  - name: Scan image with Trivy
    uses: aquasecurity/trivy-action@master
    with:
      image-ref: 'scan-target'
      format: 'sarif'
      output: 'trivy-results.sarif'

  - name: Upload scan results
    uses: github/codeql-action/upload-sarif@v2
    with:
      sarif_file: 'trivy-results.sarif'

Deployment Automation

ArgoCD Integration

# ArgoCD Application for automated deployment
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: nucleus-staging
  namespace: argocd
spec:
  project: rciis
  source:
    repoURL: [email protected]:MagnaBC/rciis-devops.git
    targetRevision: master
    path: apps/rciis/nucleus/staging
  destination:
    server: https://kubernetes.default.svc
    namespace: nucleus
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
    - CreateNamespace=true

Progressive Deployment

deploy-staging:
  needs: [test, build-and-push]
  runs-on: ubuntu-latest
  environment: staging

  steps:
  - name: Deploy to staging
    run: |
      # ArgoCD will automatically sync
      # Wait for deployment to complete
      argocd app wait nucleus-staging --timeout 600

  - name: Run smoke tests
    run: |
      curl -f https://nucleus-staging.devops.africa/health
      ./scripts/smoke-tests.sh staging

Monitoring and Alerting

Pipeline Monitoring

- name: Notify deployment status
  if: always()
  uses: 8398a7/action-slack@v3
  with:
    status: ${{ job.status }}
    channel: '#deployments'
    webhook_url: ${{ secrets.SLACK_WEBHOOK }}
    custom_payload: |
      {
        "text": "Deployment ${{ job.status }}: ${{ github.repository }}@${{ github.sha }}",
        "color": "${{ job.status == 'success' && 'good' || 'danger' }}"
      }

Performance Tracking

- name: Track deployment metrics
  run: |
    # Send deployment metrics to monitoring system
    curl -X POST https://metrics.devops.africa/deployment \
      -H "Content-Type: application/json" \
      -d '{
        "repository": "${{ github.repository }}",
        "commit": "${{ github.sha }}",
        "environment": "staging",
        "status": "success",
        "duration": "${{ steps.deploy.outputs.duration }}"
      }'

Best Practices

Pipeline Design

  1. Fast Feedback: Quick failure detection
  2. Parallel Execution: Run tests in parallel
  3. Caching: Cache dependencies and build artifacts
  4. Security First: Integrate security scanning early

Code Quality

  1. Automated Testing: Comprehensive test coverage
  2. Code Analysis: Static code analysis
  3. Formatting: Automated code formatting
  4. Documentation: Keep pipeline documentation current

Security

  1. Secret Management: Use GitHub Secrets for sensitive data
  2. Least Privilege: Minimal required permissions
  3. Vulnerability Scanning: Regular security scans
  4. Compliance: Meet regulatory requirements

Deployment Strategy

  1. Blue-Green: Zero-downtime deployments
  2. Canary: Gradual rollout for risk reduction
  3. Rollback: Quick rollback capability
  4. Monitoring: Real-time deployment monitoring

For advanced CI/CD patterns and troubleshooting, refer to the GitHub Actions and ArgoCD documentation.