SMP CI/CD Pipeline¶

Tooling: GitHub Actions + ArgoCD + Helm · Audience: DevOps, Backend

1. Pipeline overview¶

Code push (feature branch)
 │
 ▼
┌──────────────┐
│ GH Actions │ CI: lint → test → build → scan
│ (PR pipeline)│
└──────┬───────┘
 │ PR approved + merged
 ▼
┌──────────────┐
│ GH Actions │ CD: build image → push to registry → update GitOps repo
│ (main branch)│
└──────┬───────┘
 │ commit to gitops-deploys repo
 ▼
┌──────────────┐
│ ArgoCD │ Sync: pull manifest → apply to K8s cluster
│ (auto-sync) │
└──────┬───────┘
 │
 ▼
 Dev env updated
 │ manual promote
 ▼
 Staging env updated (after manual approve)
 │ manual promote
 ▼
 Prod env updated (after release manager approve)

2. Branch & deploy mapping¶

Branch	Trigger	Deploys to	Auto/Manual
`feat/`, `fix/`	PR open	(CI only, no deploy)	Auto CI
`main`	Merge	dev cluster	Auto
Tag `v..*-rc.N`	Tag push	staging cluster	Auto
Tag `v..*`	Tag push	prod cluster	Manual approve

3. CI: PR pipeline (`.github/workflows/ci.yml`)¶

name: CI

on:
 pull_request:
 branches: [main]
 push:
 branches: [main]

env:
 GO_VERSION: "1.22"

jobs:
 lint:
 name: Lint
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v4
 - uses: actions/setup-go@v5
 with:
 go-version: ${{ env.GO_VERSION }}
 cache: true
 - name: gofmt check
 run: |
 fmt_files=$(gofmt -l .)
 if [ -n "$fmt_files" ]; then
 echo "Unformatted files: $fmt_files"
 exit 1
 fi
 - name: go vet
 run: go vet ./...
 - uses: golangci/golangci-lint-action@v3
 with:
 version: latest
 args: --timeout=5m

 test:
 name: Unit + integration tests
 runs-on: ubuntu-latest
 services:
 mysql:
 image: mysql:8.0
 env:
 MYSQL_ROOT_PASSWORD: test
 MYSQL_DATABASE: smp_test
 ports: ["3306:3306"]
 options: >-
 --health-cmd "mysqladmin ping" --health-interval 10s --health-timeout 5s --health-retries 5
 redis:
 image: redis:7
 ports: ["6379:6379"]
 steps:
 - uses: actions/checkout@v4
 - uses: actions/setup-go@v5
 with:
 go-version: ${{ env.GO_VERSION }}
 cache: true
 - name: Run migrations
 run: |
 go install github.com/golang-migrate/migrate/v4/cmd/migrate@latest
 migrate -database "mysql://root:test@tcp(localhost:3306)/smp_test" -path migrations up
 - name: Unit tests
 run: go test -race -coverprofile=coverage.out ./...
 - name: Integration tests
 run: go test -tags=integration ./test/integration/...
 env:
 MYSQL_DSN: root:test@tcp(localhost:3306)/smp_test?parseTime=true
 REDIS_ADDR: localhost:6379
 - name: Coverage check
 run: |
 cov=$(go tool cover -func=coverage.out | grep total | awk '{print $3}' | sed 's/%//')
 if (( $(echo "$cov < 60" | bc -l) )); then
 echo "Coverage $cov% below threshold 60%"
 exit 1
 fi
 - uses: codecov/codecov-action@v4
 with: { file: coverage.out }

 security:
 name: Security scans
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v4
 - name: Gitleaks
 uses: gitleaks/gitleaks-action@v2
 - name: govulncheck
 run: |
 go install golang.org/x/vuln/cmd/govulncheck@latest
 govulncheck ./...
 - name: Trivy filesystem scan
 uses: aquasecurity/trivy-action@master
 with:
 scan-type: 'fs'
 severity: 'HIGH,CRITICAL'
 exit-code: '1'

 openapi:
 name: OpenAPI validation
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v4
 - name: Spectral lint
 uses: stoplightio/spectral-action@latest
 with:
 file_glob: "api/openapi.yaml"
 - name: Breaking change check (vs main)
 run: |
 npx -y @apidevtools/swagger-cli validate api/openapi.yaml

4. CD: Build & deploy (`.github/workflows/cd.yml`)¶

name: CD

on:
 push:
 branches: [main]
 tags: ["v*.*.*"]

env:
 REGISTRY: ghcr.io
 IMAGE_NAME: ${{ github.repository }}

jobs:
 build:
 runs-on: ubuntu-latest
 permissions:
 contents: read
 packages: write
 outputs:
 image_tag: ${{ steps.meta.outputs.version }}
 steps:
 - uses: actions/checkout@v4
 - uses: docker/setup-buildx-action@v3
 - uses: docker/login-action@v3
 with:
 registry: ${{ env.REGISTRY }}
 username: ${{ github.actor }}
 password: ${{ secrets.GITHUB_TOKEN }}
 - id: meta
 uses: docker/metadata-action@v5
 with:
 images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
 tags: |
 type=ref,event=branch
 type=sha,format=short
 type=semver,pattern={{version}}
 - uses: docker/build-push-action@v5
 with:
 context: .
 push: true
 tags: ${{ steps.meta.outputs.tags }}
 labels: ${{ steps.meta.outputs.labels }}
 cache-from: type=gha
 cache-to: type=gha,mode=max
 provenance: true
 sbom: true

 deploy-dev:
 needs: build
 if: github.ref == 'refs/heads/main'
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v4
 with:
 repository: trungnguyenchanh/smp-gitops
 token: ${{ secrets.GITOPS_PAT }}
 path: gitops
 - name: Update image tag in dev manifest
 run: |
 cd gitops/environments/dev
 yq e -i ".image.tag = \"${{ needs.build.outputs.image_tag }}\"" ${{ github.event.repository.name }}.yaml
 git config user.email "ci@smp.vn"
 git config user.name "ci-bot"
 git add . && git commit -m "deploy(dev): ${{ github.event.repository.name }} → ${{ needs.build.outputs.image_tag }}"
 git push

 deploy-staging:
 needs: build
 if: startsWith(github.ref, 'refs/tags/v') && contains(github.ref, '-rc.')
 runs-on: ubuntu-latest
 environment:
 name: staging
 url: https://staging-api.smp.vn
 steps:
 - uses: actions/checkout@v4
 with:
 repository: trungnguyenchanh/smp-gitops
 token: ${{ secrets.GITOPS_PAT }}
 path: gitops
 - name: Update staging manifest
 run: |
 cd gitops/environments/staging
 yq e -i ".image.tag = \"${{ needs.build.outputs.image_tag }}\"" ${{ github.event.repository.name }}.yaml
 git add . && git commit -m "deploy(staging): ${{ github.event.repository.name }} → ${{ needs.build.outputs.image_tag }}"
 git push

 deploy-prod:
 needs: build
 if: startsWith(github.ref, 'refs/tags/v') && !contains(github.ref, '-rc.')
 runs-on: ubuntu-latest
 environment:
 name: production
 url: https://api.smp.vn
 steps:
 - uses: actions/checkout@v4
 with:
 repository: trungnguyenchanh/smp-gitops
 token: ${{ secrets.GITOPS_PAT }}
 path: gitops
 - name: Update prod manifest
 run: |
 cd gitops/environments/prod
 yq e -i ".image.tag = \"${{ needs.build.outputs.image_tag }}\"" ${{ github.event.repository.name }}.yaml
 git add . && git commit -m "deploy(prod): ${{ github.event.repository.name }} → ${{ needs.build.outputs.image_tag }}"
 git push
 - name: Notify Slack
 uses: slackapi/slack-github-action@v1
 with:
 payload: |
 {"text": "🚀 Prod deploy: ${{ github.event.repository.name }} v${{ needs.build.outputs.image_tag }}"}
 env:
 SLACK_WEBHOOK_URL: ${{ secrets.SLACK_PROD_WEBHOOK }}
 SLACK_WEBHOOK_TYPE: INCOMING_WEBHOOK

5. GitOps repo structure¶

Repo smp-gitops (separate from service repos):

smp-gitops/
├── README.md
├── environments/
│ ├── dev/
│ │ ├── order-svc.yaml
│ │ ├── dispatch-engine.yaml
│ │ └── ...
│ ├── staging/
│ └── prod/
└── apps/ # ArgoCD Application manifests
 ├── dev-apps.yaml
 ├── staging-apps.yaml
 └── prod-apps.yaml

ArgoCD configured to auto-sync environments/dev, manual sync staging + prod.

6. Dockerfile template (Go service)¶

Multi-stage build:

# Build stage
FROM golang:1.22-alpine AS builder
RUN apk add --no-cache git ca-certificates
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build \
 -ldflags="-w -s -X main.version=$(git describe --tags --always) -X main.commit=$(git rev-parse HEAD)" \
 -o /app/server ./cmd/server

# Runtime stage
FROM gcr.io/distroless/static-debian12:nonroot
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=builder /app/server /app/server
EXPOSE 8080
USER nonroot:nonroot
ENTRYPOINT ["/app/server"]

7. Helm chart structure¶

Per service repo có deployments/helm/:

deployments/helm/order-svc/
├── Chart.yaml
├── values.yaml # default values
├── values-dev.yaml # dev overrides
├── values-staging.yaml
├── values-prod.yaml
└── templates/
 ├── deployment.yaml
 ├── service.yaml
 ├── configmap.yaml
 ├── hpa.yaml # auto-scale
 ├── pdb.yaml # pod disruption budget
 └── servicemonitor.yaml # prometheus scrape

8. Deployment strategy¶

Rolling update (dev, staging)¶

strategy:
 type: RollingUpdate
 rollingUpdate:
 maxSurge: 1
 maxUnavailable: 0

Blue/green (prod, critical services)¶

Order-svc, dispatch-engine, partner-svc use blue/green via Argo Rollouts:

spec:
 strategy:
 blueGreen:
 activeService: order-svc-active
 previewService: order-svc-preview
 autoPromotionEnabled: false # require manual promote
 scaleDownDelaySeconds: 300

Flow: 1. Deploy new version → preview service 2. SRE smoke test via preview URL 3. Manual promote → swap active/preview 4. Old version sleep 5 min then scale down 5. If issue → 1-click rollback (re-swap)

9. Database migrations in pipeline¶

Migrations run before service deploy:

# Helm pre-install/pre-upgrade hook
apiVersion: batch/v1
kind: Job
metadata:
 name: migrate-order-db
 annotations:
 helm.sh/hook: pre-install,pre-upgrade
 helm.sh/hook-weight: "-5"
spec:
 template:
 spec:
 containers:
 - name: migrate
 image: migrate/migrate:v4.17.0
 args:
 - "-database=$(MYSQL_DSN)"
 - "-path=/migrations"
 - "up"
 volumeMounts:
 - name: migrations
 mountPath: /migrations

Rollback safety: - Tất cả migrations phải reversible (có file .down.sql) - Breaking schema change: 2-phase deploy (expand → migrate data → contract) - Step 1: add new column (nullable) → deploy code đọc cả cũ + mới - Step 2: backfill data - Step 3: deploy code chỉ dùng mới - Step 4: drop column cũ

10. Smoke test post-deploy¶

.github/workflows/smoke-test.yml:

on:
 workflow_run:
 workflows: ["CD"]
 types: [completed]
jobs:
 smoke:
 if: github.event.workflow_run.conclusion == 'success'
 runs-on: ubuntu-latest
 steps:
 - uses: actions/checkout@v4
 - name: Smoke test
 run: |
 ENV_URL="${{ vars.ENV_API_URL }}"
 # Health check
 curl -f "$ENV_URL/health" || exit 1
 # Read endpoint check
 curl -f "$ENV_URL/api/v1/catalog/services" -H "Authorization: Bearer ${{ secrets.SMOKE_TEST_JWT }}" || exit 1
 # Critical create endpoint (idempotency)
 curl -f -X POST "$ENV_URL/api/v1/test/echo" -d '{"hello":"world"}' || exit 1

11. Monitoring deploys¶

Post-deploy alerts: - Error rate spike > 2% in 5 min → PagerDuty - Latency p95 > 2x baseline in 5 min → PagerDuty - Pod crash loop → Slack

Grafana dashboard "Deploy timeline" overlays deployments with metrics.

12. Release versioning (SemVer)¶

MAJOR.MINOR.PATCH (vd v3.3.0)
Pre-release: v3.3.0-rc.1, v3.3.0-rc.2
Tag created khi PR release merged to main

# Release flow
git checkout main && git pull
git tag -a v3.3.0 -m "Release v3.3.0 · Partner platform"
git push origin v3.3.0
# → triggers CD prod deploy with manual approval

CHANGELOG.md updated trong PR release.

13. Rollback playbook¶

# Option 1: ArgoCD rollback
argocd app rollback smp-order-svc-prod <previous-revision>

# Option 2: kubectl rollback (emergency)
kubectl rollout undo deployment/order-svc -n smp-prod

# Option 3: Re-tag previous version
git tag -a v3.3.0-rollback -m "Rollback to v3.2.5"
# manually update gitops repo

Always: - Notify in #prod-alerts Slack - Create incident ticket - Post-mortem within 48h if customer impact

14. Branch protection rules¶

GitHub settings for main: - Require PR with 1 approver from CODEOWNERS - Require status checks: lint, test, security, openapi - Require linear history - Require signed commits (Phase 2) - Lock force push - Auto-delete branches after merge

15. Container image policy¶

Base: distroless or alpine only
No :latest tags allowed
All images scanned by Trivy (block HIGH/CRITICAL)
SBOM generated and signed (cosign)
Image registry: GitHub Container Registry (ghcr.io)
Retention: prod tags forever, dev tags 30 days