Building a CI/CD Pipeline That Actually Holds Up

Every team I have worked with has that one pipeline: the one nobody wants to touch, the one that "works most of the time," the one with 23 steps and 4 || true statements. This post is the playbook I use to rebuild pipelines properly.

Start with the boring contract

Before a single YAML file is written, decide three things:

What does green mean? Build, unit tests, integration tests, lint, security scan — and how long should the whole thing take? If it takes more than 10 minutes for a service, split it.
What does shipping mean? Deploy to staging on merge to main, deploy to prod on a tagged release, or continuous deployment straight to prod? Pick one and write it down.
How do we roll back? A pipeline without a rollback story is a hostage situation. Define the rollback path up front.

The anatomy of a pipeline

Treat the pipeline like a deployment, not a script. Stages should be:

name: build-test-release
on: [push]

jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: npm ci
      - run: npm run lint
      - run: npm test -- --coverage

  build:
    needs: verify
    steps:
      - uses: actions/checkout@v4
      - run: docker build -t $IMAGE:$GITHUB_SHA .
      - run: docker push $IMAGE:$GITHUB_SHA

  deploy:
    needs: build
    if: github.ref == 'refs/heads/main'
    steps:
      - run: ./deploy.sh $GITHUB_SHA

Rules that prevent the duct tape

One source of truth. Never build the artifact twice. Build once, promote the same artifact through every environment.
Cache deliberately. Cache dependencies, not node_modules snapshots — they rot and hide updates.
Pin everything. Pin the runner image and action versions. actions/checkout@v4 gives you semantic releases; @main gives you surprise changes on Friday.
Make secrets boring. Inject secrets at deploy time from a vault, never bake them into images or commit them.
Treat the pipeline as code. Review pipeline changes like any other PR. A broken pipeline is an incident with a postmortem.

When green lies to you

Green is only as good as what it tests. A pipeline that runs tests against mockEverything gives confidence in nothing. Add contract tests and smoke tests against a real environment. Test the deployment, not just the build.

The 80/20 that matters most

You can skip coverage thresholds, elaborate matrices, and fancy dashboards. You cannot skip:

Fast feedback loops (unit tests first, slow suites later)
A single command that works locally
A pipeline that fails loudly instead of finishing with quiet warnings
A 10-minute maximum run time

Build that, and the rest is polish.

Summary

A reliable pipeline is a small pipeline. Green means something. Rollback is always possible. Everything is in git. Ship it.