How do I set up a deployment pipeline for environment promotion with self-hosted runners?

Here’s a practical way to set up a deployment pipeline with environment promotion using self-hosted runners.

1) Design the promotion flow

A common pattern is:

Build once → test → deploy to dev → promote to staging → promote to prod

Key rule:

• Build artifacts only once
• Promote the same artifact through environments
• Avoid rebuilding per environment, so what you tested is exactly what you deploy

2) Set up environments

Create distinct environments in your CI/CD platform, for example:

• dev
• staging
• prod

Each environment should have:

• its own credentials/secrets
• its own deployment target
• optional approval gates for higher environments

If your platform supports it, use:

• required reviewers / manual approvals for staging and prod
• environment-specific secrets
• deployment history and rollback tracking

3) Prepare self-hosted runners

Use self-hosted runners when you need:

• access to private networks
• custom tooling
• more control over runtime
• deployment access to internal infrastructure

Best practices:

• Use separate runner groups for:
  • build/test
  • non-prod deploy
  • prod deploy
• Restrict production runners tightly
• Prefer ephemeral runners if possible
• Keep runners patched and hardened
• Run jobs with least privilege

A good approach is:

• build jobs can run on general self-hosted runners
• deploy jobs run on runners that have network access to the target environment

4) Store build outputs as artifacts

After build/test:

• package the app or container image
• publish it to an artifact repository or registry
• record the exact version, commit SHA, and build number

Examples:

• container image in a registry
• zipped release bundle in artifact storage
• Helm chart or deployment manifest versioned in Git or package storage

5) Promote by referencing the same artifact

Each environment deployment should pull the same immutable artifact:

• image tag like app:1.4.7 or better app@sha256:...
• release package version
• immutable build artifact ID

Do not use “latest” for promotion.

6) Add approval gates between environments

A typical promotion flow:

1. CI build
2. Deploy to dev
3. Automated tests
4. Approve promotion to staging
5. Deploy to staging
6. Integration/UAT tests
7. Approve promotion to prod
8. Deploy to prod

You can implement approvals via:

• environment protection rules
• manual approval steps
• change management integration

7) Example workflow structure

If you’re using GitHub Actions, a simplified pattern looks like:

name: deploy

on:
  push:
    branches: [main]

jobs:
  build:
    runs-on: self-hosted
    outputs:
      artifact_version: ${{ steps.meta.outputs.version }}
    steps:
      - uses: actions/checkout@v4
      - run: ./build.sh
      - run: ./test.sh
      - run: ./publish-artifact.sh
      - id: meta
        run: echo "version=1.0.${GITHUB_RUN_NUMBER}" >> $GITHUB_OUTPUT

  deploy_dev:
    needs: build
    runs-on: [self-hosted, dev-deployer]
    environment: dev
    steps:
      - run: ./deploy.sh --version ${{ needs.build.outputs.artifact_version }}

  deploy_staging:
    needs: deploy_dev
    runs-on: [self-hosted, staging-deployer]
    environment: staging
    steps:
      - run: ./deploy.sh --version ${{ needs.build.outputs.artifact_version }}

  deploy_prod:
    needs: deploy_staging
    runs-on: [self-hosted, prod-deployer]
    environment: prod
    steps:
      - run: ./deploy.sh --version ${{ needs.build.outputs.artifact_version }}

In a real pipeline:

• the prod job would usually require manual approval
• the deploy jobs should use environment secrets
• the artifact version should come from the build job or release metadata

8) Use runner labels to target environments

Label runners so jobs go to the right machine pool, for example:

• self-hosted
• linux
• deploy-dev
• deploy-prod

Then use those labels in your job definitions to ensure:

• deploy jobs run only on runners with the right network and permissions
• prod deployments can’t accidentally land on non-prod infrastructure

9) Secure secrets properly

Do not store deployment secrets in the repo.

Use:

• environment secrets
• vault integration
• short-lived cloud credentials
• OIDC where supported

For production:

• limit secret access to prod environment jobs only
• rotate credentials regularly
• avoid long-lived SSH keys where possible

10) Add rollback support

A good promotion pipeline should support rollback.

Options:

• redeploy previous artifact version
• keep previous container images available
• version your manifests and config
• use blue/green or canary strategies

Make sure every release records:

• artifact version
• commit SHA
• deployed environment
• timestamp
• approver

11) Common pitfalls

Avoid these:

• rebuilding separately for each environment
• using mutable tags like latest
• giving all runners access to prod
• mixing build and deploy responsibilities on the same runner without isolation
• hardcoding environment values in scripts
• storing secrets in pipeline logs

12) Recommended architecture

A solid setup is:

• One build pipeline
• One immutable artifact
• Separate environment deploy jobs
• Self-hosted runner groups per environment
• Manual approval for staging/prod
• Environment-specific secrets
• Deployment audit trail

If you want, I can also provide:

1. a GitHub Actions example,
2. a GitLab CI example, or
3. a Jenkins pipeline example for this setup.