Continuous integration runs the same checks for each proposed change. A small pipeline catches missing dependencies, failing tests, and build errors before a change reaches production. Start with a short workflow that mirrors the commands a developer can run locally.
Add a workflow file #
Create .github/workflows/ci.yml:
name: CI
on:
pull_request:
push:
branches: [main]
jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
- run: npm ci
- run: npm test
- run: npm run build
npm ci installs the exact dependency tree recorded in the lockfile. The test and build commands should also work locally and return a non-zero exit code when they fail.
Adapt the workflow to Python #
For a Python project, use actions/setup-python, install from a pinned requirements or project lock file, then run pytest. Keep separate jobs only when they provide useful parallelism or clearer feedback; a minimal pipeline is easier to maintain.
Make CI reliable and safe #
- Commit the lockfile and use the corresponding reproducible install command.
- Do not store production secrets in a test job that does not need them.
- Cache package downloads only when cache keys include the lockfile.
- Keep test data deterministic and avoid depending on public services.
- Require the check to pass before merging if repository settings support required checks.
Treat CI as a feedback loop #
The pipeline should run checks that give fast, actionable feedback. Add linting, type checking, security scanning, or deployment previews when those checks are configured and useful; do not turn a small change into a large collection of flaky jobs.
The workflow is only the runner. A good CI process depends on understandable tests, repeatable dependencies, and a clear way to fix failures.
Design for feedback rather than green badges #
A useful workflow answers different questions at deliberate times. On each pull request, run formatting or lint checks, the fast unit suite, and integration checks for changed boundaries. Before deployment, build the artifact you intend to release and run smoke checks in the deployed environment. One giant workflow is hard to troubleshoot; many tiny workflows can duplicate setup and obscure which checks are required. Group jobs by purpose and make branch protection require the checks that protect the main branch.
Use pull_request for ordinary validation. Treat values from pull requests as untrusted: code from a fork must not receive write permissions or production secrets. The pull_request_target event runs with base-repository privileges, so it should not check out and execute untrusted contribution code with credentials. If a bot needs elevated permission to add a label or comment, keep that job separate from the job that builds or tests contributed code.
Workflow YAML is executable configuration. Review changes to actions, permissions, shell commands, and expressions with the same care as application code. Pin third-party actions to a full commit SHA when reproducibility and supply-chain controls matter; update pins through a reviewed process. Use maintained actions where practical, but still review their permissions and data access.
Make dependencies and caches predictable #
Pin the runtime version used in CI to a supported version you also use locally. If a project supports several runtimes, use a matrix, but make one version the required merge check and avoid multiplying every combination across every optional job. Four runtimes, three operating systems, and two database versions create twenty-four jobs before tests start. Add combinations when they correspond to a real support commitment.
Cache package downloads to reduce setup time, but do not treat a cache as the source of truth. Include lockfiles and runtime versions in the key so dependency changes invalidate stale data. If cache restoration fails, the job should still install from the lockfile. Caching generated application output can conceal missing build steps; prefer package-manager data and rebuild artifacts from checked-out source.
Use the lockfile in CI and fail if it is stale. For Python, create an isolated environment and install a locked dependency set. For JavaScript, use the package manager’s immutable lockfile mode. This makes a pull request reproducible and the dependency graph reviewable. Automated update tools can propose changes, but tests still need to validate each update.
Control parallelism and databases #
Parallel jobs are valuable when they do not compete for mutable resources. Use unique database names, schemas, or service containers per job. Never let jobs run migrations or destructive cleanup against the same shared test database. For service containers, wait for readiness with an explicit health check instead of assuming a process is ready immediately after startup.
Seed only the data needed for each integration scenario. A global seed script that creates hundreds of rows makes failures harder to understand and the suite slower. Prefer isolated transactions, a disposable database, or unique test tenants. If tests run in parallel and share a schema, look for collisions on emails, slugs, and sequential identifiers before enabling more workers.
Set timeouts at both job and command levels. A job timeout limits runaway work; a test-runner timeout helps identify the stuck test. Avoid continue-on-error for required tests because it reports a green workflow while discarding the status check’s signal. If a non-blocking experiment is useful, label it clearly and keep it out of merge gates.
Handle credentials and permissions deliberately #
Most test jobs need no repository write permission and no deployment credential. Start with a minimal permissions block and grant scopes only to the job that needs them. A coverage uploader may need a token; a unit-test job usually does not. Store deployment secrets in a protected environment and require the appropriate review or branch rule before exposing them.
Secrets are not safe if a workflow prints them, writes them into an artifact, or passes them to untrusted code. Avoid dumping the environment on failure. Do not put sensitive values in command-line arguments that may appear in logs or process listings. Use short-lived credentials where available, and rotate secrets if exposed in output or history.
For forked pull requests, skip steps that require secrets but continue safe validation. Make the outcome clear so contributors know which checks did not run. If a preview needs a token, use a trusted event and narrowly scoped credential or a provider integration designed for previews. Separate deployment credentials from untrusted build execution.
Preserve evidence for diagnosis #
Failure output should retain information needed to reproduce the issue without collecting unnecessary sensitive data. Upload a test report, browser trace, or application log only on failure and set a retention period. Exclude .env files, real-data database dumps, and browser profiles with authentication state. Use synthetic data in test accounts and screenshots.
Give jobs descriptive names such as unit tests (Python 3.12) and integration tests (PostgreSQL). When a matrix fails, make the failing combination visible. Avoid naming every job test, because status checks become hard to distinguish and branch protection can require the wrong one. A failure should link to an actionable log and identify the command that failed.
When a workflow flakes, capture the random seed, browser trace, or service log that explains it. Rerun once to determine whether the issue is intermittent, but keep the original failure visible. Frequent automatic retries create false confidence and waste minutes. Assign an owner to flaky checks so the team does not normalize red builds.
A merge-gate checklist #
Confirm that a new failing unit test makes the required check red, a build error blocks the merge, and a forked pull request cannot read deployment secrets. Confirm that the lockfile is used and the test database is isolated. Make sure a developer can reproduce the command locally and document required environment variable names without putting secret values in the repository.
Verify the deployment path separately. The artifact should come from the same commit that passed validation. A successful test job does not prove a deployment completed, and a successful deployment does not prove the service is healthy. Add a smoke check for the public health endpoint or key page and report deployment failures to the people responsible for release.
Revisit CI’s runtime, cache keys, permissions, and required checks when the dependency manager or hosting platform changes. Optimize measured bottlenecks instead of adding caches everywhere. A reliable pipeline has trustworthy status, explainable failures, and small enough permissions that mistakes have limited reach.
A pull-request workflow that stays understandable #
Start with a workflow that checks out the commit, selects a pinned runtime, restores package-manager cache, installs from a lockfile, and runs formatting, unit tests, and a production build. Add integration tests in a separate job with a disposable service when that isolation helps parallelism or diagnosis. Make the required status check match the job that protects the branch.
Keep the production build in CI even when another system deploys it. A source tree can pass unit checks while generated assets, content processing, or bundling fails. Upload an artifact only when a later trusted job needs it, and include commit identity in its metadata. Avoid passing a full workspace with caches or environment files between jobs.
Use concurrency groups to avoid redundant validation runs when new commits arrive. Do not cancel a release after it has changed production state unless the deployment tool supports safe cancellation. Validation is usually safe to cancel; migrations and release steps need explicit handling. Record which commit was deployed so a workflow cannot confuse stale artifacts with current source.
Harden credentials and workflow permissions #
Set workflow permissions to read-only, then elevate one job only when needed. A test job should not have repository write access because another job publishes a release. Use separate jobs and environments for publishing, and protect those environments by branch or review. Pull-request code can change scripts run by CI, so do not expose privileged credentials to it.
Review action source and maintenance before adding it. Pin third-party actions to a full commit SHA when reproducibility and supply-chain controls matter, and keep an update process. For high-risk workflows, use trusted actions and avoid downloading executable scripts at runtime. Treat action inputs and outputs as data to validate.
Use environment-scoped secrets only in the deployment job. Masking in logs is a last defense, not permission to print values. A malicious script can encode a secret, write it to an artifact, or send it over the network. Avoid giving untrusted jobs network access or credentials they do not need.
Keep the workflow fast without weakening its signal #
Measure checkout, dependency install, test runtime, and build separately. Cache package downloads using lockfile hashes and runtime version. A cache miss should still install correctly. Do not cache a test database or generated files that may hide an incomplete build. Periodically version caches when upgrading the toolchain.
Use matrices for supported versions, but do not multiply jobs without purpose. One required runtime can run the full suite while secondary runtimes run narrower compatibility checks. Split large test groups by stable boundaries so each failure remains meaningful. Keep browser and security suites at a cadence appropriate to their runtime and release risk.
Flaky tests need owners and diagnosis. Preserve traces and seeds on failure; reruns can collect evidence, but the original failure should stay visible. Do not set every job to retry until it passes. Track recurring flakes and make branch policy clear.
Validate the workflow itself #
Use a harmless pull request to confirm that a failing assertion makes the required check red, a build failure blocks promotion, and a fork cannot read deployment secrets. Check that the release job downloaded an artifact from the same commit that passed. Review branch protection after renaming checks so it does not silently stop enforcing the new one.
After deployment, run a smoke check and report it separately from the build. A green build proves code compiled and selected checks passed; it does not prove DNS, credentials, migrations, or the public path are correct. Capture the deployed commit and endpoint result. If health checks fail, stop the rollout and follow the recovery plan.
Move reusable logic into reviewed scripts or reusable workflows when duplication becomes costly. Document local command equivalents and required variable names. CI should make good practices automatic while staying understandable to the people who repair it under pressure.