Skip to content

Rollback

Rolling back a deployment safely is an operational skill — the runbook for the actual incident is runbooks/rollback.md. This page documents the strategy.

Principles

  1. Immutable images — every build is tagged by commit/version; "rollback" is re-deploying an older image, never patching a running container.
  2. Migrations are not instantly reversible — a DB migration rollback is a forward migration that undoes the previous one (or a data repair), never a re-run of migrate reset. See database/migrations.md.
  3. Separate code rollback from data rollback. They have different risk profiles and different timeframes.

Decision flow

flowchart TB
    A["Deploy misbehaves?"] -->|"no"| OK["continue"]
    A -->|"yes"| B{"Data migration shipped?"}
    B -->|"no"| C["Re-deploy previous image (fast)"]
    B -->|"yes"| D["Assess: forward-fix vs data repair"]
    C --> E["Verify health + metrics"]
    D --> E
    E -->|"ok"| DONE["Post-incident review"]

Fast rollback (code only)

# Re-deploy the previous immutable image tag
#   e.g. ghcr.io/…/dichit-backend:<previous-version>
# via the environment's deploy pipeline / ECS service update.

Verify:

  • /health passes and /meta reports the expected version.
  • Error rate returns to baseline (Sentry, CloudWatch).
  • Queue workers resume normally.

When a migration is involved

  • Prefer a forward-fix migration if the data can be corrected in place.
  • If you must revert schema changes, write a new migration that reverses the DDL — applied and tested through the same pipeline.
  • Coordinate with the team: reverting a migration can break the new code path that's still rolling out.

Prevention

  • Small, reversible releases; database changes separated from code changes when possible.
  • [skip-deploy] for image-only builds.
  • Release-please keeps every version in CHANGELOG.md so the last good version is always known (see release-process.md).
  • Monitor during the deploy window (see infrastructure/monitoring.md).