Staging Infrastructure¶

Staging follows the same deployment invariants as development and production: immutable images, deployment through AWS Systems Manager, migrations in a separate image, health-gated rollout, and automatic application rollback. PostgreSQL runs in Docker on the staging EC2 instance. Redis and BullMQ use the external Aiven Redis service; staging does not run a Docker Redis container.

Runtime layout¶

• Traefik terminates TLS for api.staging.dichit.money.
• The authenticated Traefik dashboard is available at https://traefik.staging.dichit.money/dashboard/; port 8080 is not published.
• The application and migration images are built separately from Dockerfile.staging.
• PostgreSQL is available only on the private dichit-staging-data Docker network. It is not published on an EC2 host port.
• PostgreSQL data persists in the dichit-staging-postgres Docker volume.
• The application connects outbound to Aiven Redis over TLS. No Redis image, port, or volume is created on the host.
• App, Traefik, and PostgreSQL use bounded Docker local logs: five files of up to 20 MB each.

Environment secret¶

Create a Secrets Manager secret whose value is the complete .env.staging file in plaintext dotenv format. The deployment writes it atomically with mode 0600. Do not store the payload as JSON.

These values are required for the staging data services:

NODE_ENV=staging
PORT=4500

DATABASE_HOST=db
DATABASE_PORT=5432
DATABASE_USER=dichit
DATABASE_PASSWORD=replace-with-a-random-staging-password
DATABASE_DB=dichit_staging
DATABASE_URL=postgresql://dichit:URL_ENCODED_PASSWORD@db:5432/dichit_staging
DATABASE_SSL_ENABLED=false
POSTGRES_VOLUME_NAME=dichit-staging-postgres

REDIS_HOST=YOUR_AIVEN_REDIS_HOST
REDIS_PORT=YOUR_AIVEN_REDIS_PORT
REDIS_PASSWORD=YOUR_AIVEN_REDIS_PASSWORD
REDIS_TLS_ENABLED=true
REDIS_CONNECT_TIMEOUT=10000
REDIS_MAX_RETRIES_PER_REQUEST=3

DATABASE_URL must use the Compose service name db and container port 5432, regardless of the EC2 host. URL-encode special characters in the database password. The current Redis client authenticates as Aiven's default Redis user by password; use the default service credentials. POSTGRES_VOLUME_NAME can be changed during the first cutover to attach an existing staging PostgreSQL Docker volume.

Create a separate Secrets Manager secret containing an htpasswd users file for the dashboard:

umask 077
dashboard_auth_file="$(mktemp)"
htpasswd -cB "$dashboard_auth_file" stagingadmin

aws secretsmanager create-secret \
  --region ap-south-1 \
  --name /dichit/staging/traefik-dashboard-auth \
  --secret-string "file://${dashboard_auth_file}"

rm -f "$dashboard_auth_file"

Provision the deployment path¶

The staging EC2 instance profile needs AmazonSSMManagedInstanceCore. Install Docker Engine, Docker Compose v2, AWS CLI, and SSM Agent. Authenticate Docker to GHCR with a read-only package credential before the first deployment.

Reuse the account-level GitHub Actions OIDC provider, then deploy:

aws cloudformation deploy \
  --region ap-south-1 \
  --stack-name dichit-staging-deployment \
  --template-file infra/aws/staging/deployment.yml \
  --capabilities CAPABILITY_NAMED_IAM \
  --parameter-overrides \
    GitHubOidcProviderArn=YOUR_GITHUB_OIDC_PROVIDER_ARN \
    StagingInstanceId=YOUR_STAGING_INSTANCE_ID \
    DockerHostRoleName=YOUR_STAGING_INSTANCE_ROLE \
    AppEnvironmentSecretArn=YOUR_STAGING_ENV_SECRET_ARN \
    DashboardAuthSecretArn=YOUR_DASHBOARD_AUTH_SECRET_ARN

Configure the GitHub staging environment:

The image build job runs before the protected deployment job, so configure SENTRY_AUTH_TOKEN, SENTRY_ORG, and SENTRY_PROJECT as repository-level Actions secrets, matching the existing development and production build workflows.

The infrastructure image values must use registry digests:

traefik@sha256:...
postgres@sha256:...

Resolve STAGING_POSTGRES_IMAGE from a tested PostgreSQL 17 Alpine tag and keep the immutable digest within that major version. A PostgreSQL major-version change requires a planned database upgrade, not a normal staging deployment.

The EC2 security group needs inbound TCP 80 and 443, no public PostgreSQL or Redis ingress, and outbound access to AWS APIs, GHCR, Aiven's Redis host and port, and other application integrations. Create Route 53 A records for api.staging.dichit.money and traefik.staging.dichit.money pointing to the staging Elastic IP.

First cutover from the legacy staging stack¶

The old staging Compose file used mutable images, SSH deployment, a Docker Redis container, and Compose-generated volume names. Perform this once before enabling the new workflow:

1. Back up PostgreSQL and record the current volume name:

cd PATH_TO_OLD_DICHIT_BACKEND
docker compose -f docker-compose.staging.yml --env-file .env.staging ps
docker inspect postgres --format '{{range .Mounts}}{{println .Name .Destination}}{{end}}'

1. Put the recorded volume name in the staging environment secret as POSTGRES_VOLUME_NAME=RECORDED_VOLUME_NAME. Confirm that its PostgreSQL major version matches the image selected by STAGING_POSTGRES_IMAGE.
2. Stop and remove only the legacy staging containers and network while preserving volumes:

docker compose -f docker-compose.staging.yml --env-file .env.staging down

Do not add --volumes. The old PostgreSQL volume must remain.

1. Confirm ports 80 and 443 are free and the PostgreSQL volume still exists:

docker ps --format '{{.Names}}\t{{.Ports}}'
docker volume inspect RECORDED_VOLUME_NAME

1. Run the new staging workflow manually. After application, database, Aiven Redis, LevelDB, and rollback checks pass, remove the unused legacy Redis volume in a separately reviewed cleanup.

For a brand-new staging environment, keep the default POSTGRES_VOLUME_NAME=dichit-staging-postgres.

Deployment flow¶

A push to staging:

1. Builds separate application and migration images.
2. Publishes convenience tags and captures immutable image digests with SBOM and provenance.
3. Uploads Compose, the deployment script, and Traefik configuration to encrypted S3.
4. Invokes the EC2 instance through SSM.
5. Retrieves .env.staging and dashboard authentication from Secrets Manager.
6. Starts PostgreSQL and Traefik, waits for PostgreSQL, and verifies Aiven Redis connectivity.
7. Applies database migrations.
8. Optionally seeds the database.
9. Recreates the application and verifies https://api.staging.dichit.money/health.
10. Restores the previous application image if the health check fails.

Application startup never migrates, resolves migrations, resets, or seeds the database.

Staging deploy seeding is controlled by GitHub environment variables on the staging deployment job:

• STAGING_SEED_ON_DEPLOY=true runs ./scripts/seed.staging.sh after migrations.
• STAGING_SEED_ON_DEPLOY=false skips deploy-time seeding entirely.
• STAGING_SEED_DEEP_ON_DEPLOY=true passes --deep to the seeder when seeding is enabled.
• STAGING_SEED_DEEP_ON_DEPLOY=false runs the normal staging seed.

The staging deploy script defaults both seed toggles to false when the GitHub variables are not set. Direct host runs can still override SEED_ON_DEPLOY and SEED_DEEP_ON_DEPLOY manually if needed.

Backups¶

The local PostgreSQL named volume lives on the EBS storage backing /var/lib/docker. Deploy the short-retention backup policy:

aws cloudformation deploy \
  --region ap-south-1 \
  --stack-name dichit-staging-backup \
  --template-file infra/aws/staging/backup.yml \
  --capabilities CAPABILITY_NAMED_IAM

Tag the EBS volume containing Docker data with:

Backup=dichit-staging

Verify the backup includes /var/lib/docker, and perform a restore test before relying on it. An EBS snapshot is crash-consistent; for stronger database consistency, schedule a PostgreSQL-native dump to separately protected storage.

Post-deployment verification¶

Start an approved SSM session and run:

cd /opt/dichit-staging
source .deployment-images.env
docker compose -f docker-compose.staging.yml --env-file .env.staging ps
docker compose -f docker-compose.staging.yml --env-file .env.staging logs --tail 200 app db traefik

Confirm:

• the app, PostgreSQL, and Traefik containers are healthy;
• no Redis container or Redis host port exists;
• the app logs show successful TLS connectivity to Aiven Redis;
• anonymous dashboard requests return 401;
• PostgreSQL and LevelDB data survive container recreation;
• an application rollback and an EBS restore have been tested.