Production Infrastructure¶

The production release path builds immutable application and migration images, delivers configuration through an encrypted S3 bucket, and runs deployment commands through AWS Systems Manager. The API container never performs migrations or seeding during startup.

Immediate credential response¶

Before the next release:

1. Revoke and recreate the Slack webhook that was previously committed.
2. Replace the old Traefik dashboard password. The public production dashboard is now disabled.
3. Audit .env.local, .env.test, published container images, and Git history. Rotate every credential that was real rather than a local placeholder.
4. Remove exposed values from Git history using your approved history-rewrite procedure, then require all collaborators to re-clone.
5. Delete or restrict affected GHCR image versions after replacement images have been published.

The current .dockerignore prevents environment files, Git metadata, backups, tests, documentation, and local tooling from entering future image build contexts.

AWS prerequisites¶

The existing production EC2 instance profile must have:

• AmazonSSMManagedInstanceCore
• CloudWatchAgentServerPolicy, or an equivalent least-privilege metrics policy
• the CloudWatch Logs writer policy from cloudwatch-logs.yml
• the artifact and environment-secret reader policy from deployment.yml

The instance requires Docker Compose, AWS CLI, SSM Agent, CloudWatch Agent, and authenticated read access to the private GHCR package.

Create one Secrets Manager secret containing the complete production dotenv payload as plaintext. Do not store the payload as JSON because the deployment script writes it directly to .env.prod.

Provision deployment infrastructure¶

Create the GitHub Actions OIDC provider once per AWS account, then deploy:

aws cloudformation deploy \
  --region ap-south-1 \
  --stack-name dichit-production-deployment \
  --template-file infra/aws/deployment.yml \
  --capabilities CAPABILITY_NAMED_IAM \
  --parameter-overrides \
    GitHubOidcProviderArn=YOUR_GITHUB_OIDC_PROVIDER_ARN \
    ProductionInstanceId=YOUR_INSTANCE_ID \
    DockerHostRoleName=YOUR_INSTANCE_ROLE_NAME \
    AppEnvironmentSecretArn=YOUR_APP_ENV_SECRET_ARN

Copy the stack outputs into the GitHub production environment:

Configure required reviewers on the GitHub production environment. A semantic version tag such as v1.8.0 then builds both images, publishes SBOM/provenance attestations, uploads the deployment bundle, and invokes the host through SSM.

Production deploy seeding is controlled by PROD_SEED_ON_DEPLOY on the GitHub production environment. true runs ./scripts/seed.prod.sh after migrations, and false skips deploy-time seeding. Production does not support deploy-time deep seeding.

CloudWatch logs and alarms¶

aws cloudformation deploy \
  --region ap-south-1 \
  --stack-name dichit-production-cloudwatch \
  --template-file infra/aws/cloudwatch-logs.yml \
  --capabilities CAPABILITY_NAMED_IAM \
  --parameter-overrides \
    DockerHostRoleName=YOUR_INSTANCE_ROLE_NAME \
    ProductionInstanceId=YOUR_INSTANCE_ID \
    AlarmEmail=oncall@example.com \
    RetentionInDays=30

Confirm the SNS email subscription. The stack alarms on application errors, HTTP 5xx responses, sustained instance CPU, and EC2 status-check failure.

Backups¶

Deploying the backup stack enables Vault Lock. Review retention values before deployment because locked recovery points cannot be deleted early.

aws cloudformation deploy \
  --region ap-south-1 \
  --stack-name dichit-production-backup \
  --template-file infra/aws/backup.yml \
  --capabilities CAPABILITY_NAMED_IAM

Tag every protected RDS, EBS, and supported data resource with:

Backup=dichit-production

Perform and document a restore test at least quarterly. Record the achieved recovery-point objective and recovery-time objective.

Managed data services¶

infra/aws/data.yml creates new Multi-AZ RDS PostgreSQL and ElastiCache Redis resources in supplied private subnets. It is intended for a planned migration, not an in-place update of manually created databases.

Before deploying it:

1. confirm the PostgreSQL engine version available in ap-south-1;
2. estimate RDS, ElastiCache, backup, and data-transfer costs;
3. plan replication/cutover and rollback;
4. validate application compatibility with Redis TLS;
5. restore production data into a non-production target and run functional tests.

Manual operations¶

Application startup never seeds production. When an explicitly reviewed seed is required:

MIGRATION_IMAGE='ghcr.io/dichit-money/dichit-backend@sha256:...' \
  /opt/dichit/scripts/seed.prod.sh

Every migration must follow expand-and-contract rules so the previous application digest remains compatible during an automatic rollback.