Skip to content

Monitoring

How dichit-backend is observed in production and during development.

Observability stack

Tool What it covers
Sentry errors, exceptions, performance traces, profiles
Pino logs → CloudWatch structured request/app logs (see logging.md)
Bull Board live queue/job inspection
Health/meta routes platform /health, /meta, /sentry
Slack webhook proactive DevOps alerts (e.g. startup failure)

Sentry

  • Initialized in src/interfaces/init.ts (initSentry) before the app boots.
  • Fastify error handler installed in server.ts.
  • Env: SENTRY_ENABLED, SENTRY_DSN, SENTRY_ORG, SENTRY_PROJECT, SENTRY_AUTH_TOKEN, SENTRY_TRACE_RATE, SENTRY_PROFILE_RATE.
  • Source maps are uploaded at build time (sentry:sourcemaps).

Use

  1. Open the project dashboard for the failing release.
  2. Filter by reqId (correlation id) to jump to the matching CloudWatch logs.
  3. Read performance to find slow spans (DB/queue).

Bull Board

  • Default path /admin/queues (see BULL_BOARD_BASE_PATH, credentials via BULL_BOARD_USERNAME / BULL_BOARD_PASSWORD).
  • Inspect failed jobs, retry, or delete stuck jobs.
  • Registered from src/interfaces/app.ts (registerBullBoard).

Logs

  • Pino structured logs with request/correlation ids shipped to CloudWatch (see logging.md).
  • Never log secrets/OTPs/tokens.

Alerts

  • Startup failures notify the DevOps Slack channel (notifyDevOpsTeam → SLACK_WEBHOOK_URL).
  • Deploy pipelines gate on CI checks (see ci-cd.md).
  • Add runbook-driven alerts per runbooks/README.md.

Alert flow

flowchart LR
    APP["App"] -->|"error"| S["Sentry"]
    APP -->|"logs"| CW["CloudWatch"]
    APP -->|"startup failure"| SL["Slack"]
    Q["BullMQ"] -->|"failed jobs"| BB["Bull Board"]
    S --> DEV["On-call engineer"]