feat(deploy): continuous WAL archiving to S3 for point-in-time recovery
CI / changes (pull_request) Successful in 4s
CI / unit (pull_request) Successful in 12s
CI / integration (pull_request) Successful in 24s
CI / ui (pull_request) Successful in 1m9s
CI / conformance (pull_request) Successful in 10s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m41s
CI / changes (pull_request) Successful in 4s
CI / unit (pull_request) Successful in 12s
CI / integration (pull_request) Successful in 24s
CI / ui (pull_request) Successful in 1m9s
CI / conformance (pull_request) Successful in 10s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m41s
Add pgBackRest-based PITR for the production database, shipped disarmed (archive_mode and the base-backup timer gated off) so it stays inert until the operator arms it before real payments — an un-armed deploy cannot pile WAL onto the disk. - Postgres now builds from a thin image carrying pgBackRest (archive_command runs in-process); the prod overlay wires an encrypted (AES-256-CBC), path-style S3 repository with 30-day retention and a daily full base-backup systemd timer. - Repository config/secrets flow through the PROD_PGBACKREST_* Gitea set and write-prod-env.sh; prod-rollback re-renders the same env so a rollback cannot disarm archiving. - Grafana alerts watch pg_stat_archiver (failing / stalled), absent-safe on the test contour, which never archives. - Fix the pre-migration pg_dump to snapshot the whole database (was backend-only, silently excluding payments). - Document the PITR runbook, the arming sequence and the restore drill in deploy/README.md; record the measured cost/perf assessment.
This commit is contained in:
@@ -77,7 +77,7 @@ jobs:
|
||||
# The main-stack images via compose (reuses the build args, incl. VERSION);
|
||||
# the bot separately, since it is profiled out of the prod compose.
|
||||
docker compose -f docker-compose.yml -f docker-compose.prod.yml build
|
||||
docker compose -f docker-compose.yml -f docker-compose.prod.yml push backend gateway landing validator renderer
|
||||
docker compose -f docker-compose.yml -f docker-compose.prod.yml push postgres backend gateway landing validator renderer
|
||||
docker build -f ../platform/telegram/Dockerfile --target bot --build-arg VERSION="$TAG" -t "$REGISTRY/scrabble-telegram-bot:$TAG" ..
|
||||
docker push "$REGISTRY/scrabble-telegram-bot:$TAG"
|
||||
|
||||
@@ -135,6 +135,17 @@ jobs:
|
||||
DICT_VERSION: ${{ vars.DICT_VERSION }}
|
||||
POSTGRES_DB: ${{ vars.PROD_POSTGRES_DB }}
|
||||
POSTGRES_USER: ${{ vars.PROD_POSTGRES_USER }}
|
||||
# Point-in-time recovery (pgBackRest -> S3), prod main host only. Endpoint/bucket/
|
||||
# region + the archive-mode switch are variables; the S3 keys + the repository cipher
|
||||
# passphrase are secrets. All empty/off until the operator arms archiving; flipping
|
||||
# PROD_PGBACKREST_ARCHIVE_MODE=on activates it (deploy/README.md, arming).
|
||||
PGBACKREST_ARCHIVE_MODE: ${{ vars.PROD_PGBACKREST_ARCHIVE_MODE }}
|
||||
PGBACKREST_S3_ENDPOINT: ${{ vars.PROD_PGBACKREST_S3_ENDPOINT }}
|
||||
PGBACKREST_S3_BUCKET: ${{ vars.PROD_PGBACKREST_S3_BUCKET }}
|
||||
PGBACKREST_S3_REGION: ${{ vars.PROD_PGBACKREST_S3_REGION }}
|
||||
PGBACKREST_S3_KEY: ${{ secrets.PROD_PGBACKREST_S3_KEY }}
|
||||
PGBACKREST_S3_KEY_SECRET: ${{ secrets.PROD_PGBACKREST_S3_KEY_SECRET }}
|
||||
PGBACKREST_CIPHER_PASS: ${{ secrets.PROD_PGBACKREST_CIPHER_PASS }}
|
||||
# TELEGRAM_MINIAPP_URL and GRAFANA_ROOT_URL are derived from PUBLIC_BASE_URL in
|
||||
# deploy/write-prod-env.sh, not stored variables.
|
||||
steps:
|
||||
|
||||
Reference in New Issue
Block a user