feat(deploy): continuous WAL archiving to S3 for point-in-time recovery
CI / changes (pull_request) Successful in 4s
CI / unit (pull_request) Successful in 12s
CI / integration (pull_request) Successful in 24s
CI / ui (pull_request) Successful in 1m9s
CI / conformance (pull_request) Successful in 10s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m41s
CI / changes (pull_request) Successful in 4s
CI / unit (pull_request) Successful in 12s
CI / integration (pull_request) Successful in 24s
CI / ui (pull_request) Successful in 1m9s
CI / conformance (pull_request) Successful in 10s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m41s
Add pgBackRest-based PITR for the production database, shipped disarmed (archive_mode and the base-backup timer gated off) so it stays inert until the operator arms it before real payments — an un-armed deploy cannot pile WAL onto the disk. - Postgres now builds from a thin image carrying pgBackRest (archive_command runs in-process); the prod overlay wires an encrypted (AES-256-CBC), path-style S3 repository with 30-day retention and a daily full base-backup systemd timer. - Repository config/secrets flow through the PROD_PGBACKREST_* Gitea set and write-prod-env.sh; prod-rollback re-renders the same env so a rollback cannot disarm archiving. - Grafana alerts watch pg_stat_archiver (failing / stalled), absent-safe on the test contour, which never archives. - Fix the pre-migration pg_dump to snapshot the whole database (was backend-only, silently excluding payments). - Document the PITR runbook, the arming sequence and the restore drill in deploy/README.md; record the measured cost/perf assessment.
This commit is contained in:
@@ -140,7 +140,10 @@ if [ "$MIGRATION" = 1 ]; then
|
||||
echo "migration deploy: opening maintenance window (stopping the backend = the only writer)"
|
||||
dc stop backend
|
||||
dump="$DUMP_DIR/pre-$TAG-$(date +%Y%m%d-%H%M%S).sql"
|
||||
if ! docker exec scrabble-postgres pg_dump -U "$PG_USER" -d "$PG_DB" -n backend > "$dump"; then
|
||||
# Dump the WHOLE database (no -n): the payments schema — and any schema added later —
|
||||
# must be in the belt-and-braces snapshot, not just backend. Restored manually into the
|
||||
# same instance only if a migration turned out destructive (see deploy/README.md).
|
||||
if ! docker exec scrabble-postgres pg_dump -U "$PG_USER" -d "$PG_DB" > "$dump"; then
|
||||
echo "pg_dump failed; restarting the old backend and aborting"
|
||||
dc start backend
|
||||
exit 1
|
||||
|
||||
Reference in New Issue
Block a user