feat(deploy): continuous WAL archiving to S3 for point-in-time recovery
CI / changes (pull_request) Successful in 4s
CI / unit (pull_request) Successful in 12s
CI / integration (pull_request) Successful in 24s
CI / ui (pull_request) Successful in 1m9s
CI / conformance (pull_request) Successful in 10s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m41s
CI / changes (pull_request) Successful in 4s
CI / unit (pull_request) Successful in 12s
CI / integration (pull_request) Successful in 24s
CI / ui (pull_request) Successful in 1m9s
CI / conformance (pull_request) Successful in 10s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m41s
Add pgBackRest-based PITR for the production database, shipped disarmed (archive_mode and the base-backup timer gated off) so it stays inert until the operator arms it before real payments — an un-armed deploy cannot pile WAL onto the disk. - Postgres now builds from a thin image carrying pgBackRest (archive_command runs in-process); the prod overlay wires an encrypted (AES-256-CBC), path-style S3 repository with 30-day retention and a daily full base-backup systemd timer. - Repository config/secrets flow through the PROD_PGBACKREST_* Gitea set and write-prod-env.sh; prod-rollback re-renders the same env so a rollback cannot disarm archiving. - Grafana alerts watch pg_stat_archiver (failing / stalled), absent-safe on the test contour, which never archives. - Fix the pre-migration pg_dump to snapshot the whole database (was backend-only, silently excluding payments). - Document the PITR runbook, the arming sequence and the restore drill in deploy/README.md; record the measured cost/perf assessment.
This commit is contained in:
@@ -45,6 +45,42 @@ services:
|
||||
memory: 384M
|
||||
|
||||
postgres:
|
||||
image: ${REGISTRY:?set REGISTRY}/scrabble-postgres:${TAG:?set TAG}
|
||||
# Continuous WAL archiving to S3 via pgBackRest (point-in-time recovery), on the prod
|
||||
# main host only. archive_mode is gated by PGBACKREST_ARCHIVE_MODE (default "off"), so
|
||||
# merging/redeploying this stack does NOT start archiving until the operator arms it:
|
||||
# archiving stays inert — and cannot pile WAL onto the disk — until the S3 repository is
|
||||
# set up, the stanza is created and the switch is flipped on. See deploy/README.md
|
||||
# (point-in-time recovery — arming). Non-secret settings are inline; the endpoint,
|
||||
# bucket, region, S3 keys and repository cipher passphrase come from the prod env.sh
|
||||
# (deploy/write-prod-env.sh, PROD_ secrets/variables).
|
||||
environment:
|
||||
PGBACKREST_STANZA: scrabble
|
||||
PGBACKREST_PG1_PATH: /var/lib/postgresql/data
|
||||
PGBACKREST_REPO1_TYPE: s3
|
||||
PGBACKREST_REPO1_PATH: /pgbackrest
|
||||
PGBACKREST_REPO1_S3_URI_STYLE: path
|
||||
PGBACKREST_REPO1_S3_ENDPOINT: ${PGBACKREST_S3_ENDPOINT:-}
|
||||
PGBACKREST_REPO1_S3_BUCKET: ${PGBACKREST_S3_BUCKET:-}
|
||||
PGBACKREST_REPO1_S3_REGION: ${PGBACKREST_S3_REGION:-}
|
||||
PGBACKREST_REPO1_S3_KEY: ${PGBACKREST_S3_KEY:-}
|
||||
PGBACKREST_REPO1_S3_KEY_SECRET: ${PGBACKREST_S3_KEY_SECRET:-}
|
||||
PGBACKREST_REPO1_CIPHER_TYPE: aes-256-cbc
|
||||
PGBACKREST_REPO1_CIPHER_PASS: ${PGBACKREST_CIPHER_PASS:-}
|
||||
PGBACKREST_REPO1_RETENTION_FULL: "30"
|
||||
PGBACKREST_COMPRESS_TYPE: zst
|
||||
PGBACKREST_LOG_LEVEL_CONSOLE: info
|
||||
PGBACKREST_LOG_LEVEL_FILE: "off"
|
||||
# archive_command is inert while archive_mode is off, so it is always present and only
|
||||
# the mode switches. A forced 5-minute segment switch bounds the recovery point.
|
||||
command:
|
||||
- postgres
|
||||
- -c
|
||||
- archive_mode=${PGBACKREST_ARCHIVE_MODE:-off}
|
||||
- -c
|
||||
- "archive_command=pgbackrest --stanza=scrabble archive-push %p"
|
||||
- -c
|
||||
- archive_timeout=300
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
|
||||
Reference in New Issue
Block a user