fix(deploy): run the base-backup timer's pgBackRest as the postgres role
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 18s
CI / ui (pull_request) Successful in 1m9s
CI / conformance (pull_request) Successful in 9s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m46s
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 18s
CI / ui (pull_request) Successful in 1m9s
CI / conformance (pull_request) Successful in 9s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m46s
docker exec defaults to root, so the systemd base-backup timer connected to the database as role "root" (then "postgres") — neither exists; the superuser role is POSTGRES_USER (scrabble). Run the timer's pgBackRest as the postgres OS user (-u postgres, for lock-dir/PGDATA consistency with archive-push) and connect with --pg1-user=scrabble. archive_command (run by the postgres server process) was already correct. Also record the point-in-time-recovery arming + restore drill in deploy/README.md (correct the runbook commands to the same invocation, fill the drill log) and mark the durability work done in PLAN.md.
This commit is contained in:
@@ -33,7 +33,7 @@ status — without re-deriving decisions.
|
||||
| E1 | Trusted platform signal | 1 | DONE |
|
||||
| E2 | Currency + benefit core | 1 | DONE |
|
||||
| E3 | Wallet UI | 1 | DONE |
|
||||
| E4 | Durability (PITR) | 2 | WIP |
|
||||
| E4 | Durability (PITR) | 2 | DONE |
|
||||
| E5 | Payment intake | 2 | TODO |
|
||||
| E6 | Ads | 2 | TODO |
|
||||
| E7 | Admin & reports | 2 | TODO |
|
||||
@@ -446,8 +446,8 @@ noun agreement). Svelte whitespace/`$state` naming gotchas apply.
|
||||
|
||||
## E4 — Durability (PITR)
|
||||
|
||||
**Status:** WIP — repo artifacts landed; **prod arming pending** (owner-coordinated, before
|
||||
E5). · **Release 2** · depends on: E0 (schema exists) · mechanics: PAYMENTS §14 (D4).
|
||||
**Status:** DONE — armed + restore-drilled on prod (v1.13.0, 2026-07-09). · **Release 2** ·
|
||||
depends on: E0 (schema exists) · mechanics: PAYMENTS §14 (D4).
|
||||
|
||||
**Goal.** Continuous WAL archiving with point-in-time recovery, armed **before the first
|
||||
real money** is accepted (E5 prod). Protects both money and game data.
|
||||
@@ -479,24 +479,26 @@ real money** is accepted (E5 prod). Protects both money and game data.
|
||||
`-n backend`, silently excluding `payments`); manual-restore runbook updated.
|
||||
- Full PITR runbook + arming sequence + recorded assessment in `deploy/README.md`.
|
||||
|
||||
**Prod arming (SEPARATE — owner-coordinated, before E5; NOT the artifacts PR).** Owner creates
|
||||
the Selectel S3 bucket + the `PROD_PGBACKREST_*` secrets/variables (incl. `ARCHIVE_MODE=on`);
|
||||
then promote `development → master` → `prod-deploy` (archive_mode on behind the maintenance
|
||||
window), `pgbackrest stanza-create` + first base backup + `check`, re-run Ansible with
|
||||
`-e pitr_enabled=true`, and run the restore drill on an isolated one-shot target. Exact steps:
|
||||
`deploy/README.md` (point-in-time recovery — arming).
|
||||
**Prod arming (completed 2026-07-09 with the v1.13.0 release).** Owner created the Selectel S3
|
||||
bucket (`erudite`, ru-6) + the `PROD_PGBACKREST_*` secrets/variables (incl. `ARCHIVE_MODE=on`);
|
||||
promoted `development → master` → `prod-deploy` (archive_mode on behind the maintenance window),
|
||||
then `stanza-create` + first base backup (31.9 MB cluster → 3.7 MB in the repo) + `check`, the
|
||||
Ansible `-e pitr_enabled=true` timer, and a restore drill on an isolated one-shot target (data
|
||||
intact, then wiped). Exact steps: `deploy/README.md` (point-in-time recovery — arming).
|
||||
|
||||
**Tests.** Restore drill on an isolated one-shot instance (base + WAL → target timestamp),
|
||||
recorded in `deploy/README.md`. No app-level tests. Local verification: `docker compose config`
|
||||
valid + the custom PG image builds (archiving inert on the contour).
|
||||
|
||||
**Done-criteria.** Artifacts merged with archiving inert. **Fully done** at arming: WAL
|
||||
archiving live on prod PG; a timed restore verified; cost + perf assessment reviewed; runbook
|
||||
current in `deploy/README.md`.
|
||||
**Done-criteria (met).** WAL archiving live on prod PG (v1.13.0); a restore verified on an
|
||||
isolated one-shot target; cost + perf assessment reviewed; runbook current in
|
||||
`deploy/README.md`.
|
||||
|
||||
**Notes/risks.** The repository **cipher passphrase is unrecoverable if lost** — stored apart
|
||||
from the S3 keys. Enabling `archive_mode` restarts postgres (rides the prod-deploy maintenance
|
||||
window). Migrations stay expand-contract so image rollback remains DB-safe alongside PITR.
|
||||
from the S3 keys. Manual/timer pgBackRest runs use `docker exec -u postgres … --pg1-user=scrabble`
|
||||
(docker exec is root; the DB superuser role is `scrabble`, not `postgres`) — the systemd timer
|
||||
+ the runbook carry this. Enabling `archive_mode` restarts postgres (rode the prod-deploy
|
||||
maintenance window). Migrations stay expand-contract so image rollback remains DB-safe with PITR.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user