docker exec defaults to root, so the systemd base-backup timer connected to the database as role "root" (then "postgres") — neither exists; the superuser role is POSTGRES_USER (scrabble). Run the timer's pgBackRest as the postgres OS user (-u postgres, for lock-dir/PGDATA consistency with archive-push) and connect with --pg1-user=scrabble. archive_command (run by the postgres server process) was already correct. Also record the point-in-time-recovery arming + restore drill in deploy/README.md (correct the runbook commands to the same invocation, fill the drill log) and mark the durability work done in PLAN.md.
Prod host provisioning
Idempotent Ansible that prepares the two production hosts. It installs Docker, a
non-sudo deploy service account, SSH hardening, a default-deny firewall,
fail2ban, unattended security upgrades and time sync. It does not deploy the
application — that is .gitea/workflows/prod-deploy.yaml's job, running as the
deploy account this playbook creates.
Hosts are referenced by ~/.ssh/config aliases (scrabble-main-ops,
scrabble-tg-ops), so no IPs or key paths live in the repo.
Prerequisites (controller)
ansiblewith the bundled collections (community.general,community.docker,ansible.posix).- The two hosts reachable as root via the ssh-config aliases, host keys already
accepted into
known_hosts(host_key_checking = True).
One-time: the CI deploy key
The CI prod-deploy workflow logs into the hosts as deploy using a dedicated
key. Generate it once on the controller, authorize its public half via the
playbook, and store its private half only in the Gitea PROD_SSH_KEY secret:
ssh-keygen -t ed25519 -N '' -C scrabble-ci-deploy \
-f ~/.ssh/scrabble_ci_deploy_ed25519
# private half -> Gitea secret PROD_SSH_KEY (set via API); never commit it
Run
cd deploy/ansible
ansible-playbook site.yml
The playbook reads the public key from ~/.ssh/scrabble_ci_deploy_ed25519.pub by
default; override with -e deploy_ci_pubkey_path=/path/to/key.pub. Re-running is
safe (idempotent) and survives a host resize.
What each host gets
- both (
common): docker-ce + compose plugin,daemon.json(live-restore, 10m×3 log rotation),deployuser (docker group, no sudo), key-only sshd,ufwdefault-deny incoming + allow SSH, fail2ban sshd jail, unattended upgrades, chrony,/opt/scrabble/{config,certs,dumps,images}. - main:
ufwopens 80/443/9443; the externaledgedocker network; the pgBackRest daily base-backup systemd timer — installed only whenpitr_enabled=true(off by default; turned on as part of arming point-in-time recovery, see../README.md). - tg: verifies direct
api.telegram.orgegress (the no-VPN assumption).