The tg (bot) VPS came up on Europe/Moscow while the main host is UTC, so host-level timestamps (journald, file mtimes, cron) sat 3h apart across the fleet. Add a community.general.timezone task to the common role so every provisioned host is UTC (idempotent — a no-op on the already-UTC main host). Applied live to the tg host in the same change; the containerised services run in UTC regardless, so no restart.
Prod host provisioning
Idempotent Ansible that prepares the two production hosts. It installs Docker, a
non-sudo deploy service account, SSH hardening, a default-deny firewall,
fail2ban, unattended security upgrades and time sync. It does not deploy the
application — that is .gitea/workflows/prod-deploy.yaml's job, running as the
deploy account this playbook creates.
Hosts are referenced by ~/.ssh/config aliases (scrabble-main-ops,
scrabble-tg-ops), so no IPs or key paths live in the repo.
Prerequisites (controller)
ansiblewith the bundled collections (community.general,community.docker,ansible.posix).- The two hosts reachable as root via the ssh-config aliases, host keys already
accepted into
known_hosts(host_key_checking = True).
One-time: the CI deploy key
The CI prod-deploy workflow logs into the hosts as deploy using a dedicated
key. Generate it once on the controller, authorize its public half via the
playbook, and store its private half only in the Gitea PROD_SSH_KEY secret:
ssh-keygen -t ed25519 -N '' -C scrabble-ci-deploy \
-f ~/.ssh/scrabble_ci_deploy_ed25519
# private half -> Gitea secret PROD_SSH_KEY (set via API); never commit it
Run
cd deploy/ansible
ansible-playbook site.yml
The playbook reads the public key from ~/.ssh/scrabble_ci_deploy_ed25519.pub by
default; override with -e deploy_ci_pubkey_path=/path/to/key.pub. Re-running is
safe (idempotent) and survives a host resize.
What each host gets
- both (
common): docker-ce + compose plugin,daemon.json(live-restore, 10m×3 log rotation),deployuser (docker group, no sudo), key-only sshd,ufwdefault-deny incoming + allow SSH, fail2ban sshd jail, unattended upgrades, chrony,/opt/scrabble/{config,certs,dumps,images}. - main:
ufwopens 80/443/9443; the externaledgedocker network; the pgBackRest daily base-backup systemd timer — installed only whenpitr_enabled=true(off by default; turned on as part of arming point-in-time recovery, see../README.md). - tg: verifies direct
api.telegram.orgegress (the no-VPN assumption).