Ilia Denisov
c8601c0115
Two prod-deploy hardening measures (owner-requested): - Swap file (Ansible common role, swap_size=1G, vm.swappiness=10). The per-container memory caps enforce that one service can't eat all RAM, but they overcommit the 1.9 GiB main host (~2.8 GiB of caps), so a simultaneous spike could hit the kernel OOM-killer (and it might pick postgres). A small swap absorbs the overshoot. Idempotent, builtin-only (no ansible.posix). - Edge maintenance page. prod-deploy.sh raises a flag around the rolling swap / migration window that the caddy edge serves a static 503 "технические работы" page from (deploy/caddy/maintenance.html), for the user-facing routes only — /_gm (Grafana) stays reachable. Cleared on any exit (success, health failure + rollback, or error) by a shell trap so it can never stick on. The 503 carries Retry-After + an X-Scrabble-Maintenance marker so a follow-up SPA overlay can tell a planned window apart from a transient error (the static page only catches a fresh load; an in-session user needs the app-side overlay). Not zero-downtime — the single stateful backend still blips — but the window is graceful instead of raw 502s. Verified: caddy validate; the gate 503s every non-/_gm path incl. the Connect/gRPC edge and serves the page + markers; /_gm bypasses; toggling needs no reload (per-request stat). Ansible --syntax-check + compose config (base+prod) pass.
Prod host provisioning
Idempotent Ansible that prepares the two production hosts. It installs Docker, a
non-sudo deploy service account, SSH hardening, a default-deny firewall,
fail2ban, unattended security upgrades and time sync. It does not deploy the
application — that is .gitea/workflows/prod-deploy.yaml's job, running as the
deploy account this playbook creates.
Hosts are referenced by ~/.ssh/config aliases (scrabble-main-ops,
scrabble-tg-ops), so no IPs or key paths live in the repo.
Prerequisites (controller)
ansiblewith the bundled collections (community.general,community.docker,ansible.posix).- The two hosts reachable as root via the ssh-config aliases, host keys already
accepted into
known_hosts(host_key_checking = True).
One-time: the CI deploy key
The CI prod-deploy workflow logs into the hosts as deploy using a dedicated
key. Generate it once on the controller, authorize its public half via the
playbook, and store its private half only in the Gitea PROD_SSH_KEY secret:
ssh-keygen -t ed25519 -N '' -C scrabble-ci-deploy \
-f ~/.ssh/scrabble_ci_deploy_ed25519
# private half -> Gitea secret PROD_SSH_KEY (set via API); never commit it
Run
cd deploy/ansible
ansible-playbook site.yml
The playbook reads the public key from ~/.ssh/scrabble_ci_deploy_ed25519.pub by
default; override with -e deploy_ci_pubkey_path=/path/to/key.pub. Re-running is
safe (idempotent) and survives a host resize.
What each host gets
- both (
common): docker-ce + compose plugin,daemon.json(live-restore, 10m×3 log rotation),deployuser (docker group, no sudo), key-only sshd,ufwdefault-deny incoming + allow SSH, fail2ban sshd jail, unattended upgrades, chrony,/opt/scrabble/{config,certs,dumps,images}. - main:
ufwopens 80/443/9443; the externaledgedocker network. - tg: verifies direct
api.telegram.orgegress (the no-VPN assumption).