feat(deploy): prod OOM swap cushion + edge maintenance page
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 18s
CI / ui (pull_request) Successful in 1m5s
CI / conformance (pull_request) Successful in 9s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m39s
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 18s
CI / ui (pull_request) Successful in 1m5s
CI / conformance (pull_request) Successful in 9s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m39s
Two prod-deploy hardening measures (owner-requested): - Swap file (Ansible common role, swap_size=1G, vm.swappiness=10). The per-container memory caps enforce that one service can't eat all RAM, but they overcommit the 1.9 GiB main host (~2.8 GiB of caps), so a simultaneous spike could hit the kernel OOM-killer (and it might pick postgres). A small swap absorbs the overshoot. Idempotent, builtin-only (no ansible.posix). - Edge maintenance page. prod-deploy.sh raises a flag around the rolling swap / migration window that the caddy edge serves a static 503 "технические работы" page from (deploy/caddy/maintenance.html), for the user-facing routes only — /_gm (Grafana) stays reachable. Cleared on any exit (success, health failure + rollback, or error) by a shell trap so it can never stick on. The 503 carries Retry-After + an X-Scrabble-Maintenance marker so a follow-up SPA overlay can tell a planned window apart from a transient error (the static page only catches a fresh load; an in-session user needs the app-side overlay). Not zero-downtime — the single stateful backend still blips — but the window is graceful instead of raw 502s. Verified: caddy validate; the gate 503s every non-/_gm path incl. the Connect/gRPC edge and serves the page + markers; /_gm bypasses; toggling needs no reload (per-request stat). Ansible --syntax-check + compose config (base+prod) pass.
This commit is contained in:
@@ -428,6 +428,11 @@ services:
|
||||
GM_BASICAUTH_HASH: ${GM_BASICAUTH_HASH:?set GM_BASICAUTH_HASH}
|
||||
volumes:
|
||||
- ${SCRABBLE_CONFIG_DIR:-.}/caddy/Caddyfile:/etc/caddy/Caddyfile:ro
|
||||
# Maintenance page + toggle flag: the caddy config dir holds maintenance.html and the
|
||||
# `on` flag prod-deploy.sh touches around a rolling swap; the Caddyfile serves a 503
|
||||
# from here while the flag exists (read-only mount — the deploy writes the flag on the
|
||||
# host side). See deploy/caddy/Caddyfile and deploy/prod-deploy.sh.
|
||||
- ${SCRABBLE_CONFIG_DIR:-.}/caddy:/srv/maint:ro
|
||||
- caddy-data:/data
|
||||
deploy:
|
||||
resources:
|
||||
|
||||
Reference in New Issue
Block a user