feat(deploy): prod OOM swap cushion + edge maintenance page

Two prod-deploy hardening measures (owner-requested): - Swap file (Ansible common role, swap_size=1G, vm.swappiness=10). The per-container memory caps enforce that one service can't eat all RAM, but they overcommit the 1.9 GiB main host (~2.8 GiB of caps), so a simultaneous spike could hit the kernel OOM-killer (and it might pick postgres). A small swap absorbs the overshoot. Idempotent, builtin-only (no ansible.posix). - Edge maintenance page. prod-deploy.sh raises a flag around the rolling swap / migration window that the caddy edge serves a static 503 "технические работы" page from (deploy/caddy/maintenance.html), for the user-facing routes only — /_gm (Grafana) stays reachable. Cleared on any exit (success, health failure + rollback, or error) by a shell trap so it can never stick on. The 503 carries Retry-After + an X-Scrabble-Maintenance marker so a follow-up SPA overlay can tell a planned window apart from a transient error (the static page only catches a fresh load; an in-session user needs the app-side overlay). Not zero-downtime — the single stateful backend still blips — but the window is graceful instead of raw 502s. Verified: caddy validate; the gate 503s every non-/_gm path incl. the Connect/gRPC edge and serves the page + markers; /_gm bypasses; toggling needs no reload (per-request stat). Ansible --syntax-check + compose config (base+prod) pass.
2026-07-03 22:09:37 +02:00
parent 4b4dcab9b6
commit c8601c0115
7 changed files with 172 additions and 1 deletions
@@ -33,6 +33,39 @@
 	# is the prod fix. Background + alternatives (incl. serving h3 for real): docs/EDGE_HTTP3.md.
 	header Alt-Svc clear

+	# Maintenance gate. While the deploy holds the flag file /srv/maint/on (touched by
+	# prod-deploy.sh around a rolling swap, removed on the script's exit), serve a static
+	# 503 "works in progress" page instead of proxying to a mid-recreate upstream —
+	# a graceful signal rather than raw 502s. Operator surfaces (/_gm) are excluded so
+	# Grafana stays reachable during the window. Caddy stat()s the flag per request, so
+	# toggling it needs no reload; the page + flag live in the caddy config dir bind-mounted
+	# read-only at /srv/maint (docker-compose.yml). The test contour never sets the flag
+	# (only prod-deploy.sh does), so this is inert there.
+	@maintenance {
+		not path /_gm /_gm/*
+		file {
+			root /srv/maint
+			try_files on
+		}
+	}
+	handle @maintenance {
+		error 503
+	}
+	handle_errors {
+		@maint503 expression {err.status_code} == 503
+		handle @maint503 {
+			root * /srv/maint
+			rewrite * /maintenance.html
+			header Retry-After 120
+			# Distinctive maintenance marker so the SPA can tell a planned window apart from
+			# a transient upstream 503 and show its own dimmed overlay (an in-session user
+			# never sees this static page — it only renders on a fresh load). The header
+			# rides every gated response, incl. the Connect/gRPC edge the app polls.
+			header X-Scrabble-Maintenance 1
+			file_server
+		}
+	}
+
 	# Operator surfaces under /_gm: a single shared Basic-Auth, then route.
 	@gm path /_gm /_gm/*
 	handle @gm {
@@ -0,0 +1,44 @@
+<!doctype html>
+<!--
+  Served with 503 by the edge caddy while the deploy holds the maintenance flag
+  (/srv/maint/on, toggled by deploy/prod-deploy.sh around a rolling swap). Static and
+  self-contained (no upstream, no external assets) so it renders while the backend /
+  gateway are mid-recreate.
+-->
+<html lang="ru">
+<head>
+  <meta charset="utf-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <meta name="robots" content="noindex">
+  <title>Эрудит — технические работы</title>
+  <style>
+    :root { color-scheme: light dark; }
+    html, body { height: 100%; margin: 0; }
+    body {
+      display: flex; align-items: center; justify-content: center;
+      font-family: system-ui, -apple-system, "Segoe UI", Roboto, sans-serif;
+      background: #f4f1ea; color: #2b2b2b;
+      padding: 24px; box-sizing: border-box;
+    }
+    @media (prefers-color-scheme: dark) { body { background: #1d1b17; color: #e8e4da; } }
+    .card { max-width: 30rem; text-align: center; line-height: 1.5; }
+    .tile {
+      display: inline-block; width: 3.5rem; height: 3.5rem; line-height: 3.5rem;
+      font-size: 2rem; font-weight: 700; border-radius: 10px;
+      background: #d9b451; color: #2b2b2b; box-shadow: 0 2px 4px rgba(0,0,0,.25);
+      margin-bottom: 1.25rem;
+    }
+    h1 { font-size: 1.4rem; margin: 0 0 .5rem; }
+    p { margin: .35rem 0; }
+    .en { opacity: .7; font-size: .95rem; }
+  </style>
+</head>
+<body>
+  <main class="card">
+    <div class="tile">Э</div>
+    <h1>Технические работы</h1>
+    <p>Идёт короткое обновление игры. Мы скоро вернёмся — обновите страницу через пару минут.</p>
+    <p class="en">Scrabble is briefly down for an update. Please refresh in a couple of minutes.</p>
+  </main>
+</body>
+</html>