The "Database size" stat had no thresholds, so Grafana applied its stat
default (green base, red at >=80). The query is pg_database_size_bytes, so a
healthy ~9 MiB database (9.4M >> 80) rendered permanently RED on the
Scrabble - Resources dashboard (test + prod), reading as a false alert; the
neighbouring percentunit cache-hit stat stayed green only because its 0..1
values fall under 80.
Add absolute byte thresholds sized to the 40 GiB prod disk (4.6 GiB used,
observability bounded -- Tempo <=1 GiB, Prometheus 7d -- so the DB is the
only unbounded grower): green up to 8 GiB, yellow at 8 GiB (~20% of disk),
red at 16 GiB (~40%), an early warning with ample runway before the disk
tightens, not a panic line. Cosmetic panel coloring only; there are no
Grafana alert rules provisioned.
Caddy enables HTTP/3 by default on any TLS listener and emits
Alt-Svc: h3=":443"; ma=2592000, but UDP/443 is never reachable: the prod
compose maps only "443:443" (TCP) and ufw opens 443/tcp (test contour: the
host caddy publishes only :443/tcp). A client that cached the 30-day advert
tries QUIC first on later opens, gets no response, and waits for the QUIC
attempt to time out before falling back to h2 -- which surfaced as the
Telegram Mini App intermittently hanging on load (a barely-noticeable pause
up to a blank window). The h2/TCP serving path itself is healthy (~10ms TTFB).
Emit Alt-Svc: clear site-wide at the contour caddy so clients actively drop
any cached alternative and stay on h2/h1. This caddy terminates TLS in prod
(the fix target); in the test contour it serves plain :80 and the host caddy
re-stamps its own Alt-Svc, so the live test fix lives in the host caddy. Add
docs/EDGE_HTTP3.md (symptom, diagnosis method, verify, and option B -- serving
h3 for real -- if it recurs) and link it from ARCHITECTURE.md.
developer
Ilia Denisov