The edge request-latency histogram records seconds but used the OTel SDK's
default millisecond-calibrated bucket boundaries (first boundary 5), so every
sub-5s request fell into one bucket and histogram_quantile(0.99) interpolated
to ~4.95s for every message type — regardless of the real (millisecond)
latency. That tripped the >1s "Gateway request latency p99 high" alert on
essentially every request, flapping fire/resolve on each app open (seen on the
test and prod contours).
Set explicit second-scale bucket boundaries (0.005 … 10s) straddling the 1s SLO
so the p99 reflects real latency and the alert fires only on genuine slowness.
Regression test asserts the histogram carries a sub-second boundary.
The index.html boot guard now fires a fire-and-forget beacon (POST /telemetry/unsupported)
when it turns a client away on the unsupported-engine screen, so the owner can see — on the
Users dashboard beside "app opens" — how many real clients hit it and on which engines.
- Client: navigator.sendBeacon (fetch fallback) with a localStorage dedup keyed by app version
+ reason + Chromium, so a user reopening the app is one report, not ten.
- Gateway: a new unauthenticated POST /telemetry/unsupported handler (the client never booted,
so it carries no session), per-IP public-limited and body-capped, mirroring the export-download
route. It folds the beacon into the OTel counter unsupported_engine_total {reason =
no_bigint/no_proxy/boot_error/other, chromium}, with reason allow-listed and the Chromium major
reduced to a bounded range (normalizeUnsupported) so a spoofed beacon cannot inflate the metric
cardinality; the full user agent is logged, not labelled.
- Caddy: /telemetry/* added to the @gateway matcher (else it falls to the landing catch-all).
- Grafana: two panels on the Scrabble — Users dashboard (by reason, by Chromium major).
- Docs: ARCHITECTURE.md §11.
Tests: recordUnsupportedEngine (metric split), normalizeUnsupported (cardinality bounding), and
the handler route end to end (204 / 405 / counter). go build+vet+test and gofmt clean; ui
check/build/e2e green; the client beacon + dedup verified against a forced hard-gate (BigInt
removed) — one POST, deduped on reopen, correct reason/chromium/version labels.
Add a prod-only, in-memory IP ban enforced at the edge, fed by three signals:
sustained rate-limiter rejections (the IP-keyed public/email/admin classes — the
user class stays the backend soft-flag's concern), a honeypot decoy-path hit (the
contour caddy tags decoys with X-Scrabble-Honeypot and routes them to the gateway),
and a honeytoken (a planted bearer, GATEWAY_HONEYTOKEN). A banned IP is refused with
429 by the abuseGuard middleware before any work — covering the Connect edge, the
live stream and the static SPA/landing the per-op limiter never gated.
The ban is off by default: it keys by the real client IP the shared-NAT test contour
does not expose, so a ban there would be self-inflicted; detection still logs in the
contour, only the ban action is gated (GATEWAY_ABUSE_BAN_ENABLED). Rejection bans last
GATEWAY_ABUSE_BAN_DURATION; tripwire/honeytoken hits are near-zero-false-positive and
earn longer fixed bans. Each ban increments gateway_abuse_banned_total{reason}.
Operators see and lift active bans on the admin console's Throttled page; the gateway
syncs its active set to the backend every 30s (POST /api/v1/internal/bans/sync,
backend/internal/banview) and applies the operator unbans the response returns.
PRERELEASE phase AG. Docs baked into ARCHITECTURE / FUNCTIONAL (+ru) / both READMEs.
- GATEWAY_MAX_BODY_BYTES (1 MiB): connect WithReadMaxBytes + http.MaxBytesReader
on the public mux; explicit http2.Server MaxConcurrentStreams/IdleTimeout and
an http.Server ReadHeaderTimeout (R2 report follow-up).
- gateway_rate_limited_total{class} counter, Debug per rejection, a rejection
tracker drained every 30 s into a Warn summary per key and a report POST to
/api/v1/internal/ratelimit/report (feeds the admin view + auto-flag).
- The dead AdminPerMinute/AdminBurst policy now guards the /_gm mount (429),
ahead of its Basic-Auth.
- resolve() logs the cause of infra session-resolve failures at Warn (the
transient unauthenticated dips from the R2 run); unknown tokens stay silent.
Ilia Denisov