The seed-drift guard shipped as refuse-boot: the backend exited when BACKEND_DICT_VERSION disagreed with the flat dir's recorded .seed_version. On the test contour that turned a harmless-in-intent action — bumping the TEST_DICT_VERSION variable to the active release (v1.2.1) on a volume seeded as v1.0.0 — into a crash loop, because DICT_VERSION is the *seed* of a fresh volume, not the active version (which the admin console drives). Make the marker authoritative instead: OpenWithVersions resolves the flat dir's version from .seed_version when present and ignores bootVersion on an already-seeded volume; bootVersion only seeds a fresh volume's marker. So a bumped build seed on a live volume is a no-op (it can't relabel live bytes and can't void games pinned to the prior label), and it correctly seeds the next fresh volume. The subdirectory scan now skips the resolved seed, so a version also present as a subdir (e.g. v1.2.1 uploaded via the console while the build seed is bumped to v1.2.1) is still loaded rather than shadowed by the flat bytes. Tests: marker-wins over a bumped boot version; a bumped boot keeps the matching subdir resident (the live-contour case). Docs updated (ARCHITECTURE §5, READMEs, compose/.env, PRERELEASE DV) from "refuses to boot" to "marker wins / ignored". Verified locally against v1.2.1: gofmt, build, vet, unit, integration green.
50 KiB
Pre-release plan — hardening before Stage 18
Living tracker for the pre-release hardening pass that runs before Stage 18 (the
prod cutover). Same discipline as PLAN.md: one phase per session,
interview the owner on the open details at the start of each phase, bake every
decision back into PLAN.md / docs/ / the affected READMEs / Go Doc comments in
the same PR, get CI green, then mark the phase done. Phases run as
feature/* → development PRs (the Stage 16 branch model); the owner approves+merges.
Why now: the system is feature-complete through Stage 17 and the test contour is green, but there is no prod data yet — schema, wire labels and the dictionary layout can still change for free. These phases spend that one-time freedom and harden the edge before prod. Each phase maps back to the owner's raw pre-release TODO list (numbers in the tracker).
Phase tracker
| # | Phase | Raw TODOs | Status |
|---|---|---|---|
| R1 | Schema & naming reset | 1 + 10 | done |
| R2 | Stress harness + contour observability + early run | 9a | done |
| R3 | Edge hardening | 2 + 8 + 3 | done |
| R4 | Push enrichment + kill the last poll | 4 + 5 | done |
| R5 | Bundle slimming | 6 | done |
| R6 | Refactor + docs reconciliation + de-staging | 7 | done |
| R7 | Final stress run + tuning | 9b | done |
| UI | Tab-bar navigation redesign (drop the hamburger) | owner ad-hoc | done |
| MW | "Multiple words per turn" rule for Russian games (engine v1.1.0) | owner ad-hoc | done |
| MW2 | Single-word rule connectivity fix: the word must run along its own line through an existing tile (perpendicular-only contact no longer connects); single-tile direction picks the best legal word (engine v1.1.1) | owner ad-hoc | done |
| MW3 | Graceful replay degradation: a game whose journalled move became illegal under MW2 is closed as a draw (end_reason='aborted') on open instead of erroring, with an impersonal organizer note in the history + GCG (migration 00002) |
owner ad-hoc | done |
| OW | Open auto-match: enter the game at once and wait inside it (robot after 90–180 s) | owner ad-hoc | done |
| DA | Dictionary admin: online release-archive upload → word-diff preview → install/activate; versioned dict volume; active version persisted in DB; resident label = release tag | owner ad-hoc | done |
| AB | Manual account block (admin suspension): permanent/temporary with an editable en+ru reason picklist; a block forfeits the player's active games + cancels their open ones; a backend gate refuses a blocked account with 403 account_blocked; the UI shows a terminal blocked screen and stops all push/poll; manual unblock; temporary blocks self-expire (migration 00003) |
owner ad-hoc | done |
| AI | Honest AI opponent in quick game: an explicit 🤖 AI / 👤 random selector (AI default); the robot is seated and moves at once; 7-day inactivity loss (the per-turn timeout reused); chat/nudge disabled, no statistics; the opponent is shown as 🤖 everywhere | owner ad-hoc | done |
| AD | Advertising banner ("ad network"): server-driven weighted campaigns (percent weight + validity window; the perpetual default fills the remainder up to 100%), bilingual messages shown by bot (service_language); eligibility = free account + empty hint wallet + no no_banner role (guests included); the resolved feed rides profile.get with a notify banner re-poll on eligibility change; /_gm/banners admin + global display timings; client smooth-weighted-round-robin rotation + fade-out/gap/fade-in UX. A single app.load bootstrap aggregator was considered and deferred (see ARCHITECTURE §10). |
owner ad-hoc | done (PR1 backend+admin, PR2 UI rotation) |
| GL | Simultaneous quick-game cap (10): grey "New Game" + a lobby notice at the cap; backend gate on quick enqueue + invitation creation (409 game_limit_reached), accepting invitations exempt; at_game_limit rides games.list |
owner ad-hoc | done |
| CR | In-game chat read receipts: per-message unread_seats bitmask (migration 00008); a per-viewer unread dot in the lobby + game header (a nudge counts and clears when its recipient moves); reading = opening the move history (the 💬 fade-blinks twice) or the chat, acked (chat.read) only when unread; chat_read_duration + chat_unread_messages metrics + tracing + the Scrabble — Messages Grafana dashboard (follow-up PR); a message to a disguised robot opponent is born read; admin unread-only filter / read column / per-seat read card |
owner ad-hoc | done |
| BX | Asymmetric per-user block + in-game controls: a block now silently suppresses everything from the blocked user (chat, nudge, friend requests, invitations are kept but never delivered/surfaced, born-read) while they notice nothing, without deleting the friendship (unblock restores it); auto-match excludes a block-related pair (either direction); in-game opponent card gains a ✖️ block control (mirroring 🤝, red "Block?" confirm, mutual-hide, struck name + hidden chat composer when blocked); optimistic apply + user_blocked/user_unblocked event confirm + rollback; admin user card gains blocks / blocked-by / friends cross-linked lists. Blocking a disguised-robot opponent is recorded per-game in a separate robot_blocks table (migration 00011), keyed on game+seat with the seen name — never the shared robot account — so the matchmaker keeps giving robots; it shows in the blocked list and re-marks the in-game card |
owner ad-hoc | done |
| FM | First-move tile draw (official rules): each seated player draws a tile, the one closest to "A" leads (a blank beats every letter), ties re-drawing until a single leader; honest per-draw crypto/rand entropy, not the bag seed, so the record (game_setup_draws, migration 00013) — not a seed — is the only account of the outcome, kept for future tournaments (designed as a discrete per-tile "player N draws" step). Friend/AI draws at create; auto-match draws at open against a synthetic uuid.Nil opponent whose draw rows are back-filled on join, so the opener's seat is fixed up front and the existing open-game pre-move is preserved (no reseating, no play-gating). Admin /_gm/games/:id gains the recorded draw list + a simple step-by-step board replay (ReplayTimeline). |
owner ad-hoc | done |
| SB | Single Telegram bot + per-user variant preferences: the two per-language bots collapse into one (drop accounts.service_language, supported_languages, the *_EN/*_RU env vars and game-language push routing — the single bot renders in the recipient's preferred_language); New Game variant gating moves to a profile variant_preferences set (default Erudit only, Erudit-first, server-enforced on the caller's auto-match/vs-AI/invitation-create paths, an invited friend may accept any variant); env vars collapse to unsuffixed TELEGRAM_BOT_TOKEN/TELEGRAM_GAME_CHANNEL_ID/VITE_TELEGRAM_LINK/VITE_TELEGRAM_GAME_CHANNEL_NAME and GATEWAY_DEFAULT_SUPPORTED_LANGUAGES is removed; wire drops service_language/supported_languages (Session, ValidateInitDataResponse) + the push language routing field and adds variant_preferences to Profile/UpdateProfile. |
owner ad-hoc | done |
| DV | Dictionary version hygiene: CI + image/compose seed track the current release (v1.2.1); a seed-drift guard records the flat dir's seed in an authoritative .seed_version marker so a bumped build seed on a live volume is ignored (it can't relabel live bytes — which would mis-serve the dictionary + void games pinned to the prior label); DICT_VERSION is the fresh-volume seed only, a live contour migrates through the admin console |
owner ad-hoc | done |
| → | Stage 18 — prod contour deploy | — | see PLAN.md |
Key findings (these reshaped the raw list — read before starting a phase)
- R1 (TODO 1 + 10) is one cheap moment, now. Squashing the 12 goose migrations is
safe precisely because there is no prod data and the contour DB is wiped. Folding the
new variant labels (
scrabble_ru/scrabble_en/erudit_ru) into that single baseline makes the rename need no data migration and no back-compat mapping. Today's labels (english/russian_scrabble/erudit) are persisted ingames.variant,game_invitations.variant, inpkg/fbsand the UI — ~100 files, but a mechanical sweep on a clean DB. - R4 (TODO 4 + 5): the app is already push-first. Game state refreshes on
your_turn/opponent_moved, the lobby onnotify, chat onchat_message. The only genuine periodic server poll islobby.poll(matchmaking, 2.5 s,ui/src/screens/NewGame.svelte). What remains is killing that one poll and enriching push events to carry payloads so the UI stops re-fetching after each signal. - R3 (TODO 2): identity forgery is already mitigated. Identity is always derived from
the session (
Authorization: Bearer→X-User-ID); the client cannot inject identity, the backend re-validates resource ownership, Telegram initData is HMAC-checked. The real gaps are a missing request-body size limit (cheap DoS) and invisible rate-limit rejections (no log/metric/admin view — that is TODO 8). Static landing serving is not covered by the gateway token bucket (it only guardsExecute). - R6 (TODO 7) scale: ~431
Stage Nreferences across ~104 files (incl. the file namebackend/internal/inttest/stage6_test.go). Code is the source of truth;docs/describe current state;PLAN.mdkeeps the decision history.
Locked decisions (owner interview)
- Stress test (TODO 9): early + final runs. Driver = edge protocol (Connect/FB through the gateway, moves generated by the solver) plus a separate gateway-hammer saturation test. Pacing = realistic (under limits) + saturation (ramp to the knee). Resource metrics = add cAdvisor + postgres_exporter to the contour (today only Go-runtime metrics exist). The harness stays in the repo for repeats.
- Push (TODO 4 + 5): both — kill
lobby.poll(use the existingmatch_found, keep poll as the ws-down fallback) and enrich push events with payloads. - Refactor (TODO 7): hygiene + structural changes by a reviewed list — behaviour-preserving, test-gated, contentious items surfaced to the owner before applying.
- Landing (TODO 3): separate static container behind the project caddy
(
/→ landing,/app/+/telegram/→ gateway); droplanding.htmlfrom the gatewaygo:embed. - Rate-abuse (TODO 8): metric + Grafana + admin view plus a conservative auto-flag — a soft, reversible "suspected high-rate" marker for operator review, tunable threshold, no auto-ban.
- Open auto-match (owner ad-hoc): a quick game enters a real game at once and waits inside
it (status
open, the opponent seat empty); a second human searching the same variant+rule joins it, or a robot fills it after a 90 s + random 0–90 s wait, pushing the in-app opponent_joined event. While open, the starter may move on their turn but resign, chat and nudge are disabled, and the lobby + opponent card read "searching for opponent". Matchmaking is now DB-backed open games — the in-memory pool,lobby.pollandlobby.cancelare gone. The schema is edited in the baseline (no prod data);game_players.account_idis nullable for the empty seat.
Phases
Each phase: read this tracker + the relevant docs/, interview the owner on the open
details below, implement within scope, then update the tracker + docs/code and get CI
green before marking it done.
R1 — Schema & naming reset (TODO 1 + 10) — first
Squash backend/internal/postgres/migrations/00001..00012 into one 00001_baseline.sql
(method: pg_dump --schema-only from a fully-migrated DB → wrap as the goose baseline →
prove a fresh migrate yields a schema identical to the 12-migration chain via the
integration suite → delete the old files; keep goose). Bake the new variant labels into the
baseline. Propagate scrabble_ru/scrabble_en/erudit_ru through the backend
(engine.Variant/ParseVariant, registry.dictFiles, the CHECK values), the wire
(pkg/fbs variant:string, regenerate FB) and the UI (lib/model.ts union, variants.ts,
fixtures, premium/alphabet keys, tests); i18n display keys stay display-only. Tidy
../scrabble-dictionary to a single source→dawg build point and align the dawg artifact
names to the new labels (crosses into ../scrabble-solver's committed fixtures — keep them
byte-identical). After merge, wipe the contour DB (drop the volume) so it re-provisions
on the next deploy.
- Critical files:
backend/internal/postgres/migrations/,backend/internal/engine/{engine,registry}.go,pkg/fbs/scrabble.fbs,ui/src/lib/{model,variants}.ts,../scrabble-dictionary/{Makefile,cmd/builddict,…}. - Open details to interview: the exact dawg filename scheme; whether the dict-repo tidy is one PR or split; how to script the contour DB wipe in the deploy.
R2 — Stress harness + contour observability + early run (TODO 9, part 1)
Build the reusable load harness as a new loadtest module in go.work (reuses pkg/fbs,
connect-go, and scrabble-solver for legal-move generation): a seeder that inserts
1000 guest + 10000 durable accounts with pre-created sessions (token hashes) directly in
the DB and hands the plaintext tokens to the client; a driver that runs N virtual users,
each in 3–5 concurrent 2–4-player games, exercising submit-play / pass / exchange / nudge /
chat / check-word / draft-move / profile-save through the edge protocol, in
realistic (under rate limits) and saturation (ramp) modes; plus a separate
gateway-hammer that deliberately exceeds limits to verify the limiter holds and measure
its cost. Add cAdvisor + postgres_exporter to deploy/docker-compose.yml and a Grafana
resource dashboard. Run the early pass against the freshly-wiped contour; produce a
trip report (logic/concurrency bugs + a resource baseline) that feeds R3 and R6.
- Critical files: new
loadtest/,deploy/docker-compose.yml,deploy/observability/*,docs/TESTING.md. - Open details: the scale ramp steps; the move-selection policy (a mid-ranked solver move for realistic game progress); run duration; the pass/fail bar.
R3 — Edge hardening (TODO 2 + 8 + 3)
Add a request-body size cap at the gateway h2c mux / Execute (e.g. ~1 MB). Add
rate-limit observability: a gateway_rate_limited_total{class} counter + a structured
log per rejection; an aggregate Grafana panel (request rate + rejection rate — spikes
visible without per-user label cardinality, honouring the Stage 12/17 discipline); an
admin-console view of recently throttled users/IPs (in-memory ring buffer, single-
instance, reset-on-restart, like the active_users gauge). Add the conservative
auto-flag: when a user is sustained-throttled past a tunable threshold, set a soft,
reversible account.flagged_high_rate_at marker (baked into the R1 baseline) surfaced in the
admin user list/detail — no auto-ban; the operator clears it. Split the landing into
its own static container (deploy/ + a Caddyfile route / → landing) and drop
landing.html from the gateway go:embed.
- Critical files:
gateway/internal/connectsrv/server.go,gateway/internal/ratelimit/,gateway/internal/connectsrv/metrics.go,backend/internal/adminconsole/,deploy/caddy/Caddyfile,deploy/docker-compose.yml,gateway/internal/webui/. - Open details: the auto-flag threshold/window + whether the marker is persisted vs in-memory; the landing image base (caddy vs nginx).
R4 — Push enrichment + kill the last poll (TODO 4 + 5)
Replace lobby.poll with the existing match_found push (keep the poll as a ws-down
fallback). Enrich your_turn/opponent_moved/notify to carry the state payload so the UI
renders from the event without a follow-up game.state (removes the lobby↔game nav latency
the owner noticed). Wire-contract change: pkg/fbs event payloads → backend notify emit →
UI stream consumers (ui/src/lib/app.svelte.ts), with the per-game cache as the landing
spot; regenerate FB.
- Critical files:
pkg/fbs/scrabble.fbs,backend/internal/notify/events.go,ui/src/lib/{app.svelte,transport}.ts,ui/src/screens/NewGame.svelte. - Open details: which events carry full vs delta payloads; the fallback-poll cadence when the stream is down.
R5 — Bundle slimming (TODO 6) — done
Analysed the bundle against the 100 KB-gzip budget; no code slimming was warranted, and the budget metric was retargeted to measure the app correctly. The build already minifies + tree-shakes; the dominant cost is the Connect/FlatBuffers transport runtime + generated bindings
- the Svelte runtime (≈⅔ of
main's source is third-party/generated) — irreducible within scope. Lazy-loading was rejected:bundle-size.mjssums every emitted chunk, so code-splitting yields no total-size win and adds request latency (+N gateway fetches on first navigation to a split screen). i18n lazy-load was skipped (the catalogs are a sliver of a Svelte-runtime-dominated shared chunk, andenmust stay bundled as theMessageKeytype source + fallback). Instead,bundle-size.mjsnow measures per HTML entry, with three independent gates on the natural chunk boundaries — app entry ≤ 100 KB, the Svelte+i18n shared chunk ≤ 30 KB, the landing's own chunk ≤ 5 KB — since the app's real payload is its entry chunk plus the shared chunk (≈97 KB), while the landing (≈24 KB) is reported separately and kept minimal. Same CLI + exit-code contract, so the CI step is unchanged.
- Critical files:
ui/scripts/bundle-size.mjs; no app code changed.
R6 — Refactor + docs reconciliation + de-staging (TODO 7) — done
Behaviour-preserving only. Three separable, separately-committed passes: (a) mechanical
de-staging — remove Stage N/TODO-N references from code, comments and service
READMEs (rename stage6_test.go); (b) docs↔code reconciliation — reconcile
docs/ARCHITECTURE.md / docs/FUNCTIONAL.md(+_ru) against the code-as-truth, fixing drift
and Go Doc comments; (c) structural changes by a reviewed list — surface a list of
proposed optimizations / test-suite consolidations to the owner, apply only the approved,
behaviour-preserving, test-gated ones. The full suite + the final stress run (R7) are the
regression gate. Incorporates the early-run (R2) bug fixes not already shipped.
- Open details: the structural-changes list itself (owner-approved before applying); the test consolidation targets.
R7 — Final stress run + tuning (TODO 9, part 2) — done
Re-run the R2 harness against the final, refactored system on a clean contour; analyse resource consumption across all components (gateway, backend, Postgres, the metrics/observability stack, docker log volume) and agree the tuning (pool sizes, rate limits, cache TTLs, container limits, GOMAXPROCS, log levels). Apply the agreed tuning; record the methodology + results in the repo.
→ Stage 18 (prod contour) then proceeds per PLAN.md.
Sequencing rationale
R1 first (cheapest now; everything builds on the final schema/naming and the stress test
must run against it). R2 builds the harness and runs the early pass to surface bugs and
a resource baseline that feed R3 and R6. R3/R4/R5 harden and improve the system.
R6 (de-stage + reconcile + structural) runs near the end so it sweeps settled code once and
benefits from all accumulated bug knowledge. R7 validates the final system and tunes it.
Then Stage 18.
Regression-safety discipline (cross-cutting)
- Every phase is a
feature/* → developmentPR; CI (unit+integration+uibehind theCI / gatecheck) must be green before the owner merges; watch the post-merge contour deploy withgitea-ci-watch.py. R6structural changes are behaviour-preserving, test-gated, and split from the mechanical sweeps; contentious items are owner-approved first.- The two stress runs (
R2early,R7final) are the system-level regression gate.
Verification (per phase)
go build ./<module>/...,go vet,gofmt -l .clean,go test -count=1 ./<module>/...; UI:pnpm check && pnpm test:unit && pnpm build; the integration suite (-tags integration) for DB/schema changes;docker compose configfor deploy changes; green CI on the PR + a healthy contour deploy.R1: prove the squashed baseline yields a schema identical to the 12-migration chain (integration suite on a fresh DB) before deleting the old files.R2/R7: the harness runs end-to-end against the contour; the trip report lists concrete defects + a resource profile from the Grafana cAdvisor/postgres_exporter panels.
Refinements logged during implementation
-
R1 (interview + implementation):
- Variant labels
english/russian_scrabble/erudit→scrabble_en/scrabble_ru/erudit_ruacross the backend (engine.Variant.String/ParseVariant; thegames/game_invitationsvariantCHECK in the baseline; GCG#lexiconand thevariantmetric attribute both flow fromString), the wire (pkg/fbsvariantis astringfield — values change with no FlatBuffers regen) and the UI (model.tsunion,variants.tsrecords,codec/premiums/mocks/tests, the admindictionary.gohtml). Kept: the Go enum identifiers (VariantEnglish…, internal) and the i18n display keys (new.english/new.russian/new.erudit, display-only).complaints.variantstays free-text (no CHECK, as before). - dawg filenames kept descriptive (
en_sowpods/ru_scrabble/ru_erudit) — only the registry'sVariantkey carries the rename, soregistry.go, the publishedscrabble-solverfixtures and the dictionary release artifact are untouched (decouples the three repos). - Migrations squashed 12 → one hand-written
00001_baseline.sql. Verified by apg_dump --schema-onlydiff (the chain vs the baseline are identical but for the two intended variant-CHECK values) plus the green integration suite. No data migration (no production data). - Done (cross-repo + contour): the
scrabble-dictionarytidy merged (PR #2) and was re-cut as the byte-identicalv1.0.1release for clean provenance (the backend stays onv1.0.0— same bytes, no rewire; the backend pulls a version-pinned release artifact, not master). Post-merge the contourbackendschema was wiped (DROP SCHEMA backend CASCADE+ restart, not a volume drop) and re-migrated to the baseline — verified the new variant CHECK (scrabble_en/scrabble_ru/erudit_ru),games=0 and a clean boot.
- Variant labels
-
R2 (interview + implementation):
- Locked decisions: game assembly via invitations (real path, no robots; not direct game-row
inserts); moderate ramp 50 → 200 → 500 at 10 min/step; diagnostic pass bar (no SLO gate);
run as a one-shot container on
scrabble-internalin this PR. - Harness = new
scrabble/loadtestmodule (use ./loadtest+ areplace scrabble/gatewayfor the dot-free edge-proto import). It seeds 1000 guest + 10000 durable accounts + sessions directly in Postgres (token hash mirrorsbackend/internal/session), drives players over the edge protocol, generates mid-ranked legal moves locally with the embeddedscrabble-solverby replayinggame.history(the edge carries no board — mirrorsengine.ReplayBoardvia the public API), and a gateway-hammer. Compact CLI (run/cleanup), distroless Dockerfile (DAWGs baked), Go unit tests. - Adding the module broke the other images' builds — backend/gateway/telegram Dockerfiles reduce the
workspace but still referenced
./loadtest(not in their context); each now also-dropuse=./loadtest(backend/telegram additionally-dropreplacethe gateway replace). Caught by the first deploy run; verified by building all four images. - Harness payload fixes found by the smoke pass: the draft DTO's
rack_orderis a string (was sent as[]→bad_request); the display-name validator forbids digits/colons, so the cleanup marker became a letters-onlyZzloadtestsoprofile.updateresends the seeded name.chat_not_your_turn/nudge_own_turnare by-design turn gates, correctly exercised. - Observability: added cAdvisor + postgres_exporter + the Scrabble — Resources dashboard +
two Prometheus jobs. Finding: cAdvisor yields only the root cgroup on the contour host (separate
XFS
/var/lib/dockerbreaks its layer-ID resolution — the existing galaxy deploy has the same limit), so per-container CPU/RSS for the early pass was captured viadocker stats. R7: adopt the otelcoldocker_statsreceiver (already the contrib image) for per-container metrics in Grafana. - Early run (2026-06-09): ramped clean to 500 players, no crash/deadlock, cleanup removed all 11000
accounts. 1.2 M edge calls, 48 870 plays, 2 798 games finished; the per-user limiter held under the
hammer (99.97 % rejected, p99 2 ms). Top finding: ~14 %
transport_errorongame.stateat 500 players, under CPU saturation (backend/gateway/Postgres each ~1 core) and amplified by the harness's single sharedhttp2.Transport; the harness itself peaked at 86 % of a core on the same host, so the figures are pessimistic. Full trip report in../loadtest/REPORT-R2.md; it feeds R3 (h2cMaxConcurrentStreams/timeouts, body-size cap), R6 and R7 (per-player transports, separate hardware, pool/limit sizing). - CI:
./loadtest/...added to the path filter + vet/build/test;go.work.sumcarries the new deps.
- Locked decisions: game assembly via invitations (real path, no robots; not direct game-row
inserts); moderate ramp 50 → 200 → 500 at 10 min/step; diagnostic pass bar (no SLO gate);
run as a one-shot container on
-
R3 (interview + implementation):
- Locked decisions: the flag column lands by editing the R1 baseline (+ a contour schema
wipe after merge — no migration chain accrues before prod); auto-flag defaults 1000 rejected /
10 min (
BACKEND_HIGHRATE_FLAG_THRESHOLD/_WINDOW, rolling window, set-once, operator clears, no auto-ban); landing image = caddy:2-alpine; throttle data flows gateway → backend (a 30 s per-key summary POST to the new/api/v1/internal/ratelimit/report, the existing trusted direction) with the episode window + flag rule in the backend (internal/ratewatch); rejection logging = Warn summary per key per window + Debug per rejection — a deliberate deviation from the phase's "structured log per rejection" (the R2 hammer would have logged ~522k lines in minutes); all three R2-report tails included (explicit h2c sizing, the session-resolve failure cause at Warn, reviving the admin limiter). - Body cap:
GATEWAY_MAX_BODY_BYTES(default 1 MiB) as both the Connect per-message read limit and anhttp.MaxBytesReaderwrap of the public mux; an oversized Execute isresource_exhausted. - Dead config found:
AdminPerMinute/AdminBurstwere never wired — the gateway/_gmmount is now 429-guarded per IP ahead of its Basic-Auth. The caddy-fronted contour path stays unlimited (stock caddy has no limiter) — an accepted gap, recorded indocs/ARCHITECTURE.md§12. - Landing split: a
landingtarget ingateway/Dockerfile(the UI build stage is shared; identical compose build args keep it one cached build); the gateway dropslanding.htmlfrom the embed and 308-redirects/→/app/; the contour caddy routes/app/,/telegram/and the Connect path to the gateway and the catch-all to the landing container; the CI deploy probe now checks both/(landing) and/app/(gateway). - Observability:
gateway_rate_limited_total{class}(user/public/email/admin, aggregate-only)- a rate-vs-rejections panel on the Edge/UX dashboard; the admin console gains the Throttled
page (the in-memory episode window, reset-on-restart like
active_users, plus the flagged-account queue) and the flag badge / clear action on the user list / card.
- a rate-vs-rejections panel on the Edge/UX dashboard; the admin console gains the Throttled
page (the in-memory episode window, reset-on-restart like
- The jet regen also restored the previously missing
game_drafts/game_hiddengenerated models (their tables were added after the last jetgen run; no behaviour change).
- Locked decisions: the flag column lands by editing the R1 baseline (+ a contour schema
wipe after merge — no migration chain accrues before prod); auto-flag defaults 1000 rejected /
10 min (
-
R4 (interview + implementation):
- Locked decisions: delta-first, not full snapshots — an event carries only the new move and
the UI applies it to its per-game cache, keyed on
move_count(idempotent + gap-safe: a gap or the actor's own move falls back to agame.state+game.historyrefetch).match_found/game_startedcarry the recipient's initialStateView(instant lobby→game); the fallback refetch stays the existing two calls (no merged endpoint); the matchmaking poll runs only while the stream is down (2.5 s); all UI-state-changing events carry their payload (incl. lobbynotify). - Enriched events (
pkg/fbstrailing fields — backward-compatible, no FB regen of values, only the schema):opponent_moved(+move/game/bag_len),your_turn(+move_count),match_found(+state),game_over(+game),notify(+account/invitation/state). The pre-R4opponent_movedscalars (seat/action/score/total) stay for wire back-compat, now redundant withmove/game— slated for the R6 de-stage. - Encoding placement: the
notifypackage keeps ownership of the FlatBuffers encoding (a newencode.gomirrors the gateway transcode but reads wire-agnosticnotify.*input structs +engine.MoveRecord); the game/lobby/social services map their domain types to those structs, so the wire schema stays out of the domain. Flagged for R6: this partly duplicates the gateway encoders (different source types) — a candidate consolidation. - Actor self-fetch killed too (beyond literal "push"): the
submit_play/pass/exchange/resignresponse (MoveResult) now returns the actor's refilled rack + bag size, so the mover renders the next turn from the response —Game.svelte'scommit/pass/exchange/resigndrop theirawait load(). match_foundenrichment needs a per-seat initial state:lobby.GameCreatorgainedInitialState, andgame.Service.InitialStatebuilds thenotify.PlayerState(rack re-encoded to wire indices, the variant alphabet embedded for a first-seen variant).- UI: a pure
lib/gamedelta.tsreducer (applyMoveDelta/applyGameOver/seedInitialState, unit-tested) advances the cache;app.svelteseeds it onmatch_found/game_started;Game.svelteapplies the delta (falling back toload()while composing, on a gap, or on its own move's new rack);NewGame.sveltepolls only whenapp.streamAliveis false and guards its teardown so a push-delivered match is not cancelled. - notify (friends/invitations) scope: the backend carries the full account / invitation payload on the
wire (per "all events → push"); the UI seeds the game cache from
game_startedbut keeps its lightweight authoritative badge refresh (refreshNotifications, on the rarenotifyevent + on foreground) rather than adding client-side friend/invitation caches — the per-move hot path is fully de-fetched, which was the goal. Deeper lobby-cache consumption is an easy follow-up. - No schema change (no migration); the contour needs no DB wipe. Tests:
notifyFB round-trips +emitMovedelta + thegamedeltareducer; the e2e mock now emits the enriched delta.
- Locked decisions: delta-first, not full snapshots — an event carries only the new move and
the UI applies it to its per-game cache, keyed on
-
R5 (interview + implementation):
- No code slimming — by analysis. A gzip measure + sourcemap attribution of the real
distshowed the app bundle is already minified + tree-shaken and dominated by the Connect/FlatBuffers transport runtime + generated FB/PB bindings (≈⅔ ofmain's source) and the Svelte runtime — all third-party/generated, irreducible within R5's scope. App-authored code carries no hand-trimmable fat. - Lazy-load rejected (screens and i18n):
bundle-size.mjssums every emitted chunk, so code-splitting moves bytes between chunks for zero total-size win while adding request latency (+N gateway fetches on first navigation to a split screen). i18n lazy-load additionally buys ≤3 KB (en-only users) at the cost of an asynct(), andenmust stay bundled (it is theMessageKeytype source + fallback). Chunk-collapsing rejected too — keeping the near-static Svelte runtime in its own cacheable chunk is the recommended practice (an app deploy then re-busts onlymain, not the runtime), and HTTP/2 makes the extra preload request negligible. - Metric retargeted to the app. The two-entry build (
index.htmlapp +landing.html) makes Rollup hoist the code shared by both (Svelte runtime + i18n +aboutContent) into one preloaded chunk, so the app actually loads its entry chunk + the shared chunk (≈74 + ≈23 = ≈97 KB), neverlanding.js(≈1.6 KB). The old script summed all three chunks (98.8 KB), over-counting the app bylanding.js.bundle-size.mjsnow parses each built HTML for the JS it eagerly loads and gates three parts independently — app entry ≤ 100 KB, shared (Svelte+i18n) ≤ 30 KB, landing-own ≤ 5 KB — reporting the app total (≈97) and landing total (≈24.5). Same CLI + exit-code contract, so the CI step is unchanged. - No app/source/build change (
App.svelte,lib/i18n/,vite.config.tsuntouched); no schema change, no contour wipe. The stale "~82 KB" figure was corrected inbundle-size.mjsandui/README.md.
- No code slimming — by analysis. A gzip measure + sourcemap attribution of the real
-
R6 (interview + implementation):
- Locked decisions: apply both wire/code structural changes (B + A) and only C1+C2 of
the test consolidation (not C3/C5); strip the
*(Stage N)*tags from all current-state docs (ARCHITECTURE / FUNCTIONAL+_ru/ TESTING / UI_DESIGN), keeping PLAN.md / PRERELEASE.md / CLAUDE.md as history; splitstage6_test.goby domain. Theh2cMaxConcurrentStreamssizing stays an R7 concern (tuning, not behaviour-preserving); the R2 early run forced no code fix, so nothing was carried in. - (a) De-staging: removed the
Stage N/TODO-N/(RN)references across code, comments, service READMEs and the current-state docs, rewording narratives to present tense (no technical content lost). Renamed the only stage-named identifiers (registerStage8→registerSocialOps,registerStage11→registerLinkOps) and splitstage6_test.go(TestEmailLoginFlow→email_test.go;TestGuestAutoMatchLeavesNoStats+provisionGuest→account_test.go). De-staged the.fbs/.protocomments and regenerated: only the.proto-derived Go docstrings (*_grpc.pb.go,push.pb.go) changed — flatc strips schema comments, so the FB Go/TS bindings were untouched. - (b) Reconciliation: the docs were accurate (each R-phase baked its own); the one drift was a stale
"guest-reaping deferred (TODO-3)" note in
ARCHITECTURE.md§3 — guest reaping is implemented, so the note was replaced with the current behaviour (FUNCTIONAL/TESTING already described it). - (c) B — dead
opponent_movedscalars: removedseat/action/score/totalfromOpponentMovedEvent(pkg/fbs/scrabble.fbs+ thenotifyemit + the round-trip test); regenerated FB Go + TS. No reader used them (the UI codec/mock takemove/game/bag_len; the gateway forwards the payload verbatim). A pre-release wire-slot renumber — free with no prod data, no DB change. - (c) A — shared FB builders: new
scrabble/pkg/wireholds the single definition of the nested wire tables (GameView / MoveRecord / StateView / AccountRef / Invitation) shared by the backendnotifyencoder and the gatewaytranscode; both map their own source types to neutralwire.*structs and delegate. Honest tradeoff: the verboseStart/Add/End+ reverse-prepend boilerplate is now written once, but the field set is still mapped per side, and the new package makes the change net +~145 LOC — a single-source / anti-drift win for the fiddly mechanics rather than a line-count cut. Behaviour- preserving: the two sides' field sets were verified identical and the round-trip tests pass unchanged. - (c) C1+C2 — inttest fixtures: moved the cross-file service/game fixtures (
newGameServicewas used by 10 files) intobackend/internal/inttest/helpers.go; single-file helpers stay local. Pure relocation. - No schema change → no contour DB wipe. Regression gate: the full unit + integration + UI suites plus the R7 stress run.
- Locked decisions: apply both wire/code structural changes (B + A) and only C1+C2 of
the test consolidation (not C3/C5); strip the
-
R7 (interview + implementation):
- Locked decisions: run the harness same-host (one-shot container on
scrabble-internal, capped--cpus=3so the contour keeps spare cores); apply container limits +GOMAXPROCSnow (not just a prod recommendation); replace cAdvisor with the otelcoldocker_statsreceiver (it resolved only the root cgroup on this host); keep rate-limit / h2c knobs compiled-in (change values only if the data demands — it did not). - Harness refinements (pre-run): each virtual player builds its own
edge.Client(its own h2c connection for its Subscribe stream + Execute calls) instead of all players sharing onehttp2.Transport— the R2transport_errorartifact; andplayTurnnow reports a finished game so the player drops it from rotation. Effect, measured:game.statetransport_error14 % (R2) → 2.49 %;game_finishedon chat ≈ 3 900 → 35. - Observability: added the
docker_statsreceiver tootelcol(api_version: "1.44"— the daemon's minimum is 1.40; the receiver defaults to 1.25 and crash-looped until pinned), mounted the docker socket read-only withgroup_add(the contrib image runs as UID 10001), dropped the cAdvisor service + its Prometheus job, and retargeted the Scrabble — Resources dashboard to the docker_stats metric names (container_cpu_utilization/100 == cores). Cross-checked againstdocker statswithin sampling error. - Profile (final run, 500 players, limits in force): the gateway is the binding constraint — with
one connection per player it bursts into its 2-core cap (the residual 2.49 %
transport_error); backend ~0.85 core and postgres ~1.4 cores had headroom; tempo reached its 1 GiB cap; the backend pool sat at itsMaxOpenConns=25cap (28 backends); docker logs were unbounded (~14 MiB / 30 min on the backend at info). Full write-up in../loadtest/REPORT-R7.md. - Round-2 tuning (owner-agreed, all in
deploy/docker-compose.yml, no code change): gateway 2 → 3 cores +GOMAXPROCS=3; tempo memory 1 → 2 GiB; backendMAX_OPEN_CONNS25 → 40; a json-file log-rotation default (10m × 3) applied contour-wide via a YAML anchor (level stays info). backend/postgres kept at 2 cores / 512 MiB (headroom is cheap on the shared host). - Validation: the same gradual ramp on the tuned contour cut
game.statetransport_errorto 0.72 % (gateway ~2 cores, now under the 3-core cap, no throttle; tempo ~1.27 GiB, under 2 GiB). A separate burst run (a single 100 → 500 jump) pegged the gateway at 3 cores (≈296 % sustained, 9.27 % error), confirming it is connection-CPU-bound — a true arrival spike is a horizontal-scaling lever, not more cores per node (recorded in the prod-sizing recommendation). - No schema change → no contour DB wipe. Bake-back:
loadtest/REPORT-R7.md(new),loadtest/README.md,docs/TESTING.md, the telemetry/observability section ofdocs/ARCHITECTURE.md, the repo-layout line inCLAUDE.md.
- Locked decisions: run the harness same-host (one-shot container on
-
UI — Tab-bar navigation redesign (owner ad-hoc, not on the raw TODO list): drop the hamburger
Menu.svelteeverywhere (it fought the Telegram-fullscreen layout, where it had to be re-centred).- Locked decisions (interview): the in-Settings sub-nav is a bottom TabBar with the active tab highlighted (icon-only); Export GCG moves to the left slot of the move-history header (free in a finished game, where 🏁 leave does not apply); the lobby ⚙️ badge counts incoming friend requests only (invitations keep their own lobby section); unread chat is badged on the score bar and the 💬.
- What shipped: a ⚙️ Settings hub (
screens/SettingsHub.svelte) over the existing Settings/Profile/Friends/About bodies and an in-game comms hub (game/CommsHub.svelte) over chat + dictionary, both with in-place tabs and a fixed back target; the game's menu items relocate into the open move history (🏁 leave / 📤 export + 💬 comms header) and the player cards (🤝 add-friend); a shared TapConfirm (components/TapConfirm.svelte,lib/tapconfirm.ts) — tap → fading ✅ → tap — replaces the Skip/Hint press-and-hold popovers and drives the add-friend confirm. Fixed the move-history "jump" bug (the slid board is now inert and the stage can't scroll, so a swipe up genuinely closes it).Menu.svelte+HoldConfirm.svelteremoved. - No schema/wire change → no contour DB wipe. Bake-back:
docs/UI_DESIGN.md,docs/FUNCTIONAL.md(+_ru). Regression gate: UIcheck+ unit (tapconfirm) + build + bundle budget + e2e (Chromium & WebKit), all green.
-
UI — Merge Exchange/Pass; drop the dead Tournaments tab (owner ad-hoc, not on the raw TODO list): the lobby's 🏆 Tournaments tab was an inert
lobby.soontoast — removed (the lobby is back to three tabs, matchingdocs/FUNCTIONAL.md). In-game the separate 🥺 Skip (pass) tab folds into the 🔄 tab, now Exchange/Pass, whose dialog passes when no tile is selected and exchanges when tiles are.- Decision — a pass is NOT an exchange of zero (verified against the rules + GCG): the merge is
UI-only. Pass and exchange stay distinct game actions end-to-end — wire (
GameActionRequestvsExchangeRequest), engine (ActionPassvsActionExchange), and the GCG Poslfit dialect (a pass is a bare-, an exchange is-TILES). The engine forbids a zero-tile exchange (ErrNothingToExchange) and allows an exchange only with a full rack left in the bag (ErrNotEnoughTilesToExchange), while a pass is always legal — collapsing them would lose a real distinction. The dialog dispatches the existinggateway.pass/gateway.exchange. - What shipped:
Lobby.svelte(tab removed);Game.svelte(one 🔄 Exchange/Pass tab no longer gated on an empty bag; the dialog disables tile selection while the bag is below a full rack (bagLen >= RACK_SIZE), its confirm button reading Pass without exchanging / Exchange N); i18n (game.draw→ Exchange/Pass, newgame.passNoExchange, droppedgame.skip/lobby.tournaments/lobby.soon). No backend/wire/history/GCG change. - No schema/wire change → no contour DB wipe. Bake-back:
docs/UI_DESIGN.md,docs/FUNCTIONAL.md(+_ru). Regression gate: UIcheck+ unit + build + bundle budget + e2e (Chromium & WebKit).
- Decision — a pass is NOT an exchange of zero (verified against the rules + GCG): the merge is
UI-only. Pass and exchange stay distinct game actions end-to-end — wire (
-
AI — Honest AI opponent in quick game (owner ad-hoc, not on the raw TODO list): a second quick-game opponent the player knowingly chooses, distinct from the disguised robot of the random/open path (which is kept as-is). New Game's quick-game mode replaces the "auto-match" subtitle with a two-button selector 🤖 AI / 👤 Random player (the
.seg/.optsegmented style, AI the default); for AI the move-clock line reads "Loss after 7 days of inactivity" and the "searching" hint is hidden.- Locked decisions (interview): AI move is event-driven (the robot replies the instant the
player's move commits; the 30 s driver is the fallback); AI games do not touch
account_stats(practice, like guests); the Stage 5 strength logic is reused unchanged (playToWin40 % from the seed + margin band); no per-move timeout — a 7-day inactivity loss instead; the 7-day line lives on the New Game screen (the in-game screen has no move-clock line); chat + nudge disabled, word-check kept, add-friend never drawn, opponent shown as 🤖 everywhere. - The 7-day rule reuses the existing per-turn timeout: an AI game is created with
turn_timeout_secs = AIInactivityTimeout(7 days) and the existing timeout sweeper resigns the overdue seat — since the robot moves at once, only the human is ever on the clock, so the per-turn timeout is the abandon rule (no new column, no new sweeper). - One game flag drives everything:
games.vs_ai(edited into the R1 baseline — pre-release, so a contour DB wipe after merge). It is set only on AI-started games, so a robot-filled random game keepsvs_ai=falseand the disguised opponent is never revealed; the UI derives 🤖 / the gates from the flag, never from the opponent account. New backend pathMatchmaker.StartVsAI(picks a pooled robot via the existingPick, creates an active seated game viagame.Service.Create, random seat order) — the AI request never enters the open pool, so the open-game reaper never touches it. The robot driver gains avs_aibranch (no sleep, no proactive nudge, zero delay) and a focusedDriveGame/TriggerMovefast path wired from the game service's after-create/after-commit hook (SetAITrigger, a func value so the game package never imports the robot package). Chat/nudge gated by a newsocialVsAIcheck (ErrGameVsAI→ 409ai_game); statistics skipped incommitwhenvs_ai. - Wire:
EnqueueRequest+=vs_ai,GameView+=vs_ai(trailing FB fields, regenerated Go + TS), threaded through the backend DTO, the gateway transcode and thepkg/wire+notifybuilders. - Tests:
lobbyunit (StartVsAI seats a robot + flags the game; empty pool leaves no game); backend integration (ai_game_test.go: active+seated+vs_ai+7-day clock, robot moves immediately, stats skipped, 7-day timeout resigns the human, chat/nudge rejected); UI codec round-trip (vs_aion enqueue + game view); e2e (an AI game shows 🤖, no "searching", chat disabled, the dictionary still works) + the existing quick-match e2e updated to pick Random player (the default is now AI). - Schema/wire change → a contour DB wipe after merge (
DROP SCHEMA backend CASCADE+ restart, the R1/R3 pattern). Bake-back:docs/ARCHITECTURE.md,docs/FUNCTIONAL.md(+_ru),docs/UI_DESIGN.md,backend/README.md, Go Doc comments. - Post-review refinements (owner, same PR): (1) the GCG export labels the robot seat "AI" rather
than its human-like pool name (
ExportGCGoverrides the name viaaccounts.IsRobot; the in-app 🤖 is unchanged); (2) honest-AI games emit noyour_turn— the robot replies instantly, so the signal would arrive with the move and be pointless;opponent_movedstill advances the UI; (3) the admin console surfaces the AI flag — a 🤖 column in/gamesand an "AI game" line on the game card (GameRow/GameDetailViewgainVsAI); (4)games_started_total/games_abandoned_totalgain avs_aiattribute and the Grafana Game domain dashboard splits started/abandoned into human and AI panels. - Follow-up (separate PR — strategy deviation): the robot now plays ≈20% of opening/midgame moves
against its per-game
playToWinintent (toward the opposite margin band — a winning robot eases off, a losing one surges ahead), tapering linearly to 0 over the last 14 bag tiles and 0 once the bag is empty, so the endgame follows the chosen strategy strictly while earlier outcomes can swing the human's way. Deterministic from the seed (mix(seed,"deviate",moveCount)), applied to both robot paths via the sharedselectMove; the per-game intent (and the admin card) is unchanged. Tests:robotunit (taper bounds + monotonicity, never-in-endgame, determinism, ~20% distribution). Bake-back:docs/ARCHITECTURE.md§7,docs/FUNCTIONAL.md(+_ru),backend/README.md,PLAN.mdStage 5.
- Locked decisions (interview): AI move is event-driven (the robot replies the instant the
player's move commits; the 30 s driver is the fallback); AI games do not touch
-
GL — Simultaneous quick-game cap (owner ad-hoc, not on the raw TODO list): a player may hold at most 10 active quick games; at the cap the lobby greys New Game and shows a plain notice "Вы достигли лимита одновременных партий", both clearing automatically when an active game finishes.
- Locked decisions (interview): what counts = active + open (searching) quick games, including
AI (
vs_ai); friend games (invitation-linked) never count. The backend gate refuses all new-game creation at the cap —lobby/enqueueandinvitations— with 409game_limit_reached; accepting an invitation is never gated, so friend games are capped "from the other end". Delivery = a booleanat_game_limiton the existinggames.list(no per-event payload: a turn change does not move the count, and the lobby already re-fetchesgames.liston entry + every game event); the first uncached lobby frame defaults the button enabled (the backend gate is the authority). - What shipped:
game.MaxActiveQuickGames+Store/Service.CountActiveQuickGames(active/open seats, nogame_invitationsrow; hidden games still count → a dedicated count, not a filter over the lobby list);Server.atGameLimit/ensureUnderGameLimitgatinghandleEnqueue+handleCreateInvitation;gameListDTO.at_game_limit; the FBGameListtrailingat_game_limit(regenerated Go + TS) threaded through the gateway transcode + UI codec;lib/model+lobbycachesnapshot +Lobby.svelte(disabled tab + a muted.limitnotice); i18nlobby.limitReached(en authoritative + ru). - Caveat (logged): the gate is a pre-check, not transaction-atomic — concurrent creates from one account could momentarily exceed by 1–2 (harmless soft cap; the UI disables the button regardless). Strict atomicity was judged a disproportionate diff across the two create paths.
- No schema change → no contour DB wipe (only a trailing FB field, no migration). Tests: backend integration
(
game_limit_test.go: count rule + HTTP gate 409 + accept bypass), server unit (error mapping), gateway transcode round-trip, UI codec + lobbycache unit, e2e (gamelimit.spec.ts). Bake-back:docs/FUNCTIONAL.md(+_ru),docs/ARCHITECTURE.md§8,docs/UI_DESIGN.md,backend/README.md.
- Locked decisions (interview): what counts = active + open (searching) quick games, including
AI (