EvaluatePlay (the hottest gameplay call, fired on every tile placement) now uses
the warm live-game cache directly: an active game stays cached (mutated in place
across moves, evicted only on finish), so the cached engine game and its immutable
seat list answer the membership check and the score with no DB read. The cold path
(eviction / first load) still loads and validates via the store. The seat list is
cached alongside the engine game for the membership fast path.
GetGame also folds its two round-trips (game, then seats) into one LEFT JOIN,
preserving the contract (same Game, a seatless game still returns empty seats, seat
order kept) — one round-trip for every remaining caller.
Measured at 500 players: evaluate p99 halves (200 -> 100 ms) and the per-op query
count drops. It does NOT cut postgres CPU — that is write-bound (per-move CommitMove
plus draft upserts and journal replays), the cheap indexed GetGame reads were never
its bottleneck, and postgres runs with headroom (~1.5 of 2 cores). So this is a
latency / query-volume optimization, not a DB-CPU one.
Regression cover: a non-player evaluate against a warm game asserts the cached-seat
membership path; the integration suite exercises GetGame's join across every game op.
The loadtest harness never modelled game.evaluate — the debounced per-tile
play preview a real client fires several times per turn, the hottest gameplay
call. Model it (one evaluate per placed tile + reconsideration re-previews +
draft.save, human-paced; --eval / --eval-recon toggle it).
That realistic load surfaced the real bottleneck: the gateway's backend HTTP
client used the default transport (MaxIdleConnsPerHost=2), so every sync call
to the single backend host churned a fresh TCP connection — ~26500 TIME_WAIT
sockets at 500 players (near the ephemeral-port ceiling), burning ~1.75 gateway
cores while the backend sat near-idle. It was the unfixed root of the residual
transport_error the earlier passes chased on the client side.
Widen the keep-alive pool (backendMaxIdleConns=512, ~2x the observed 225-conn
peak). At 500 players the churn collapses to ~0 and peak gateway CPU drops ~7x
(~1.75 -> ~0.26 cores); postgres (~1.65 cores) becomes the busiest service.
This overturns the earlier "gateway is the binding constraint, scale it
horizontally" sizing — that was sizing around this bug, not a real floor.
Consolidate the loadtest trip reports into one loadtest/REPORT.md (drop the
R2/R7 split) and bake the finding into README / PRERELEASE / ARCHITECTURE /
TESTING.
The main bot is now an admin in the moderated discussion group and receives its
messages (allowed_updates includes message). Its default handler replied to
every message with a Mini App launch button — an inline web_app button, which
Telegram permits only in private chats — so replying in the group failed with
BUTTON_TYPE_INVALID (silently: the send fails, no user-facing error). Reply only
in a private chat; in the group the bot only manages permissions. The promo bot
gets the same guard.
- Bake the final default-allow + mute-the-ineligible strategy into
docs/ARCHITECTURE.md, docs/FUNCTIONAL.md (+_ru), platform/telegram/README.md,
the deploy compose comment and the PRERELEASE tracker. The live test proved a
per-user grant cannot exceed a deny-by-default group (Telegram intersects the
chat default with the per-user permission), so the chat allows sending by
default and the bot restricts the ineligible instead of granting the eligible.
- Lower the per-event chat_member trace and eligibility evaluation to Debug;
keep the actual mute/unmute actions, the startup self-check and warnings at
Info, so prod logs only what the bot did.
- Update game.searchingForOpponent (Searching -> Waiting for opponent / Поиск ->
Ждём соперника) and the quickmatch e2e assertions to match.
Telegram intersects the chat default with each user's permissions, so a per-user
grant can never exceed a deny-by-default group: the original default-deny +
grant design could not let any user write (can_send=true was AND-ed with the
denying default). Invert it — the chat allows sending by default and the bot
MUTES an ineligible member (unregistered, admin-suspended, or chat_muted) and
restores an eligible one it had muted, acting only when the current state
differs (idempotent, no self-loop). The block/unblock/chat_muted/registration
path already sets can_send to the eligibility, so it is unchanged.
The CanSendMessages loop-guard skipped exactly the stuck case — a restricted
member whose chat_member event reports can_send=true yet who cannot actually
write. Replace it with a precise loop guard (skip only the bot's own restrict
action, i.e. the update whose performer is the bot) and grant any eligible
in-chat member (member or restricted) otherwise. Also log the new member's
can_send, is_member and the actor id for full visibility.
A default-deny discussion group reports a present or freshly joined member as
`restricted` (no send right), not `member`. The join filter required `member`,
so the real case never matched and a registered user stayed muted. Grant any
eligible in-chat member (member or restricted) that still lacks the send right,
with a loop guard (skip when send is already allowed) so the bot's own grant
does not re-fire. Revoking a now-ineligible user stays the chat-gate path's job,
so this never fights a chat_muted/block.
Two follow-ups from a contour test where a user joined the chat, then
registered, and got no write access — with silent logs.
Observability: log every chat_member update (chat id, configured id, user,
old->new status), the eligibility result and the grant outcome; plus a startup
self-check that warns loudly when the bot is not an administrator in the chat
with the restrict-members ("Ban users") right — the common misconfiguration,
previously invisible in the logs.
Grant on first registration: a user who joins the moderated chat BEFORE
registering is covered by no chat_member event, so the join-time grant never
fires for them. ProvisionTelegram now reports first contact, and the Telegram
auth handler emits chat_access_changed on it, so the gateway re-evaluates and
grants write access if the user is already in the chat.
tgbot.New validates the token with getMe, so a bad or unreachable promo token
would otherwise return an error from run() and crash-loop the whole bot process
— taking the main game bot down with it, since they share the container. Log it
and skip the promo bot instead; the main bot and bot-link are unaffected.
Add a second standalone promo bot to the bot container (answers /start with a
localized message + a URL button into the main bot's Mini App) and gate write
access in a channel's linked discussion chat: grant on join when the Telegram
user is registered and neither admin-suspended nor holding a new chat_muted
role, and revoke/grant on the matching moderation change for a member currently
in the chat.
Eligibility (registered AND NOT suspended AND NOT chat_muted; the game
suspension dominates) is resolved once in the backend and reached two ways: the
bot's join-time unary ResolveChatEligibility over the existing mTLS bot-link,
and a backend chat_access_changed event -> gateway -> ChatGate command
(idempotent; a temporary-block-expiry sweeper may over-emit). The bot guards the
block/unblock path with getChatMember, since bots cannot list members.
A web_app button cannot open another bot's Mini App (it signs initData with the
sending bot's token), so the promo button is a t.me ?startapp URL reusing the
UI's VITE_TELEGRAM_LINK. The bot must be a chat admin with the restrict-members
right and chat_member in its allowed updates.
No schema change: chat_muted reuses the data-driven account_roles table.
The dismiss check fired at the instant a word finished laying, so a word
was never held: ЭРУДИТ fell straight into the lobby (too fast) and
ЗАГРУЗКА got no readable pause. The pause was a *leading* gap before the
next word, not a hold after the current one.
Move the hold to after each word and run the check after it: every word
(ЭРУДИТ included) now stays up for PAUSE_MS before the splash either
dismisses or lays the next word. prefixMs = WORD_MS + PAUSE_MS,
cycleMs = 2*(WORD_MS + PAUSE_MS).
On a cold app open the lobby's game list arrives over the network; on a
slow link the empty "no games yet" line flashed before the games loaded.
Add a full-screen tile splash that lays a Scrabble crossword of ЭРУДИТ /
ЗАГРУЗКА / ОЖИДАНИЕ (Эрудит point values, hardcoded since the alphabet
table is not cached at boot) until the lobby's first load settles, then
removes itself to reveal the populated list.
- lib/splash.ts: pure layout + reveal schedule (unit-tested).
- components/Splash.svelte: App-level overlay; per-tile drop-in; loops
ЗАГРУЗКА → ОЖИДАНИЕ until ready, dismisses on a word boundary. Static
ЭРУДИТ under reduced motion / the mock build.
- app state: lobbyReady (set by Lobby on first settle) + splashDone,
reset on logout.
- App.svelte: overlay while routeIsLobby && !splashDone; the plain text
splash now only covers non-lobby deep-links during bootstrap.
- docs: UI_DESIGN + FUNCTIONAL (+ _ru).
The @honeypot block both deleted and set X-Scrabble-Honeypot in one reverse_proxy.
Caddy applies header_up deletions *after* sets, so the tag we set was immediately
stripped: the gateway never saw it, and a decoy hit (e.g. GET /.env) fell through
to the gateway's /app redirect (308) instead of tripping the honeypot. Drop the
delete — the bare set already replaces any client-supplied value. The real
endpoints keep stripping the header in the @gateway block (delete-only, no
conflicting set). Caught on the live test contour (no caddy locally).
Add a prod-only, in-memory IP ban enforced at the edge, fed by three signals:
sustained rate-limiter rejections (the IP-keyed public/email/admin classes — the
user class stays the backend soft-flag's concern), a honeypot decoy-path hit (the
contour caddy tags decoys with X-Scrabble-Honeypot and routes them to the gateway),
and a honeytoken (a planted bearer, GATEWAY_HONEYTOKEN). A banned IP is refused with
429 by the abuseGuard middleware before any work — covering the Connect edge, the
live stream and the static SPA/landing the per-op limiter never gated.
The ban is off by default: it keys by the real client IP the shared-NAT test contour
does not expose, so a ban there would be self-inflicted; detection still logs in the
contour, only the ban action is gated (GATEWAY_ABUSE_BAN_ENABLED). Rejection bans last
GATEWAY_ABUSE_BAN_DURATION; tripwire/honeytoken hits are near-zero-false-positive and
earn longer fixed bans. Each ban increments gateway_abuse_banned_total{reason}.
Operators see and lift active bans on the admin console's Throttled page; the gateway
syncs its active set to the backend every 30s (POST /api/v1/internal/bans/sync,
backend/internal/banview) and applies the operator unbans the response returns.
PRERELEASE phase AG. Docs baked into ARCHITECTURE / FUNCTIONAL (+ru) / both READMEs.
The gateway and bot run on distroless nonroot (UID 65532) and bind-mount the
cert dir read-only, but gen-certs.sh wrote the keys 0600 (owner-only, the deploy
user), so both crash-looped at boot with "open /certs/*.key: permission denied"
and the deploy probe correctly failed (the contour's gateway was down).
The .crt files were already 0644 (openssl default); make the leaf keys 0644 too
so UID 65532 can read them. These are ephemeral TEST certificates regenerated
every deploy on the trusted runner; prod keys come from PROD_ secrets. The CA
private key stays 0600 (containers never read it).
Move all Telegram egress off the main host. The single connector held the
bot token, long-polled Telegram and answered the gateway/backend over the
trusted internal network, so the whole component (including login validation)
shared fate with its VPN sidecar. Split it into two binaries that share the
token:
- cmd/validator (home, no VPN): Mini App initData + Login Widget HMAC only,
never calls the Bot API. The gateway dials it for Telegram auth, so game
login is now independent of Telegram reachability.
- cmd/bot (remote): Bot API long-poll + sendMessage, the only component
reaching Telegram. It holds no inbound port — it dials the gateway over a
new reverse mTLS bot-link (pkg/proto/botlink/v1) and executes the send
commands the gateway pushes.
The gateway funnels sends to the bot-link: out-of-app push is fire-and-forget
(at-most-once, dropped if no bot is connected); the backend admin broadcasts
reach a gateway-served relay that forwards them and awaits the bot's ack
(SendToUser/SendToGameChannel contract preserved). mTLS (pkg/mtls) is the one
inter-service link that leaves the trusted segment; validator<->gateway and
the relay stay plaintext internal. The bot is Telegram-rate-limited.
One bot now; the gateway bot registry, an owns_updates flag and per-command
ids leave seams for N later. Webhook rejected (one URL per token, adds inbound
+ a static address).
The unified test contour runs the split (the bot keeps its VPN sidecar and
dials the gateway by its internal name; bot-link certs from deploy/gen-certs.sh,
generated in CI). The prod wiring — the bot on a separate host (no VPN), the
gateway bot-link port published, PROD_ certs with scheduled rotation, an SSH
deploy of both hosts together — is the deferred final stage (PRERELEASE.md TX,
Stage 18).
Docs: ARCHITECTURE, PRERELEASE (phase TX), platform/telegram + gateway +
backend + deploy READMEs, FUNCTIONAL(+ru), CLAUDE.md, .env.example.
The lobby tinted only the viewer's own number, and a tie counted as
"leading" — so an even score showed only the viewer's number green,
reading as if the viewer were ahead. A fresh 0:0 board did the same,
accenting the start of a game where nobody has scored.
scoreStanding is now per-seat: the viewer's seat stays green when
leading or tied and red when trailing; an opponent's seat greens only
when it ties the viewer for the lead, so an equal non-zero score paints
both numbers green. When the top score is 0 (nobody has moved) every
number is left muted, like a finished game.
The post-deploy probe checked only the static landing and the gateway-served SPA shell, so a crash-looping backend passed the deploy green. Add an http://backend:8080/readyz probe on the internal network (and dump backend logs on failure) so an unready backend fails the deploy loudly.
The seed-drift guard shipped as refuse-boot: the backend exited when
BACKEND_DICT_VERSION disagreed with the flat dir's recorded .seed_version. On
the test contour that turned a harmless-in-intent action — bumping the
TEST_DICT_VERSION variable to the active release (v1.2.1) on a volume seeded as
v1.0.0 — into a crash loop, because DICT_VERSION is the *seed* of a fresh
volume, not the active version (which the admin console drives).
Make the marker authoritative instead: OpenWithVersions resolves the flat dir's
version from .seed_version when present and ignores bootVersion on an
already-seeded volume; bootVersion only seeds a fresh volume's marker. So a
bumped build seed on a live volume is a no-op (it can't relabel live bytes and
can't void games pinned to the prior label), and it correctly seeds the next
fresh volume. The subdirectory scan now skips the resolved seed, so a version
also present as a subdir (e.g. v1.2.1 uploaded via the console while the build
seed is bumped to v1.2.1) is still loaded rather than shadowed by the flat bytes.
Tests: marker-wins over a bumped boot version; a bumped boot keeps the matching
subdir resident (the live-contour case). Docs updated (ARCHITECTURE §5, READMEs,
compose/.env, PRERELEASE DV) from "refuses to boot" to "marker wins / ignored".
Verified locally against v1.2.1: gofmt, build, vet, unit, integration green.
The dictionary release moved to v1.2.1 while DICT_VERSION stayed pinned at
v1.0.0 in CI and the image/compose seed defaults. Two problems:
1. CI validated against a stale dictionary.
2. The contour seed could be bumped on a live volume, which silently relabels
the already-seeded bytes — voiding games pinned to the prior label and
serving the wrong dictionary for new ones. The flat DAWGs carry no embedded
version, so this drift was undetectable.
Changes:
- Seed-drift guard: OpenWithVersions records the flat dir's version in a
.seed_version marker on first boot and refuses to start when a later
BACKEND_DICT_VERSION disagrees. DICT_VERSION is now the seed for a *fresh*
volume only; a live contour migrates through the admin console (old versions
stay resident, in-progress games keep replaying).
- Track the current release: CI's DICT_VERSION centralised to one workflow-level
env (v1.2.1); image/compose/.env seed defaults bumped to v1.2.1. The deploy
job keeps reading the per-contour vars.TEST_DICT_VERSION.
- Docs: ARCHITECTURE §5 (decision record), backend/deploy READMEs, PRERELEASE
tracker (DV row).
Verified locally against the v1.2.1 artifact: gofmt, build, vet, unit and
integration (-tags=integration) all green.
Consolidate the incremental goose migrations (00001-00014) into one
baseline. There is no production data, so the squash carries no data
migration. The baseline was generated from the end-state schema and
verified schema-identical to the squashed set (pg_dump diff) plus a full
integration run; the default house ad-campaign seed is carried over (a
schema-only dump omits it). The per-feature narrative that lived in the
squashed migrations is preserved in git history and docs/ARCHITECTURE.md.
Deploy note: this breaks goose's version continuity, so the test contour
DB must be wiped once — DROP SCHEMA backend CASCADE + restart backend —
for goose to re-apply the single baseline fresh. No prod data exists.
Collapse the two per-language Telegram bots into one unified bot and
replace language-based variant gating with explicit per-user variant
preferences.
- Telegram: one bot; drop service_language and the supported_languages
set everywhere (DB, account, auth, FlatBuffers Session wire, gateway,
connector proto). The single bot renders chat and out-of-app push in
the recipient's preferred_language; remove the game-language push
routing override (notify Intent.Language / push Event.language).
- Preferences: new accounts.variant_preferences (text[], DB default
{erudit_ru}, CHECK non-empty + subset of the three variants). Gates
the New Game picker, vs-AI and the friend invitation the player
creates, enforced server-side (HTTP 400 otherwise); an invited friend
may still accept any variant. Edited on the Settings screen; variants
are Erudit-first everywhere.
- Admin: drop the per-bot language selectors (broadcast / send-to-user)
and the feedback channel_lang column/field.
- Env/CI: collapse TELEGRAM_BOT_TOKEN_{EN,RU}, TELEGRAM_GAME_CHANNEL_ID_{EN,RU},
VITE_TELEGRAM_LINK{,_EN,_RU} and VITE_TELEGRAM_GAME_CHANNEL_NAME_{EN,RU}
to single unsuffixed names; drop GATEWAY_DEFAULT_SUPPORTED_LANGUAGES.
- Docs updated (ARCHITECTURE, FUNCTIONAL + _ru, platform/telegram, gateway,
backend, ui, UI_DESIGN, PRERELEASE).
The migration squash is deferred to a follow-up PR.
Decide who moves first by the official rule: each seated player draws one
tile, the one closest to "A" leads (a blank beats every letter), ties
re-drawing until a single leader remains. Each draw uses honest per-draw
crypto/rand entropy (not the deterministic bag seed), so the recorded draw —
not a seed — is the only account of the outcome. The leader takes seat 0, so
the engine and journal replay are unchanged.
The draw is recorded with the game (game_setup_draws, migration 00013) for
future tournaments, designed as a discrete "player N draws a tile" step.
Friend/AI games draw at creation. Auto-match draws when the game opens,
against a synthetic uuid.Nil opponent whose draw rows (NULL account_id) are
back-filled to the real opponent on join — so the opener's seat is fixed up
front and the existing open-game pre-move is preserved with no reseating.
Admin /_gm/games/:id gains the recorded draw list and a simple step-by-step
board replay (game.ReplayTimeline + a vanilla-JS stepper): a board with
A-O/1-15 headers and highlighted premium squares, placed letters with their
tile value as a subscript, rack panels around the board (seat 0 top, 1
bottom, 2 left, 3 right) with the current player highlighted, and a per-move
log with the tiles drawn and the bag remainder.
Docs: ARCHITECTURE §6/§9, FUNCTIONAL (+_ru), PRERELEASE (FM row), design spec.
Svelte trims leading/trailing whitespace inside an element, so the literal
`<span class="sep"> : </span>` rendered as a bare ":" ("123:123"). Emit the
separator as a string expression `{' : '}`, which Svelte preserves, restoring
"123 : 123".
Per owner follow-up: the whole lobby score line is now bold (font-weight 700,
matching the over-the-board score plaques) and a hair smaller (0.8rem) to offset
the heavier weight. The viewer's own number keeps its green/red standing tint.
Functional: the in-game add-friend handshake aimed at an auto-match opponent who is
secretly a pooled robot now records the request per (game, seat) in a new
robot_friend_requests table -- never against the shared robot account -- mirroring the
robot_blocks pattern. The shared account stays out of friendships, the "requested" state
is pinned to the seat (not leaked across the player's other games), the robot ignores it,
and a background reaper drops the row 7 days after its game finishes. The outgoing-requests
list carries these per-game rows so the seat control stays disabled across reloads. No
withdraw UI, per owner decision.
Cosmetics:
- Lobby: an in-progress game tints the viewer's own score number green when leading or
tied, red when trailing (reusing --ok/--danger); scores now render in seat-number order,
matching the over-the-board scoreboard.
- Stats: the per-variant best move is laid out on two lines -- the variant label, then the
score (right-aligned in its column) and the word tiles (left-aligned) below it.
- Dark theme: the played-tile background is a touch darker so a player-placed tile reads
with more contrast (light theme unchanged).
Docs (ARCHITECTURE, FUNCTIONAL + _ru mirror, backend README, UI_DESIGN) updated in the
same change.
Three owner-requested polish changes:
- robot: replace the lengthening 60-90 min -> 6 h proactive-nudge ramp with a
flat uniform 9-12 h wait before every nudge; the existing sleep-window gate
still skips and defers a nudge that would land in the robot's night.
- ui: colour the lobby/in-game unread dot by type -- the regular danger colour
when a chat message is unread, a softer amber (--warn) when only nudges are.
Adds a per-viewer unread_messages flag (chat_messages.kind='message') across
the backend DTO, FlatBuffers wire, gateway transcode and the UI store.
- ui: float games with any unread notification to the top of the lobby's
your-turn and opponent-turn sections (finished keeps its order), reusing the
existing unread_chat flag.
Docs (ARCHITECTURE 7, FUNCTIONAL + _ru) updated. No DB migration; the new wire
field is backward-compatible.
In a dead-drawn endgame — the two most recent journal moves are both
passes, so the board and the robot's rack are frozen and the robot is
bound to pass again — the robot still waited out its long late-game think
time (up to 90 min) before passing, needlessly dragging out a decided game.
Shorten that delay to a [0.8, 1.5]x band around the human's last-move think
time (the gap between the last two journal entries), clamped to [30s, 8min]
and taken as a min with the normal schedule, so the robot never moves
slower. A slow human collapses to the 8-min cap; a fast human is tracked,
with the floor keeping the robot from passing suspiciously instantly. The
anchor reads the move journal only (no schema change), stays deterministic
from the seed, and still defers to the sleep window.
RobotTurns now carries EndgamePass + OppLastMove, filled by one batched
journal query on the scan; the honest-AI single-game trigger keeps the
normal path (it moves at once). NextMoveAt (admin ETA) is left as the
normal-schedule upper bound.
A hint's MoveRecord words are upper-cased on decode (for the move-history view),
whereas an EvalResult keeps the backend's lower case. Seeding the move preview
from the hint verbatim flipped the score caption to upper case and nudged its
height. Lower-case the words in previewFromHint so the caption reads the same
as the evaluate path it replaces.
Taking a hint auto-placed the suggested tiles and then called recompute(),
which round-trips a debounced evaluate for that exact placement. The hint is
the engine's own top-ranked, fully scored legal move, so its move already
carries the score, words and direction an evaluate would return — the second
call was pure duplicate work and added a visible disabled->enabled flicker on
the submit button over slow links.
Seed the move preview directly from the hint move via previewFromHint and
cancel any pending evaluate timer so a stale one cannot clobber it. A later
manual edit re-arms recompute() as before, so rearranged tiles are re-evaluated
normally. Client-only; no backend, wire or schema change.
- The rack now animates the opening drop gap for a board-tile recall drag too
(the reorder transition was gated on a rack-source drag only).
- While a board tile is dragged over the rack (a recall, gap shown), the ✅
confirm and its preview caption are hidden so they don't sit over the gap;
they return if the tile is dragged back out over the board.
- A recall drop lands off the boardwrap, so its pointerup never reached the
boardwrap handler and left a stale id in the active-pointer set — the next
board swipe then read as multi-touch and the shuffle / history pull went dead.
Reconcile the pointer when a drag ends or cancels. Regression e2e added.
developer
Ilia Denisov