Add the VK Mini App logo next to the Telegram one on the landing hero,
linked via the new VITE_VK_APP_LINK build-arg (full URL, wired through
compose, CI and prod-deploy from TEST_/PROD_VITE_VK_APP_LINK).
Also restore the landing's Telegram link itself: commit 57c778f
collapsed the per-language vars to VITE_TELEGRAM_GAME_CHANNEL_NAME in
compose/CI but left gateway/Dockerfile with the stale _EN/_RU ARGs and
without the plain one, so the built bundle saw the var as undefined and
dead-code-eliminated the whole channel-link branch — deployed landings
(prod included) have shown no Telegram logo since.
Measure uptake of the client-side local move-preview accelerator (§5) so
adoption can be watched before defaulting it on: app cold starts, dictionary
loads by result (fetched / cache_hit / miss) and move previews by path
(local / network — the backend load shed).
A small best-effort client beacon (POST /metrics/local-eval, session-gated)
batches counter deltas and posts them on a 60s timer and when the app is
backgrounded — never on the gameplay path: the in-app counters are plain
in-memory increments, only the periodic flush touches the network and it is
fire-and-forget. The gateway folds each batch into three OTel counters
(local_eval_cold_start_total, local_eval_dict_load_total,
local_eval_preview_total), clamped against a spoofed inflation.
- gateway: counters + recordLocalEval + session-gated /metrics/local-eval handler
- ui: localeval-metrics accumulator/beacon; hooks in the dict loader, in
Game.recompute and in bootstrap (skipped under the mock harness)
- caddy: route /metrics/* to the gateway
- docs: ARCHITECTURE §11; Grafana "Scrabble — Users" dashboard panels
Score and validate a tentative move on-device instead of a per-arrangement network
round trip. The dawg reader and the validate/score/direction slice of the
scrabble-solver engine are ported to TypeScript (ui/src/lib/dict), pinned
byte-for-byte to the Go engine by a `conformance` CI job (full-dictionary reader
parity plus a battery of plays across every variant and both cross-word rules,
including the inferred orientation). The server stays authoritative — submit_play
re-validates — so the local result is an advisory accelerator only.
- backend: Registry.DictBytes + an authed GET /api/v1/user/dict/{variant}/{version}
(immutable) streaming the pinned per-game dawg; caddy routes /dict to the gateway.
- gateway: a session-gated /dict edge route proxying it; fetchDict on the transport.
- client: the dictionary loads on game open (low priority so it never starves the
game on a slow link; aborted at a 5s cap or when leaving the game), is cached in
IndexedDB (best-effort, self-healing on a rejected blob) and reused across
sessions; a warm-up overlay covers a cold load, then the network preview is the
fallback; a bad-connection breaker stops warming after repeated misses; the move
preview cancels its in-flight request when the tiles change.
- parity generators backend/cmd/{dictgen,validategen} + gated Vitest suites, run in
CI against the release dictionaries. A hidden debug readout lists the cached
dictionaries + breaker state, and its reset clears the cache.
- docs: ARCHITECTURE §5, TESTING, UI_DESIGN, FUNCTIONAL (+ru).
Mirror the Telegram Mini App wrapper for VK: the SPA loads at a new /vk/
entry, authenticates from VK's signed launch parameters, and provisions a
'vk' platform identity — the minimum to run the game in VK test mode.
- Gateway verifies the launch signature in-process (internal/vkauth:
HMAC-SHA256 over the sorted vk_* params under GATEWAY_VK_APP_SECRET,
base64url) — a pure offline check, no side-service. New auth.vk op
(gated on the secret), backendclient.VKAuth, /vk/ SPA mount.
- Backend: KindVK + ProvisionVK/vkSeed, /sessions/vk handler, identity
kind widened to include 'vk' (migration 00005, expand-contract).
- UI: src/lib/vk.ts (VK Bridge, lazy-imported), bootVK + the /vk/ boot
dispatch, encodeVKLogin + authVK across transport/client/mock. VK omits
the name from the signed params, so the client reads it via
VKWebAppGetUserInfo as an unsigned display seed.
- Deploy: /vk in the edge Caddyfile, GATEWAY_VK_APP_SECRET wired through
compose + .env.example + CI (TEST_) + prod-deploy (PROD_).
- Admin console: surface the VK user id (link to the VK profile) next to
the Telegram id on the user card.
- Docs: ARCHITECTURE §12/§13, FUNCTIONAL (+ _ru), gateway README; VK
integration reference under .claude/.
Signature algorithm verified against dev.vk.com plus independent Node/Python
references and a %2C edge-case vector.
The promo bot button carries a configurable variant-seed start-param (default verudit_ru-scrabble_en). The gateway parses start_param from the validated initData and forwards it; the backend, on first contact only, seeds the new account variant_preferences from it (English Scrabble alongside the default Erudit).
No schema change (the scrabble_en CHECK is already in the baseline) and the gateway<->backend REST field is additive, so the rolling deploy is safe in either order. TELEGRAM_PROMO_START_PARAM configures the payload (empty forwards the user own /start payload). Covered by account unit tests, a gateway transcode test, and an integration test asserting new-only seeding.
Users who DM the bot anything but /start are relayed into a private
forum supergroup, one topic per user. Operators (the chat's admins)
reply in the topic and the bot copies it back to the user; an info card
opening each topic carries a Block/Unblock toggle and a Clear button.
State is a small JSON file on a new /data volume — the bot host has no
database. Off by default (TELEGRAM_SUPPORT_CHAT_ID=0): the prod bot is
unchanged until the operator sets the chat id and adds the bot as a
forum admin.
- internal/support: concurrency-safe JSON store (topic map, block list,
relayed message ids) with field-targeted mutators and atomic save
- bot/support.go: relay both ways via copyMessage, short-TTL admin
cache, callback buttons, per-user topic-create lock, loop guard
(skip the bot's own posts), reopen a deleted topic on the next message
- config + compose + CI/prod-deploy: TELEGRAM_SUPPORT_CHAT_ID per
contour + TELEGRAM_SUPPORT_STATE_DIR; bot-state named volume; /data
pre-owned by UID 65532 so a fresh volume is writable under distroless
- docs: ARCHITECTURE §15 + decision record, FUNCTIONAL (+ru), README
The prod rolling deploy rolls each service with `compose up -d --no-deps <svc>`.
For caddy that is a no-op on a config-only release: its image is pinned
(caddy:2-alpine, no $TAG), so the compose definition is unchanged between
releases, compose treats the container as current and does not recreate it, and
admin is off so there is no hot reload. The new bind-mounted Caddyfile is seeded
to the host but never loaded -- the v1.2.2 `Alt-Svc: clear` edge fix deployed
green yet did not take effect until caddy was restarted by hand.
Force a recreate for caddy on its roll (every other service already recreates on
its new $TAG image), so a bind-mounted Caddyfile change always applies. Costs a
~1-2s caddy blip per deploy, acceptable for the infrequent manual prod rollout.
The "Database size" stat had no thresholds, so Grafana applied its stat
default (green base, red at >=80). The query is pg_database_size_bytes, so a
healthy ~9 MiB database (9.4M >> 80) rendered permanently RED on the
Scrabble - Resources dashboard (test + prod), reading as a false alert; the
neighbouring percentunit cache-hit stat stayed green only because its 0..1
values fall under 80.
Add absolute byte thresholds sized to the 40 GiB prod disk (4.6 GiB used,
observability bounded -- Tempo <=1 GiB, Prometheus 7d -- so the DB is the
only unbounded grower): green up to 8 GiB, yellow at 8 GiB (~20% of disk),
red at 16 GiB (~40%), an early warning with ample runway before the disk
tightens, not a panic line. Cosmetic panel coloring only; there are no
Grafana alert rules provisioned.
Caddy enables HTTP/3 by default on any TLS listener and emits
Alt-Svc: h3=":443"; ma=2592000, but UDP/443 is never reachable: the prod
compose maps only "443:443" (TCP) and ufw opens 443/tcp (test contour: the
host caddy publishes only :443/tcp). A client that cached the 30-day advert
tries QUIC first on later opens, gets no response, and waits for the QUIC
attempt to time out before falling back to h2 -- which surfaced as the
Telegram Mini App intermittently hanging on load (a barely-noticeable pause
up to a blank window). The h2/TCP serving path itself is healthy (~10ms TTFB).
Emit Alt-Svc: clear site-wide at the contour caddy so clients actively drop
any cached alternative and stay on h2/h1. This caddy terminates TLS in prod
(the fix target); in the test contour it serves plain :80 and the host caddy
re-stamps its own Alt-Svc, so the live test fix lives in the host caddy. Add
docs/EDGE_HTTP3.md (symptom, diagnosis method, verify, and option B -- serving
h3 for real -- if it recurs) and link it from ARCHITECTURE.md.
Drop the literal version default from the build files (backend Dockerfile both
stages, loadtest Dockerfile, the compose build-arg) so the release tag is not
duplicated as a stale-prone default a newcomer can't tell from the real source.
DICT_VERSION is now required: compose uses ${DICT_VERSION:?…} and the Dockerfiles
have no ARG default, so a missing value fails loudly instead of baking a stale tag.
The tag lives only in its genuine sources — ci.yaml env (CI tests), the Gitea
TEST_/PROD_DICT_VERSION variables (deploy seed) and deploy/.env.example (local).
Adds a "Bumping the dictionary version" section to deploy/README and fixes the bare
docker-build examples (CLAUDE.md, README.md, loadtest/README) to pass --build-arg.
Bump DICT_VERSION v1.2.1 -> v1.3.0 across the seed surface: .env.example, the
compose build-arg default, both backend Dockerfile stages, the loadtest
Dockerfile, the CI dawg-download version, and the deploy/backend docs.
v1.3.0 drops the abbreviation class from the Russian word list (scrabble-dictionary
#6). This only seeds a FRESH volume; a live contour/prod volume is unaffected (the
.seed_version marker wins — seed-drift guard) and moves to v1.3.0 through the admin
console (ARCHITECTURE §5). Per-contour deploy still overrides via TEST_/PROD_DICT_VERSION.
The project is live in production, so the staged-development scaffolding is removed.
- Delete the staged trackers PLAN.md and PRERELEASE.md.
- Rewrite CLAUDE.md: drop the per-stage workflow; codify the ongoing development
principles (How we work) and the production model (Branching, CI & production):
manual prod-deploy / prod-rollback, semver release tags, Ansible provisioning,
expand-contract migrations.
- De-stage the living docs (README, ARCHITECTURE, TESTING, deploy/ansible, loadtest,
platform/telegram READMEs) and the docker-compose tuning comments: drop the
Stage N / R1-R7 / pre-release labels, keep every number and rationale, and fix the
now-dangling PLAN.md / PRERELEASE.md references to describe the current state.
- Reword stale 'later stage' Go doc comments for subsystems that have shipped.
- prod-deploy.yaml is now four visible sequential jobs (build -> deploy-main ->
deploy-bot -> verify) so the rollout stages show in the Actions UI; the
per-service rolling stays in the deploy-main log.
- prod-rollback.yaml: a separate manual workflow_dispatch. Leave target_version
blank to roll back to the previous deployed version (the host now tracks
DEPLOYED_TAG + PREVIOUS_TAG), or pick a release tag. Re-deploys an already
published image rolling + health-gated, image-only (no rebuild, no DB migration).
- prod-deploy.sh tracks the previous tag (commit_tag) for the blank-input rollback.
- Docs: ARCHITECTURE §13 + deploy/README runbook cover versioning + rollback.
pkg/version.Version (default "dev") is set at link time via -ldflags from each
service Dockerfile's VERSION build-arg, which the deploy passes as the git tag
(git describe --tags). It surfaces as the OpenTelemetry service.version resource
attribute (so Grafana/Tempo are version-aware), alongside the SPA's existing
About version. Adds the VERSION build-arg to the backend/gateway/validator/bot
compose builds and a serviceResource test covering service.name + service.version.
- ARCHITECTURE §13 prod bullet -> the realized mechanism: registry transport,
two-host, rolling + auto-rollback, migration maintenance window, node_exporter,
the undersized launch; the contour paragraph notes node_exporter + the
telegram-local profile.
- deploy/README gains a prod rollout runbook (how to run, migrations/restore, cert
rotation, sizing/monitoring, the full PROD_ set) + node_exporter row, the
telegram-local profile note, and the soft AWG_CONF note.
- PLAN Stage 18 records the resolved open details and the remaining live cutover
(pending erudit-game.ru DNS); the tracker reads 'machinery built; cutover pending DNS'.
- PRERELEASE TX/AG note the prod wiring is built.
A workflow_dispatch-only rollout from master (confirm=deploy):
- .gitea/workflows/prod-deploy.yaml builds + pushes the images to the registry,
ships the compose/config/certs/env over SSH, deploys the main host via
prod-deploy.sh, then the bot host, then verifies the public site.
- deploy/prod-deploy.sh rolls the main stack one service at a time in dependency
order (postgres->backend->gateway->landing->validator->caddy), health-checking
after each; any failure rolls the whole stack back to the previous tag. A schema
migration adds a maintenance window: the backend (sole writer) is stopped for a
consistent pg_dump before migrating; image rollback stays DB-safe (expand-contract),
the dump is kept for a manual restore.
- prod overlay: pull the four main images from the registry by tag.
- Runtime secrets reach the host via a sourced env.sh (single-quoted values keep the
bcrypt hash's literal $ intact, unlike a --env-file).
Split the contour across the two prod hosts and retune for the small main host:
- Gate vpn+bot to the telegram-local profile. The CI test deploy now passes
--profile telegram-local so the test contour still brings them; the prod main
host omits both, and the prod bot runs standalone from docker-compose.bot.yml.
- docker-compose.prod.yml (main-host overlay): publish caddy 80/443 (no host
caddy in prod; caddy owns ACME) and gateway 9443 (the remote bot dials in over
mTLS); GOMAXPROCS=2, smaller memory caps and 7d Prometheus retention for the
2 vCPU / 1.9 GiB host. It launches deliberately undersized; resize reactively.
- docker-compose.bot.yml: standalone bot for the tg host (no VPN, OTLP off since
otelcol is unreachable from there, dials the main host's bot-link).
- Add node_exporter + a Prometheus scrape so host memory pressure (the OOM
signal on the tight main host), not just per-container docker_stats, is visible.
- Soften AWG_CONF to a default: only the profiled vpn sidecar consumes it, and
compose interpolates profiled-out services too, so prod must not require it.
Idempotent playbooks under deploy/ansible/ prepare both production hosts:
docker-ce + compose plugin, a non-sudo deploy service account holding the CI
deploy key, key-only sshd, default-deny ufw, fail2ban, unattended upgrades and
chrony. The main host also opens 80/443/9443 and creates the external edge
network; the tg host verifies direct Bot API egress (the no-VPN assumption).
The application is deployed separately by the prod-deploy workflow (later
phase), running as the deploy account this playbook provisions.
- Bake the final default-allow + mute-the-ineligible strategy into
docs/ARCHITECTURE.md, docs/FUNCTIONAL.md (+_ru), platform/telegram/README.md,
the deploy compose comment and the PRERELEASE tracker. The live test proved a
per-user grant cannot exceed a deny-by-default group (Telegram intersects the
chat default with the per-user permission), so the chat allows sending by
default and the bot restricts the ineligible instead of granting the eligible.
- Lower the per-event chat_member trace and eligibility evaluation to Debug;
keep the actual mute/unmute actions, the startup self-check and warnings at
Info, so prod logs only what the bot did.
- Update game.searchingForOpponent (Searching -> Waiting for opponent / Поиск ->
Ждём соперника) and the quickmatch e2e assertions to match.
Add a second standalone promo bot to the bot container (answers /start with a
localized message + a URL button into the main bot's Mini App) and gate write
access in a channel's linked discussion chat: grant on join when the Telegram
user is registered and neither admin-suspended nor holding a new chat_muted
role, and revoke/grant on the matching moderation change for a member currently
in the chat.
Eligibility (registered AND NOT suspended AND NOT chat_muted; the game
suspension dominates) is resolved once in the backend and reached two ways: the
bot's join-time unary ResolveChatEligibility over the existing mTLS bot-link,
and a backend chat_access_changed event -> gateway -> ChatGate command
(idempotent; a temporary-block-expiry sweeper may over-emit). The bot guards the
block/unblock path with getChatMember, since bots cannot list members.
A web_app button cannot open another bot's Mini App (it signs initData with the
sending bot's token), so the promo button is a t.me ?startapp URL reusing the
UI's VITE_TELEGRAM_LINK. The bot must be a chat admin with the restrict-members
right and chat_member in its allowed updates.
No schema change: chat_muted reuses the data-driven account_roles table.
The @honeypot block both deleted and set X-Scrabble-Honeypot in one reverse_proxy.
Caddy applies header_up deletions *after* sets, so the tag we set was immediately
stripped: the gateway never saw it, and a decoy hit (e.g. GET /.env) fell through
to the gateway's /app redirect (308) instead of tripping the honeypot. Drop the
delete — the bare set already replaces any client-supplied value. The real
endpoints keep stripping the header in the @gateway block (delete-only, no
conflicting set). Caught on the live test contour (no caddy locally).
Add a prod-only, in-memory IP ban enforced at the edge, fed by three signals:
sustained rate-limiter rejections (the IP-keyed public/email/admin classes — the
user class stays the backend soft-flag's concern), a honeypot decoy-path hit (the
contour caddy tags decoys with X-Scrabble-Honeypot and routes them to the gateway),
and a honeytoken (a planted bearer, GATEWAY_HONEYTOKEN). A banned IP is refused with
429 by the abuseGuard middleware before any work — covering the Connect edge, the
live stream and the static SPA/landing the per-op limiter never gated.
The ban is off by default: it keys by the real client IP the shared-NAT test contour
does not expose, so a ban there would be self-inflicted; detection still logs in the
contour, only the ban action is gated (GATEWAY_ABUSE_BAN_ENABLED). Rejection bans last
GATEWAY_ABUSE_BAN_DURATION; tripwire/honeytoken hits are near-zero-false-positive and
earn longer fixed bans. Each ban increments gateway_abuse_banned_total{reason}.
Operators see and lift active bans on the admin console's Throttled page; the gateway
syncs its active set to the backend every 30s (POST /api/v1/internal/bans/sync,
backend/internal/banview) and applies the operator unbans the response returns.
PRERELEASE phase AG. Docs baked into ARCHITECTURE / FUNCTIONAL (+ru) / both READMEs.
The gateway and bot run on distroless nonroot (UID 65532) and bind-mount the
cert dir read-only, but gen-certs.sh wrote the keys 0600 (owner-only, the deploy
user), so both crash-looped at boot with "open /certs/*.key: permission denied"
and the deploy probe correctly failed (the contour's gateway was down).
The .crt files were already 0644 (openssl default); make the leaf keys 0644 too
so UID 65532 can read them. These are ephemeral TEST certificates regenerated
every deploy on the trusted runner; prod keys come from PROD_ secrets. The CA
private key stays 0600 (containers never read it).
Move all Telegram egress off the main host. The single connector held the
bot token, long-polled Telegram and answered the gateway/backend over the
trusted internal network, so the whole component (including login validation)
shared fate with its VPN sidecar. Split it into two binaries that share the
token:
- cmd/validator (home, no VPN): Mini App initData + Login Widget HMAC only,
never calls the Bot API. The gateway dials it for Telegram auth, so game
login is now independent of Telegram reachability.
- cmd/bot (remote): Bot API long-poll + sendMessage, the only component
reaching Telegram. It holds no inbound port — it dials the gateway over a
new reverse mTLS bot-link (pkg/proto/botlink/v1) and executes the send
commands the gateway pushes.
The gateway funnels sends to the bot-link: out-of-app push is fire-and-forget
(at-most-once, dropped if no bot is connected); the backend admin broadcasts
reach a gateway-served relay that forwards them and awaits the bot's ack
(SendToUser/SendToGameChannel contract preserved). mTLS (pkg/mtls) is the one
inter-service link that leaves the trusted segment; validator<->gateway and
the relay stay plaintext internal. The bot is Telegram-rate-limited.
One bot now; the gateway bot registry, an owns_updates flag and per-command
ids leave seams for N later. Webhook rejected (one URL per token, adds inbound
+ a static address).
The unified test contour runs the split (the bot keeps its VPN sidecar and
dials the gateway by its internal name; bot-link certs from deploy/gen-certs.sh,
generated in CI). The prod wiring — the bot on a separate host (no VPN), the
gateway bot-link port published, PROD_ certs with scheduled rotation, an SSH
deploy of both hosts together — is the deferred final stage (PRERELEASE.md TX,
Stage 18).
Docs: ARCHITECTURE, PRERELEASE (phase TX), platform/telegram + gateway +
backend + deploy READMEs, FUNCTIONAL(+ru), CLAUDE.md, .env.example.
The seed-drift guard shipped as refuse-boot: the backend exited when
BACKEND_DICT_VERSION disagreed with the flat dir's recorded .seed_version. On
the test contour that turned a harmless-in-intent action — bumping the
TEST_DICT_VERSION variable to the active release (v1.2.1) on a volume seeded as
v1.0.0 — into a crash loop, because DICT_VERSION is the *seed* of a fresh
volume, not the active version (which the admin console drives).
Make the marker authoritative instead: OpenWithVersions resolves the flat dir's
version from .seed_version when present and ignores bootVersion on an
already-seeded volume; bootVersion only seeds a fresh volume's marker. So a
bumped build seed on a live volume is a no-op (it can't relabel live bytes and
can't void games pinned to the prior label), and it correctly seeds the next
fresh volume. The subdirectory scan now skips the resolved seed, so a version
also present as a subdir (e.g. v1.2.1 uploaded via the console while the build
seed is bumped to v1.2.1) is still loaded rather than shadowed by the flat bytes.
Tests: marker-wins over a bumped boot version; a bumped boot keeps the matching
subdir resident (the live-contour case). Docs updated (ARCHITECTURE §5, READMEs,
compose/.env, PRERELEASE DV) from "refuses to boot" to "marker wins / ignored".
Verified locally against v1.2.1: gofmt, build, vet, unit, integration green.
The dictionary release moved to v1.2.1 while DICT_VERSION stayed pinned at
v1.0.0 in CI and the image/compose seed defaults. Two problems:
1. CI validated against a stale dictionary.
2. The contour seed could be bumped on a live volume, which silently relabels
the already-seeded bytes — voiding games pinned to the prior label and
serving the wrong dictionary for new ones. The flat DAWGs carry no embedded
version, so this drift was undetectable.
Changes:
- Seed-drift guard: OpenWithVersions records the flat dir's version in a
.seed_version marker on first boot and refuses to start when a later
BACKEND_DICT_VERSION disagrees. DICT_VERSION is now the seed for a *fresh*
volume only; a live contour migrates through the admin console (old versions
stay resident, in-progress games keep replaying).
- Track the current release: CI's DICT_VERSION centralised to one workflow-level
env (v1.2.1); image/compose/.env seed defaults bumped to v1.2.1. The deploy
job keeps reading the per-contour vars.TEST_DICT_VERSION.
- Docs: ARCHITECTURE §5 (decision record), backend/deploy READMEs, PRERELEASE
tracker (DV row).
Verified locally against the v1.2.1 artifact: gofmt, build, vet, unit and
integration (-tags=integration) all green.
Collapse the two per-language Telegram bots into one unified bot and
replace language-based variant gating with explicit per-user variant
preferences.
- Telegram: one bot; drop service_language and the supported_languages
set everywhere (DB, account, auth, FlatBuffers Session wire, gateway,
connector proto). The single bot renders chat and out-of-app push in
the recipient's preferred_language; remove the game-language push
routing override (notify Intent.Language / push Event.language).
- Preferences: new accounts.variant_preferences (text[], DB default
{erudit_ru}, CHECK non-empty + subset of the three variants). Gates
the New Game picker, vs-AI and the friend invitation the player
creates, enforced server-side (HTTP 400 otherwise); an invited friend
may still accept any variant. Edited on the Settings screen; variants
are Erudit-first everywhere.
- Admin: drop the per-bot language selectors (broadcast / send-to-user)
and the feedback channel_lang column/field.
- Env/CI: collapse TELEGRAM_BOT_TOKEN_{EN,RU}, TELEGRAM_GAME_CHANNEL_ID_{EN,RU},
VITE_TELEGRAM_LINK{,_EN,_RU} and VITE_TELEGRAM_GAME_CHANNEL_NAME_{EN,RU}
to single unsuffixed names; drop GATEWAY_DEFAULT_SUPPORTED_LANGUAGES.
- Docs updated (ARCHITECTURE, FUNCTIONAL + _ru, platform/telegram, gateway,
backend, ui, UI_DESIGN, PRERELEASE).
The migration squash is deferred to a follow-up PR.
Visualises the chat read-receipt metrics added with the read-receipt feature
(the dashboard was deferred there per the owner): posted rate by kind, the
unread backlog (chat_unread_messages gauge), and the publish-to-read latency
(chat_read_duration — average by kind plus overall p50/p95). Mirrors the
game-domain dashboard's structure; the file provider auto-discovers it.
Follow-ups on the honest-AI game, same PR:
- GCG export labels the robot seat "AI" instead of its pool name (ExportGCG
overrides via accounts.IsRobot); the in-app 🤖 is unchanged.
- vs_ai games emit no your_turn (the robot replies instantly, so it would be
redundant); opponent_moved still advances the UI.
- Admin console shows the AI flag: a 🤖 column in /games and an "AI game" line
on the game card (GameRow/GameDetailView gain VsAI).
- games_started_total / games_abandoned_total gain a vs_ai attribute; the
Grafana Game-domain dashboard splits started/abandoned into human and AI
panels.
Tests: metrics unit (vs_ai split); integration (no your_turn, GCG "AI").
The friend-code 'share' was an <a> that just opened the bot. Make it a real share:
Telegram's native share-to-chat picker inside the Mini App (openTelegramLink +
t.me/share/url), the system share sheet (navigator.share) on the web, else copy the
link. The shared deep link points at the same bot the player is in — it picks
VITE_TELEGRAM_LINK_EN/_RU by the session's service language, falling back to the
single VITE_TELEGRAM_LINK. Adds the per-bot build args across Dockerfile / compose /
ci.yaml / .env / docs; PLAN TODO-5 updated.
Replace the dictionary hot-reload with an online update flow in the GM console
(/_gm/dictionary): the operator uploads scrabble-dawg-vX.Y.Z.tar.gz, previews the
per-variant words added/removed against the active dictionary, and confirms to
install + activate it. Versions are immutable; in-progress games keep their pinned
version while new games use the new one.
- engine: DiffWords (enumerate both DAWGs, decode only the differences), OpenFinder,
Registry.Finder, DictFiles; OpenWithVersions skips the .staging area.
- dictadmin: hardened release-archive validation + extraction (path-traversal,
symlink, oversize, entry-count rejection) and staging -> install (atomic rename).
- game: active dictionary version persisted in the dictionary_state singleton
(single source of truth, restored on boot), concurrency-safe accessor.
- storage: BACKEND_DICT_DIR is a named volume seeded from the image (nonroot-owned),
so uploaded versions persist across redeploys; the build's DICT_VERSION labels the
seed and equals the resident tag (BACKEND_DICT_VERSION).
- docs: ARCHITECTURE §5, FUNCTIONAL (+ru), backend README, TESTING, PRERELEASE.
Tests: engine + dictadmin unit; integration upload->preview->install->activate->
restart->pin->immutability->CSRF.
Round-2 tuning, decided from the 500-player resource profile:
- gateway: 2 -> 3 cores + GOMAXPROCS=3. It holds one h2c connection per player, so
at 500 players it burst into the 2-core cap (~2.49% transport_error on game.state);
3 cores absorbs the bursts. The per-connection cost is the realistic prod load.
- tempo: memory 1G -> 2G. It reached the 1 GiB cap during the run (OOM risk).
- backend Postgres pool: MAX_OPEN_CONNS 25 -> 40. The pool sat at its 25-conn cap
(28 backends) at peak; headroom trims the p99 tail. Postgres (2c/512M) handles it.
- docker log volume: a json-file rotation default (10m x 3 = 30 MiB/container) applied
contour-wide via a YAML anchor; the backend logs ~14 MiB / 30 min at info under load
and was previously unbounded. Log level stays info.
backend/postgres stay at 2 cores / 512 MiB (peak ~0.85 / ~1.4 cores — headroom is cheap
on the shared host). A validation re-run confirms the gateway fix before merge.
The receiver defaults to Docker API 1.25, but the contour daemon's minimum is
1.40 (it speaks up to 1.54), so otelcol crash-looped on start with "client
version 1.25 is too old". Pinning api_version to 1.44 (accepted by both the
receiver's bundled client and the daemon) starts the receiver cleanly —
verified by running the image against the host socket ("Everything is ready",
no start error).
Observability: replace cAdvisor (which resolves only the root cgroup on the
contour host — separate-XFS /var/lib/docker) with the otelcol docker_stats
receiver, which reads per-container CPU/memory/network straight from the Docker
API and works the same in prod. The collector joins the host docker group
(DOCKER_GID, default 989) and mounts the socket read-only; its metrics flow out
through the existing prometheus exporter, so the cAdvisor scrape job and the
privileged cAdvisor service are removed. The Resources dashboard panels are
retargeted to the docker_stats metric names (container_name label;
container.cpu.utilization/100 == cores).
Container limits: apply deploy.resources.limits (honoured by Compose v2) across
the contour and pin GOMAXPROCS to the CPU limit on the Go services so the runtime
matches the cgroup quota. Starting values are generous over the R2 peak (~1 core /
<=100 MiB per app service) to avoid skewing or OOM-killing the measurement run;
they are tightened to the agreed prod sizing after the final stress run (R7
Round 2). The privileged VPN sidecar is left unconstrained.
- gateway/Dockerfile gains a `landing` target: caddy:2-alpine + the shared
Vite build (identical build args keep the ui stage a single cached build);
the gateway target drops landing.html from the embed.
- The contour caddy routes /app/, /telegram/ and the Connect path to the
gateway; the catch-all — the landing at / and any stray path — goes to the
new landing service, so junk traffic is absorbed by static file serving.
- deploy/landing/Caddyfile mirrors the webui caching (immutable assets,
no-cache shells) and falls back unknown paths to the landing shell.
- The gateway's / now 308-redirects to /app/ (keeps a local no-caddy run
usable); webui placeholder landing.html removed.
- CI deploy probe checks both / (landing) and /app/ (gateway).
Verified: both images build; the landing container serves landing.html at /
(no-cache) with junk-path fallback; the gateway image redirects / to /app/
and carries no landing content.
New scrabble/loadtest module (the pre-release stress harness): seeds 1000 guest +
10000 durable accounts with pre-created sessions directly in Postgres (token hash
matches backend/internal/session), drives virtual players through the edge protocol
(real 2-4p games assembled via invitations, mid-ranked legal moves generated locally
by the embedded scrabble-solver — the edge carries no board, so the client replays
history), plus nudge/chat/check-word/draft/profile/stats and a gateway-hammer that
verifies the rate limiter. Prints a trip-report summary (per-op latency percentiles,
result codes, live-event tally). Go unit tests cover the pure pieces; the DAWG-backed
move test runs under BACKEND_DICT_DIR.
Contour: add cAdvisor + postgres_exporter + a 'Scrabble - Resources' Grafana
dashboard and the two Prometheus scrape jobs, for the R2/R7 stress-run resource
baseline.
CI: gate ./loadtest/... (path filter + vet/build/test). Docs: TESTING, ARCHITECTURE,
project CLAUDE repo layout.
- Client IP: the compose caddy trusts X-Forwarded-For from private-range
upstreams (trusted_proxies private_ranges), so the real client IP survives
the host-caddy hop (it was logging the docker caddy hop 172.18.0.x for chat
moderation and bucketing the gateway per-IP rate limiter on it). Correct and
spoof-safe in both contours (prod has no host caddy); peerIP unit-tested.
- Ad banner gated off behind a compile-time SHOW_AD_BANNER=false (the if-branch,
the AdBanner import and banner.ts are tree-shaken out of the prod bundle).
- Landing: the Telegram entry is just the 64px logo (clickable, no button/text).
- TG-fullscreen header: title + menu centred as a pair (hamburger right of the
title), pinned to the bottom of the TG nav band.
- Edge-swipe back (Screen): a left-edge rightward drag navigates to back
(touch/pen only, armed from <=24px; skipped inside Telegram).
- Chat soft-keyboard: a bottom-sheet Modal lifted above the keyboard by a
visualViewport-driven transform (compositor-only, no page/sheet relayout).
iOS-specific, needs on-device tuning; native resize=none awaits Capacitor.
- Tests: e2e for the in-game '✓ in friends' item and a board→board tile
relocation; codec units for last_activity_unix + OutgoingRequestList.
Deferred to the next PR (agreed): #4 enrich the your-turn/game-end push; #5 hide
finished games from the lobby.
Owner review-pass rework of the landing page:
- Rename the per-language Telegram link build var
VITE_TELEGRAM_LINK_EN/_RU -> VITE_TELEGRAM_GAME_CHANNEL_NAME_EN/_RU
(it carries a channel username; the landing builds https://t.me/<name> --
the same channels the connector posts to via TELEGRAM_GAME_CHANNEL_ID_*).
- Language switcher -> a globe icon dropdown (flags + names), saved + synced
to the app prefs.
- Theme switcher -> a sun/moon icon toggle, ephemeral (follows the system
scheme, no auto, never persisted) -- galaxy-game style.
- Drop the "Play in browser" CTA (no standalone-web onboarding yet).
Docs: FUNCTIONAL(+ru), PLAN, deploy + ui READMEs.
Close out Stage 17 round 6:
- Landing page at / — one Vite build with two entries (index.html = game
SPA, landing.html = a lightweight landing reusing the theme/i18n/
aboutContent leaf modules, not the app store).
- Move the web game SPA to /app/; the Telegram Mini App stays at /telegram/
(gateway webui.Handler(stripPrefix, indexName): landing at /, SPA at /app/
+ /telegram/). Per-language "Play in Telegram" link via new
VITE_TELEGRAM_LINK_EN/_RU build vars (button hides when unset).
- Cache headers: hash-named /assets/* immutable, HTML shells no-cache (the
go:embed zero modtime emitted no validators, so the client re-downloaded
the whole bundle every launch).
- Live-stream 15s abort fix: an immediate heartbeat on open + a 10s default
interval (the first tick at 15s raced the edge idle timeout -> reconnect
storm).
PLAN/ARCHITECTURE(§13)/FUNCTIONAL(+ru)/gateway+ui+deploy READMEs updated;
round 6 closed. Tests: gateway webui/connectsrv units, ui landing unit + e2e,
full e2e (60) green.
- About screen: prominent localized title (Scrabble / Эрудит (Скрэббл)), a rules link
(en/ru Wikipedia), and the Random-game / Game-with-friends sections; copy lives in a
shared aboutContent module (the landing will reuse it). The random-game move limit
inlines the 24h auto-match clock.
- App version: Vite define __APP_VERSION__ from VITE_APP_VERSION (default 'dev'), wired as
a Docker build-arg sourced from `git describe --tags --always` in the deploy step — no
manual version bumps. The fallback keeps a plain/local build working.
- Grafana: disable Live (GF_LIVE_MAX_CONNECTIONS=0) so its WebSocket no longer trips caddy Basic-Auth and re-prompts; admin console gains a Grafana nav link
- deploy: force-recreate config-only services so reseeded Grafana dashboards / Caddyfile are actually picked up (the move-duration panel was invisible because the bind-mount went stale)
- rate-limit: raise per-user budget 120/40 -> 300/80; UI skips reloading on the echo of the player's own move (fewer requests, no double-load)
- iOS/Telegram reconnect: suppress the connection banner while backgrounded and for a short grace after resume; reconnect silently; wire visibilitychange + pageshow/pagehide + Telegram activated/deactivated (Bot API 8.0)
- hint button disabled when 0 hints remain; nudge button shows a disabled state on your own turn
- players plaque: invert so the active seat pops (accent chip, raised) and others recede
- make-move UX: a direct ✅ commit button (no hold/popover); the Shuffle tab becomes ↩️ Reset while tiles are pending
- #10 a `changes` job path-filters unit/integration/ui; an always-running `gate` job aggregates them (success-or-skipped) and becomes the only required check
- #9 deploy adds a Telegram-connector liveness probe (docker inspect: running, not restarting, stable restart count) with a VPN-handshake grace period
- #1a Game-domain dashboard gains a 'Move think-time by phase (p50/p95)' panel
- deploy README: branch protection now requires only CI / gate
Root cause of the Grafana "readdirent /etc/grafana/dashboards: no such file or
directory": the CI runner checks out into an ephemeral act workspace that is
removed after the job, so binding the compose config files straight from it
dangles the mounts in the long-lived containers (verified the act source dir is
emptied after the job). caddy/otelcol/prometheus/tempo read their config once at
startup so they survive, but would break on a restart — same latent bug.
Fix (mirrors ../galaxy-game's $HOME/.galaxy-dev/monitoring): the deploy job seeds
the config dirs to a stable $HOME/.scrabble-deploy and the compose binds them via
${SCRABBLE_CONFIG_DIR:-.} (local runs keep "."). Documented in the compose header,
deploy/README.md and the ci.yaml step.
- deploy/docker-compose.yml: mount the provisioned dashboards at
/etc/grafana/dashboards, not /var/lib/grafana/dashboards — the grafana-data
volume mounts over the latter and shadows the nested bind, so the provider
logged "readdirent /var/lib/grafana/dashboards: no such file or directory".
dashboards.yaml provider path updated to match.
- Connector telemetry stays OTLP. The VPN sidecar's netns reaches the collector's
internal IP fine (connected route, off-tunnel), but the sidecar's DNS hijacks
name resolution: AWG_CONF must NOT carry a DNS= directive, else otelcol won't
resolve ("produced zero addresses"). Without DNS= the netns uses Docker's
resolver (resolves both otelcol and api.telegram.org). Documented in
deploy/README.md (AWG_CONF row + wiring note), ARCHITECTURE §13, compose comment.
- PLAN.md: new Stage 17 "Test-contour verification & defect fixes" (exercise the
deployed contour end-to-end and fix what it surfaces — connector liveness check,
path-conditional CI); the former prod-deploy stage becomes Stage 18.
- Renumber every "Stage 17" prod-deploy reference to "Stage 18" across docs,
compose, Caddyfile, ci.yaml and CLAUDE.md; the post-Stage-14 split range is now
"Stages 15–18".
- bot.New now selects Telegram's test environment with the library's native
tgbot.UseTestEnvironment() instead of a token += "/test" hack (functionally
identical URL /bot<token>/test/METHOD, but idiomatic) + a bot test asserting
the getMe path for both test and prod.
- ci.yaml pins TELEGRAM_TEST_ENV=true for the test contour (it IS the test
environment) instead of a TEST_TELEGRAM_TEST_ENV variable: removes the
confusing double-TEST, telegram-specific, prefixed operator knob and the
secret-vs-variable footgun. Prod (Stage 17) leaves it false.
- deploy/README.md + PLAN.md updated.
Ilia Denisov
developer