The account time zone defaults to UTC until a player saves a profile, so a
report's Filed time could only render in UTC even for a player clearly in
another zone. Capture the client's detected "±HH:MM" offset (browser_tz) with
each submission and show the Filed time in three zones in the operator console
— UTC, the browser offset detected at submit, and the sender's saved profile
zone — each shown "N/A" when not known, so the operator can tell what is
certainly known from what is merely defaulted.
- Thread browser_tz through the fbs envelope (+ Go/TS codegen), the gateway
transcode + backend client, and the backend feedback service/store; add the
column via migration 00004 (additive, image-rollback-safe).
- Fix fmtTimeIn to resolve "±HH:MM" offsets via account.ResolveZone;
time.LoadLocation alone silently fell back to UTC for offset zones, which is
the second reason a "+02:00" sender showed only UTC.
- Update ARCHITECTURE/FUNCTIONAL(+ru) docs and the feedback integration test.
Each feedback submission now carries the client app version (__APP_VERSION__),
snapshotted like the interface language: FlatBuffers FeedbackSubmitRequest gains
a version field → gateway transcode → backend, persisted in a new nullable
feedback_messages.app_version column (migration 00003, additive so an image
rollback stays DB-safe). The operator console detail shows the app version and
renders the Filed time in UTC plus the sender's time zone (fmtTimeIn).
Touches: fbs schema + regenerated Go/TS codegen, codec + transport (the client
attaches its build), gateway transcode + backendclient, feedback store/service,
admin view + template, docs (ARCHITECTURE §15, FUNCTIONAL + _ru). Verified:
feedback integration tests (migration + version round-trip), codec round-trip,
check/unit/build green.
The UI language follows the device (the local choice / system guess) and is
deliberately not overridden from the account, but the advertising banner and
out-of-app push routing are resolved server-side from preferred_language. A
saved device choice the account had not recorded — picked while a guest, or
differing from the Telegram system-language seed — left the banner (and pushes)
in the wrong language until a Settings change rewrote preferred_language.
On profile load (adoptSession and the in-place link path) push the saved local
choice to the account when it differs (new pure helper languageNeedsServerSync;
no-op for guests and when already equal), so the banner and pushes match the
visible UI from the first open. Unit-tested.
Migration 00002 replaces the default (house) campaign's single seed tip with
the 47 curated, language-agnostic Scrabble tips — one bilingual ad_messages row
each (body_en + body_ru), which the client round-robins per ARCHITECTURE §ads.
Data-only: the ad_messages schema is unchanged, so a backend image rollback
stays DB-safe. Down restores the original single seed tip. Verified up/down
against a throwaway Postgres (47 rows; apostrophes escaped).
Drop the literal version default from the build files (backend Dockerfile both
stages, loadtest Dockerfile, the compose build-arg) so the release tag is not
duplicated as a stale-prone default a newcomer can't tell from the real source.
DICT_VERSION is now required: compose uses ${DICT_VERSION:?…} and the Dockerfiles
have no ARG default, so a missing value fails loudly instead of baking a stale tag.
The tag lives only in its genuine sources — ci.yaml env (CI tests), the Gitea
TEST_/PROD_DICT_VERSION variables (deploy seed) and deploy/.env.example (local).
Adds a "Bumping the dictionary version" section to deploy/README and fixes the bare
docker-build examples (CLAUDE.md, README.md, loadtest/README) to pass --build-arg.
Bump DICT_VERSION v1.2.1 -> v1.3.0 across the seed surface: .env.example, the
compose build-arg default, both backend Dockerfile stages, the loadtest
Dockerfile, the CI dawg-download version, and the deploy/backend docs.
v1.3.0 drops the abbreviation class from the Russian word list (scrabble-dictionary
#6). This only seeds a FRESH volume; a live contour/prod volume is unaffected (the
.seed_version marker wins — seed-drift guard) and moves to v1.3.0 through the admin
console (ARCHITECTURE §5). Per-contour deploy still overrides via TEST_/PROD_DICT_VERSION.
Top-aligning it still read a touch low; move the empty-blank star up
(top 8% -> 0.5%) so its ink centres against the rack letters' block,
matching the board tile's centred mark. Size unchanged.
Rack: the empty-blank star is now top-anchored level with the letters and a
touch larger (was centred, sitting low). Board and Stats best-move tiles: the
placed-blank star's ink is centred on the value digits' line (was slightly
high). CSS-only nudges; pixel offsets measured against the rendered glyphs.
The Erudit variant's blank is the "звёздочка", so render it with a star.
An empty rack blank (and its drag ghost) shows ✻ centred; a placed blank
keeps its designated letter and carries ✻ where the (absent) point value
sits — on the board and in the Stats best-move tiles. The Scrabble variants
are unchanged. Gated by usesStarBlank() in lib/variants.ts.
When the Telegram first name yields no usable letters, fall back to the @username
taken whole (trimmed + length-capped, never character-stripped like the real name)
rather than a sanitized form; the generated placeholder is reached only when no
username is set. Precedence: real name -> @username (verbatim) -> placeholder.
- Lobby friend-invitation card: icon-only checkmark/cross actions stacked in a
min-width right column; the middle column (From <name> + flag + variant rules,
like New Game) grows and wraps. The cross now opens a decline-confirmation modal
(mirroring the in-game resign confirm) instead of declining on first tap.
- New Game with a friend: a lone offered variant is pre-selected and its picker
disabled (nothing else to choose); relabel 'Тип игры' -> 'Вариант' and
'Подсказок на игрока' -> 'Подсказки'.
- Quick game: pin the Start button to the bottom of the screen, mirroring the
friend-game Send-invitation button.
The project is live in production, so the staged-development scaffolding is removed.
- Delete the staged trackers PLAN.md and PRERELEASE.md.
- Rewrite CLAUDE.md: drop the per-stage workflow; codify the ongoing development
principles (How we work) and the production model (Branching, CI & production):
manual prod-deploy / prod-rollback, semver release tags, Ansible provisioning,
expand-contract migrations.
- De-stage the living docs (README, ARCHITECTURE, TESTING, deploy/ansible, loadtest,
platform/telegram READMEs) and the docker-compose tuning comments: drop the
Stage N / R1-R7 / pre-release labels, keep every number and rationale, and fix the
now-dangling PLAN.md / PRERELEASE.md references to describe the current state.
- Reword stale 'later stage' Go doc comments for subsystems that have shipped.
- prod-deploy.yaml is now four visible sequential jobs (build -> deploy-main ->
deploy-bot -> verify) so the rollout stages show in the Actions UI; the
per-service rolling stays in the deploy-main log.
- prod-rollback.yaml: a separate manual workflow_dispatch. Leave target_version
blank to roll back to the previous deployed version (the host now tracks
DEPLOYED_TAG + PREVIOUS_TAG), or pick a release tag. Re-deploys an already
published image rolling + health-gated, image-only (no rebuild, no DB migration).
- prod-deploy.sh tracks the previous tag (commit_tag) for the blank-input rollback.
- Docs: ARCHITECTURE §13 + deploy/README runbook cover versioning + rollback.
pkg/version.Version (default "dev") is set at link time via -ldflags from each
service Dockerfile's VERSION build-arg, which the deploy passes as the git tag
(git describe --tags). It surfaces as the OpenTelemetry service.version resource
attribute (so Grafana/Tempo are version-aware), alongside the SPA's existing
About version. Adds the VERSION build-arg to the backend/gateway/validator/bot
compose builds and a serviceResource test covering service.name + service.version.
- ARCHITECTURE §13 prod bullet -> the realized mechanism: registry transport,
two-host, rolling + auto-rollback, migration maintenance window, node_exporter,
the undersized launch; the contour paragraph notes node_exporter + the
telegram-local profile.
- deploy/README gains a prod rollout runbook (how to run, migrations/restore, cert
rotation, sizing/monitoring, the full PROD_ set) + node_exporter row, the
telegram-local profile note, and the soft AWG_CONF note.
- PLAN Stage 18 records the resolved open details and the remaining live cutover
(pending erudit-game.ru DNS); the tracker reads 'machinery built; cutover pending DNS'.
- PRERELEASE TX/AG note the prod wiring is built.
A workflow_dispatch-only rollout from master (confirm=deploy):
- .gitea/workflows/prod-deploy.yaml builds + pushes the images to the registry,
ships the compose/config/certs/env over SSH, deploys the main host via
prod-deploy.sh, then the bot host, then verifies the public site.
- deploy/prod-deploy.sh rolls the main stack one service at a time in dependency
order (postgres->backend->gateway->landing->validator->caddy), health-checking
after each; any failure rolls the whole stack back to the previous tag. A schema
migration adds a maintenance window: the backend (sole writer) is stopped for a
consistent pg_dump before migrating; image rollback stays DB-safe (expand-contract),
the dump is kept for a manual restore.
- prod overlay: pull the four main images from the registry by tag.
- Runtime secrets reach the host via a sourced env.sh (single-quoted values keep the
bcrypt hash's literal $ intact, unlike a --env-file).
Split the contour across the two prod hosts and retune for the small main host:
- Gate vpn+bot to the telegram-local profile. The CI test deploy now passes
--profile telegram-local so the test contour still brings them; the prod main
host omits both, and the prod bot runs standalone from docker-compose.bot.yml.
- docker-compose.prod.yml (main-host overlay): publish caddy 80/443 (no host
caddy in prod; caddy owns ACME) and gateway 9443 (the remote bot dials in over
mTLS); GOMAXPROCS=2, smaller memory caps and 7d Prometheus retention for the
2 vCPU / 1.9 GiB host. It launches deliberately undersized; resize reactively.
- docker-compose.bot.yml: standalone bot for the tg host (no VPN, OTLP off since
otelcol is unreachable from there, dials the main host's bot-link).
- Add node_exporter + a Prometheus scrape so host memory pressure (the OOM
signal on the tight main host), not just per-container docker_stats, is visible.
- Soften AWG_CONF to a default: only the profiled vpn sidecar consumes it, and
compose interpolates profiled-out services too, so prod must not require it.
Idempotent playbooks under deploy/ansible/ prepare both production hosts:
docker-ce + compose plugin, a non-sudo deploy service account holding the CI
deploy key, key-only sshd, default-deny ufw, fail2ban, unattended upgrades and
chrony. The main host also opens 80/443/9443 and creates the external edge
network; the tg host verifies direct Bot API egress (the no-VPN assumption).
The application is deployed separately by the prod-deploy workflow (later
phase), running as the deploy account this playbook provisions.
Inside Telegram, a failed initData authentication (e.g. the backend down
during a deploy) dropped the user onto the web login screen — the /app/
experience, which has no place inside the Mini App. bootstrap now retries the
launch a few times in silence and then renders a dedicated boot-error screen
with a Retry button (new BootError.svelte, app.bootError), never falling back
to the web sign-in. A blocked account is still terminal and goes straight to
the blocked screen.
The profile "Link an account" section (email + Telegram link) is hidden while
sign-in is provider-only; the anonymous /app/ guest whose upgrade path this is
comes later. The flow is kept wired (`hidden` on .emailbox) and its two e2e
specs are skipped, both to be re-enabled together.
Adds i18n boot.* copy (en/ru), a mock authTelegram failure hook plus an e2e
covering the retry screen, and bakes both behaviours into FUNCTIONAL(.md/_ru).
EvaluatePlay (the hottest gameplay call, fired on every tile placement) now uses
the warm live-game cache directly: an active game stays cached (mutated in place
across moves, evicted only on finish), so the cached engine game and its immutable
seat list answer the membership check and the score with no DB read. The cold path
(eviction / first load) still loads and validates via the store. The seat list is
cached alongside the engine game for the membership fast path.
GetGame also folds its two round-trips (game, then seats) into one LEFT JOIN,
preserving the contract (same Game, a seatless game still returns empty seats, seat
order kept) — one round-trip for every remaining caller.
Measured at 500 players: evaluate p99 halves (200 -> 100 ms) and the per-op query
count drops. It does NOT cut postgres CPU — that is write-bound (per-move CommitMove
plus draft upserts and journal replays), the cheap indexed GetGame reads were never
its bottleneck, and postgres runs with headroom (~1.5 of 2 cores). So this is a
latency / query-volume optimization, not a DB-CPU one.
Regression cover: a non-player evaluate against a warm game asserts the cached-seat
membership path; the integration suite exercises GetGame's join across every game op.
The loadtest harness never modelled game.evaluate — the debounced per-tile
play preview a real client fires several times per turn, the hottest gameplay
call. Model it (one evaluate per placed tile + reconsideration re-previews +
draft.save, human-paced; --eval / --eval-recon toggle it).
That realistic load surfaced the real bottleneck: the gateway's backend HTTP
client used the default transport (MaxIdleConnsPerHost=2), so every sync call
to the single backend host churned a fresh TCP connection — ~26500 TIME_WAIT
sockets at 500 players (near the ephemeral-port ceiling), burning ~1.75 gateway
cores while the backend sat near-idle. It was the unfixed root of the residual
transport_error the earlier passes chased on the client side.
Widen the keep-alive pool (backendMaxIdleConns=512, ~2x the observed 225-conn
peak). At 500 players the churn collapses to ~0 and peak gateway CPU drops ~7x
(~1.75 -> ~0.26 cores); postgres (~1.65 cores) becomes the busiest service.
This overturns the earlier "gateway is the binding constraint, scale it
horizontally" sizing — that was sizing around this bug, not a real floor.
Consolidate the loadtest trip reports into one loadtest/REPORT.md (drop the
R2/R7 split) and bake the finding into README / PRERELEASE / ARCHITECTURE /
TESTING.
The main bot is now an admin in the moderated discussion group and receives its
messages (allowed_updates includes message). Its default handler replied to
every message with a Mini App launch button — an inline web_app button, which
Telegram permits only in private chats — so replying in the group failed with
BUTTON_TYPE_INVALID (silently: the send fails, no user-facing error). Reply only
in a private chat; in the group the bot only manages permissions. The promo bot
gets the same guard.
- Bake the final default-allow + mute-the-ineligible strategy into
docs/ARCHITECTURE.md, docs/FUNCTIONAL.md (+_ru), platform/telegram/README.md,
the deploy compose comment and the PRERELEASE tracker. The live test proved a
per-user grant cannot exceed a deny-by-default group (Telegram intersects the
chat default with the per-user permission), so the chat allows sending by
default and the bot restricts the ineligible instead of granting the eligible.
- Lower the per-event chat_member trace and eligibility evaluation to Debug;
keep the actual mute/unmute actions, the startup self-check and warnings at
Info, so prod logs only what the bot did.
- Update game.searchingForOpponent (Searching -> Waiting for opponent / Поиск ->
Ждём соперника) and the quickmatch e2e assertions to match.
Telegram intersects the chat default with each user's permissions, so a per-user
grant can never exceed a deny-by-default group: the original default-deny +
grant design could not let any user write (can_send=true was AND-ed with the
denying default). Invert it — the chat allows sending by default and the bot
MUTES an ineligible member (unregistered, admin-suspended, or chat_muted) and
restores an eligible one it had muted, acting only when the current state
differs (idempotent, no self-loop). The block/unblock/chat_muted/registration
path already sets can_send to the eligibility, so it is unchanged.
The CanSendMessages loop-guard skipped exactly the stuck case — a restricted
member whose chat_member event reports can_send=true yet who cannot actually
write. Replace it with a precise loop guard (skip only the bot's own restrict
action, i.e. the update whose performer is the bot) and grant any eligible
in-chat member (member or restricted) otherwise. Also log the new member's
can_send, is_member and the actor id for full visibility.
A default-deny discussion group reports a present or freshly joined member as
`restricted` (no send right), not `member`. The join filter required `member`,
so the real case never matched and a registered user stayed muted. Grant any
eligible in-chat member (member or restricted) that still lacks the send right,
with a loop guard (skip when send is already allowed) so the bot's own grant
does not re-fire. Revoking a now-ineligible user stays the chat-gate path's job,
so this never fights a chat_muted/block.
Two follow-ups from a contour test where a user joined the chat, then
registered, and got no write access — with silent logs.
Observability: log every chat_member update (chat id, configured id, user,
old->new status), the eligibility result and the grant outcome; plus a startup
self-check that warns loudly when the bot is not an administrator in the chat
with the restrict-members ("Ban users") right — the common misconfiguration,
previously invisible in the logs.
Grant on first registration: a user who joins the moderated chat BEFORE
registering is covered by no chat_member event, so the join-time grant never
fires for them. ProvisionTelegram now reports first contact, and the Telegram
auth handler emits chat_access_changed on it, so the gateway re-evaluates and
grants write access if the user is already in the chat.
tgbot.New validates the token with getMe, so a bad or unreachable promo token
would otherwise return an error from run() and crash-loop the whole bot process
— taking the main game bot down with it, since they share the container. Log it
and skip the promo bot instead; the main bot and bot-link are unaffected.
Add a second standalone promo bot to the bot container (answers /start with a
localized message + a URL button into the main bot's Mini App) and gate write
access in a channel's linked discussion chat: grant on join when the Telegram
user is registered and neither admin-suspended nor holding a new chat_muted
role, and revoke/grant on the matching moderation change for a member currently
in the chat.
Eligibility (registered AND NOT suspended AND NOT chat_muted; the game
suspension dominates) is resolved once in the backend and reached two ways: the
bot's join-time unary ResolveChatEligibility over the existing mTLS bot-link,
and a backend chat_access_changed event -> gateway -> ChatGate command
(idempotent; a temporary-block-expiry sweeper may over-emit). The bot guards the
block/unblock path with getChatMember, since bots cannot list members.
A web_app button cannot open another bot's Mini App (it signs initData with the
sending bot's token), so the promo button is a t.me ?startapp URL reusing the
UI's VITE_TELEGRAM_LINK. The bot must be a chat admin with the restrict-members
right and chat_member in its allowed updates.
No schema change: chat_muted reuses the data-driven account_roles table.
The dismiss check fired at the instant a word finished laying, so a word
was never held: ЭРУДИТ fell straight into the lobby (too fast) and
ЗАГРУЗКА got no readable pause. The pause was a *leading* gap before the
next word, not a hold after the current one.
Move the hold to after each word and run the check after it: every word
(ЭРУДИТ included) now stays up for PAUSE_MS before the splash either
dismisses or lays the next word. prefixMs = WORD_MS + PAUSE_MS,
cycleMs = 2*(WORD_MS + PAUSE_MS).
On a cold app open the lobby's game list arrives over the network; on a
slow link the empty "no games yet" line flashed before the games loaded.
Add a full-screen tile splash that lays a Scrabble crossword of ЭРУДИТ /
ЗАГРУЗКА / ОЖИДАНИЕ (Эрудит point values, hardcoded since the alphabet
table is not cached at boot) until the lobby's first load settles, then
removes itself to reveal the populated list.
- lib/splash.ts: pure layout + reveal schedule (unit-tested).
- components/Splash.svelte: App-level overlay; per-tile drop-in; loops
ЗАГРУЗКА → ОЖИДАНИЕ until ready, dismisses on a word boundary. Static
ЭРУДИТ under reduced motion / the mock build.
- app state: lobbyReady (set by Lobby on first settle) + splashDone,
reset on logout.
- App.svelte: overlay while routeIsLobby && !splashDone; the plain text
splash now only covers non-lobby deep-links during bootstrap.
- docs: UI_DESIGN + FUNCTIONAL (+ _ru).
The @honeypot block both deleted and set X-Scrabble-Honeypot in one reverse_proxy.
Caddy applies header_up deletions *after* sets, so the tag we set was immediately
stripped: the gateway never saw it, and a decoy hit (e.g. GET /.env) fell through
to the gateway's /app redirect (308) instead of tripping the honeypot. Drop the
delete — the bare set already replaces any client-supplied value. The real
endpoints keep stripping the header in the @gateway block (delete-only, no
conflicting set). Caught on the live test contour (no caddy locally).
Add a prod-only, in-memory IP ban enforced at the edge, fed by three signals:
sustained rate-limiter rejections (the IP-keyed public/email/admin classes — the
user class stays the backend soft-flag's concern), a honeypot decoy-path hit (the
contour caddy tags decoys with X-Scrabble-Honeypot and routes them to the gateway),
and a honeytoken (a planted bearer, GATEWAY_HONEYTOKEN). A banned IP is refused with
429 by the abuseGuard middleware before any work — covering the Connect edge, the
live stream and the static SPA/landing the per-op limiter never gated.
The ban is off by default: it keys by the real client IP the shared-NAT test contour
does not expose, so a ban there would be self-inflicted; detection still logs in the
contour, only the ban action is gated (GATEWAY_ABUSE_BAN_ENABLED). Rejection bans last
GATEWAY_ABUSE_BAN_DURATION; tripwire/honeytoken hits are near-zero-false-positive and
earn longer fixed bans. Each ban increments gateway_abuse_banned_total{reason}.
Operators see and lift active bans on the admin console's Throttled page; the gateway
syncs its active set to the backend every 30s (POST /api/v1/internal/bans/sync,
backend/internal/banview) and applies the operator unbans the response returns.
PRERELEASE phase AG. Docs baked into ARCHITECTURE / FUNCTIONAL (+ru) / both READMEs.
developer
Ilia Denisov