Compare commits

...

48 Commits

Author SHA1 Message Date
developer deaa7a29c5 Merge pull request 'Promote development → master (docs finalize + UI tweaks + Telegram name fallback)' (#108) from development into master 2026-06-22 07:27:40 +00:00
developer 91de26d80b Merge pull request 'Finalize docs to production + lobby/new-game UI tweaks + Telegram name fallback' (#107) from feature/finalize-docs into development
CI / changes (push) Successful in 2s
CI / unit (push) Successful in 10s
CI / integration (push) Successful in 16s
CI / ui (push) Successful in 56s
CI / gate (push) Successful in 0s
CI / deploy (push) Successful in 1m22s
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 11s
CI / integration (pull_request) Successful in 16s
CI / ui (pull_request) Successful in 57s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Has been skipped
2026-06-22 07:15:38 +00:00
Ilia Denisov 6b6362a629 fix(account): Telegram display-name falls back to the @username verbatim
CI / changes (pull_request) Successful in 3s
CI / unit (pull_request) Successful in 12s
CI / integration (pull_request) Successful in 16s
CI / ui (pull_request) Successful in 57s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m22s
When the Telegram first name yields no usable letters, fall back to the @username
taken whole (trimmed + length-capped, never character-stripped like the real name)
rather than a sanitized form; the generated placeholder is reached only when no
username is set. Precedence: real name -> @username (verbatim) -> placeholder.
2026-06-22 09:11:36 +02:00
Ilia Denisov 8a06fbc3c7 feat(ui): lobby invitation card redesign + new-game tweaks
- Lobby friend-invitation card: icon-only checkmark/cross actions stacked in a
  min-width right column; the middle column (From <name> + flag + variant rules,
  like New Game) grows and wraps. The cross now opens a decline-confirmation modal
  (mirroring the in-game resign confirm) instead of declining on first tap.
- New Game with a friend: a lone offered variant is pre-selected and its picker
  disabled (nothing else to choose); relabel 'Тип игры' -> 'Вариант' and
  'Подсказок на игрока' -> 'Подсказки'.
- Quick game: pin the Start button to the bottom of the screen, mirroring the
  friend-game Send-invitation button.
2026-06-22 09:11:36 +02:00
Ilia Denisov 48b06f4594 docs: finalize documentation to the production state
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 16s
CI / ui (pull_request) Successful in 57s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m21s
The project is live in production, so the staged-development scaffolding is removed.

- Delete the staged trackers PLAN.md and PRERELEASE.md.
- Rewrite CLAUDE.md: drop the per-stage workflow; codify the ongoing development
  principles (How we work) and the production model (Branching, CI & production):
  manual prod-deploy / prod-rollback, semver release tags, Ansible provisioning,
  expand-contract migrations.
- De-stage the living docs (README, ARCHITECTURE, TESTING, deploy/ansible, loadtest,
  platform/telegram READMEs) and the docker-compose tuning comments: drop the
  Stage N / R1-R7 / pre-release labels, keep every number and rationale, and fix the
  now-dangling PLAN.md / PRERELEASE.md references to describe the current state.
- Reword stale 'later stage' Go doc comments for subsystems that have shipped.
2026-06-22 08:33:30 +02:00
developer 24017bcb7f Merge pull request 'Promote development → master (deploy v2: versioning + visible jobs + rollback)' (#106) from development into master 2026-06-22 06:01:03 +00:00
developer 40d8f06588 Merge pull request 'Deploy v2 — release versioning + visible deploy jobs + manual rollback' (#105) from feature/release-versioning into development
CI / changes (push) Successful in 2s
CI / unit (push) Successful in 11s
CI / integration (push) Successful in 15s
CI / ui (push) Successful in 57s
CI / changes (pull_request) Successful in 2s
CI / gate (push) Successful in 0s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 16s
CI / ui (pull_request) Successful in 56s
CI / deploy (push) Successful in 1m19s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Has been skipped
2026-06-22 05:40:55 +00:00
Ilia Denisov c59e522732 feat(deploy): visible prod-deploy jobs + manual prod-rollback
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 11s
CI / integration (pull_request) Successful in 16s
CI / ui (pull_request) Successful in 57s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m23s
- prod-deploy.yaml is now four visible sequential jobs (build -> deploy-main ->
  deploy-bot -> verify) so the rollout stages show in the Actions UI; the
  per-service rolling stays in the deploy-main log.
- prod-rollback.yaml: a separate manual workflow_dispatch. Leave target_version
  blank to roll back to the previous deployed version (the host now tracks
  DEPLOYED_TAG + PREVIOUS_TAG), or pick a release tag. Re-deploys an already
  published image rolling + health-gated, image-only (no rebuild, no DB migration).
- prod-deploy.sh tracks the previous tag (commit_tag) for the blank-input rollback.
- Docs: ARCHITECTURE §13 + deploy/README runbook cover versioning + rollback.
2026-06-22 07:37:08 +02:00
Ilia Denisov 8d45ae6e3b feat: stamp the build version into every service
pkg/version.Version (default "dev") is set at link time via -ldflags from each
service Dockerfile's VERSION build-arg, which the deploy passes as the git tag
(git describe --tags). It surfaces as the OpenTelemetry service.version resource
attribute (so Grafana/Tempo are version-aware), alongside the SPA's existing
About version. Adds the VERSION build-arg to the backend/gateway/validator/bot
compose builds and a serviceResource test covering service.name + service.version.
2026-06-22 07:28:27 +02:00
developer 2c4f4b10dc Merge pull request 'Promote development → master (initial production release: pre-release line + Stage 18)' (#104) from development into master 2026-06-22 05:05:48 +00:00
developer 520a9092fe Merge pull request 'Stage 18 — prod contour deploy (two-host registry rollout, rolling + auto-rollback)' (#103) from feature/prod-contour-deploy into development
CI / changes (push) Successful in 2s
CI / unit (push) Successful in 11s
CI / integration (push) Successful in 17s
CI / ui (push) Successful in 57s
CI / changes (pull_request) Successful in 2s
CI / gate (push) Successful in 0s
CI / deploy (push) Successful in 1m4s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 15s
CI / ui (pull_request) Successful in 56s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Has been skipped
2026-06-22 04:59:46 +00:00
Ilia Denisov 9f970495ee fix(deploy): guard cd and split DOCKER_GID assignment (shellcheck)
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 16s
CI / ui (pull_request) Successful in 56s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m7s
cd $COMPOSE_DIR now aborts on failure instead of deploying from the wrong dir;
DOCKER_GID is declared then exported so the subshell exit isn't masked.
2026-06-22 00:35:20 +02:00
Ilia Denisov 3d9ba3ac3d docs(deploy): bake Stage 18 prod-deploy decisions into the live docs
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 16s
CI / ui (pull_request) Successful in 57s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m8s
- ARCHITECTURE §13 prod bullet -> the realized mechanism: registry transport,
  two-host, rolling + auto-rollback, migration maintenance window, node_exporter,
  the undersized launch; the contour paragraph notes node_exporter + the
  telegram-local profile.
- deploy/README gains a prod rollout runbook (how to run, migrations/restore, cert
  rotation, sizing/monitoring, the full PROD_ set) + node_exporter row, the
  telegram-local profile note, and the soft AWG_CONF note.
- PLAN Stage 18 records the resolved open details and the remaining live cutover
  (pending erudit-game.ru DNS); the tracker reads 'machinery built; cutover pending DNS'.
- PRERELEASE TX/AG note the prod wiring is built.
2026-06-22 00:30:30 +02:00
Ilia Denisov 171b71b7e0 feat(deploy): manual prod-deploy pipeline with rolling rollback (Stage 18)
A workflow_dispatch-only rollout from master (confirm=deploy):

- .gitea/workflows/prod-deploy.yaml builds + pushes the images to the registry,
  ships the compose/config/certs/env over SSH, deploys the main host via
  prod-deploy.sh, then the bot host, then verifies the public site.
- deploy/prod-deploy.sh rolls the main stack one service at a time in dependency
  order (postgres->backend->gateway->landing->validator->caddy), health-checking
  after each; any failure rolls the whole stack back to the previous tag. A schema
  migration adds a maintenance window: the backend (sole writer) is stopped for a
  consistent pg_dump before migrating; image rollback stays DB-safe (expand-contract),
  the dump is kept for a manual restore.
- prod overlay: pull the four main images from the registry by tag.
- Runtime secrets reach the host via a sourced env.sh (single-quoted values keep the
  bcrypt hash's literal $ intact, unlike a --env-file).
2026-06-22 00:25:09 +02:00
Ilia Denisov 2b399d0838 feat(deploy): prod compose split + host-memory monitoring (Stage 18)
Split the contour across the two prod hosts and retune for the small main host:

- Gate vpn+bot to the telegram-local profile. The CI test deploy now passes
  --profile telegram-local so the test contour still brings them; the prod main
  host omits both, and the prod bot runs standalone from docker-compose.bot.yml.
- docker-compose.prod.yml (main-host overlay): publish caddy 80/443 (no host
  caddy in prod; caddy owns ACME) and gateway 9443 (the remote bot dials in over
  mTLS); GOMAXPROCS=2, smaller memory caps and 7d Prometheus retention for the
  2 vCPU / 1.9 GiB host. It launches deliberately undersized; resize reactively.
- docker-compose.bot.yml: standalone bot for the tg host (no VPN, OTLP off since
  otelcol is unreachable from there, dials the main host's bot-link).
- Add node_exporter + a Prometheus scrape so host memory pressure (the OOM
  signal on the tight main host), not just per-container docker_stats, is visible.
- Soften AWG_CONF to a default: only the profiled vpn sidecar consumes it, and
  compose interpolates profiled-out services too, so prod must not require it.
2026-06-22 00:12:43 +02:00
Ilia Denisov f5f45e7afb feat(deploy): Ansible provisioning for prod hosts (Stage 18)
Idempotent playbooks under deploy/ansible/ prepare both production hosts:
docker-ce + compose plugin, a non-sudo deploy service account holding the CI
deploy key, key-only sshd, default-deny ufw, fail2ban, unattended upgrades and
chrony. The main host also opens 80/443/9443 and creates the external edge
network; the tg host verifies direct Bot API egress (the no-VPN assumption).

The application is deployed separately by the prod-deploy workflow (later
phase), running as the deploy account this playbook provisions.
2026-06-21 23:54:57 +02:00
developer b54cb8878d Merge pull request 'fix(ui): retry Mini App launch on backend failure; hide account linking' (#102) from feature/tg-boot-retry-hide-linking into development
CI / changes (push) Successful in 2s
CI / unit (push) Has been skipped
CI / integration (push) Has been skipped
CI / ui (push) Successful in 57s
CI / gate (push) Successful in 0s
CI / deploy (push) Successful in 1m6s
2026-06-21 19:38:37 +00:00
Ilia Denisov e336638ca8 fix(ui): retry Mini App launch on backend failure; hide account linking
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Has been skipped
CI / integration (pull_request) Has been skipped
CI / ui (pull_request) Successful in 57s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m8s
Inside Telegram, a failed initData authentication (e.g. the backend down
during a deploy) dropped the user onto the web login screen — the /app/
experience, which has no place inside the Mini App. bootstrap now retries the
launch a few times in silence and then renders a dedicated boot-error screen
with a Retry button (new BootError.svelte, app.bootError), never falling back
to the web sign-in. A blocked account is still terminal and goes straight to
the blocked screen.

The profile "Link an account" section (email + Telegram link) is hidden while
sign-in is provider-only; the anonymous /app/ guest whose upgrade path this is
comes later. The flow is kept wired (`hidden` on .emailbox) and its two e2e
specs are skipped, both to be re-enabled together.

Adds i18n boot.* copy (en/ru), a mock authTelegram failure hook plus an e2e
covering the retry screen, and bakes both behaviours into FUNCTIONAL(.md/_ru).
2026-06-21 21:23:27 +02:00
developer 62f42ed102 Merge pull request 'perf(gateway): pool backend conns; loadtest evaluate hot path' (#101) from feature/loadtest-evaluate-hotpath into development
CI / changes (push) Successful in 2s
CI / unit (push) Successful in 10s
CI / integration (push) Successful in 15s
CI / ui (push) Has been skipped
CI / gate (push) Successful in 1s
CI / deploy (push) Successful in 1m26s
2026-06-21 18:51:58 +00:00
Ilia Denisov ecb21bd218 perf(backend): cut evaluate's DB round-trips; load the game in one query
CI / changes (pull_request) Successful in 3s
CI / unit (pull_request) Successful in 11s
CI / integration (pull_request) Successful in 17s
CI / ui (pull_request) Has been skipped
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 2m18s
EvaluatePlay (the hottest gameplay call, fired on every tile placement) now uses
the warm live-game cache directly: an active game stays cached (mutated in place
across moves, evicted only on finish), so the cached engine game and its immutable
seat list answer the membership check and the score with no DB read. The cold path
(eviction / first load) still loads and validates via the store. The seat list is
cached alongside the engine game for the membership fast path.

GetGame also folds its two round-trips (game, then seats) into one LEFT JOIN,
preserving the contract (same Game, a seatless game still returns empty seats, seat
order kept) — one round-trip for every remaining caller.

Measured at 500 players: evaluate p99 halves (200 -> 100 ms) and the per-op query
count drops. It does NOT cut postgres CPU — that is write-bound (per-move CommitMove
plus draft upserts and journal replays), the cheap indexed GetGame reads were never
its bottleneck, and postgres runs with headroom (~1.5 of 2 cores). So this is a
latency / query-volume optimization, not a DB-CPU one.

Regression cover: a non-player evaluate against a warm game asserts the cached-seat
membership path; the integration suite exercises GetGame's join across every game op.
2026-06-21 20:47:13 +02:00
Ilia Denisov e2771826fd perf(gateway): pool backend conns; loadtest evaluate hot path
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 18s
CI / ui (pull_request) Has been skipped
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m5s
The loadtest harness never modelled game.evaluate — the debounced per-tile
play preview a real client fires several times per turn, the hottest gameplay
call. Model it (one evaluate per placed tile + reconsideration re-previews +
draft.save, human-paced; --eval / --eval-recon toggle it).

That realistic load surfaced the real bottleneck: the gateway's backend HTTP
client used the default transport (MaxIdleConnsPerHost=2), so every sync call
to the single backend host churned a fresh TCP connection — ~26500 TIME_WAIT
sockets at 500 players (near the ephemeral-port ceiling), burning ~1.75 gateway
cores while the backend sat near-idle. It was the unfixed root of the residual
transport_error the earlier passes chased on the client side.

Widen the keep-alive pool (backendMaxIdleConns=512, ~2x the observed 225-conn
peak). At 500 players the churn collapses to ~0 and peak gateway CPU drops ~7x
(~1.75 -> ~0.26 cores); postgres (~1.65 cores) becomes the busiest service.
This overturns the earlier "gateway is the binding constraint, scale it
horizontally" sizing — that was sizing around this bug, not a real floor.

Consolidate the loadtest trip reports into one loadtest/REPORT.md (drop the
R2/R7 split) and bake the finding into README / PRERELEASE / ARCHITECTURE /
TESTING.
2026-06-21 19:55:57 +02:00
developer dec6fac013 Merge pull request 'fix(telegram): reply to /start only in private chats' (#100) from feature/telegram-private-reply-guard into development
CI / changes (push) Successful in 2s
CI / unit (push) Successful in 11s
CI / integration (push) Successful in 16s
CI / ui (push) Has been skipped
CI / gate (push) Successful in 0s
CI / deploy (push) Successful in 1m8s
2026-06-21 15:32:11 +00:00
Ilia Denisov c494da553a fix(telegram): reply to /start only in private chats
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 18s
CI / ui (pull_request) Has been skipped
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m7s
The main bot is now an admin in the moderated discussion group and receives its
messages (allowed_updates includes message). Its default handler replied to
every message with a Mini App launch button — an inline web_app button, which
Telegram permits only in private chats — so replying in the group failed with
BUTTON_TYPE_INVALID (silently: the send fails, no user-facing error). Reply only
in a private chat; in the group the bot only manages permissions. The promo bot
gets the same guard.
2026-06-21 17:28:01 +02:00
developer fa8abf22db Merge pull request 'feat(telegram): promo bot + channel-chat moderation gate' (#99) from feature/telegram-promo-bot-chat-moderation into development
CI / changes (push) Successful in 2s
CI / unit (push) Successful in 10s
CI / integration (push) Successful in 16s
CI / ui (push) Successful in 56s
CI / gate (push) Successful in 0s
CI / deploy (push) Successful in 1m6s
2026-06-21 15:21:23 +00:00
Ilia Denisov 1ba789a1f1 docs(telegram): invert chat-gate strategy in docs; tune logs; i18n text
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 11s
CI / integration (pull_request) Successful in 18s
CI / ui (pull_request) Successful in 57s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m15s
- Bake the final default-allow + mute-the-ineligible strategy into
  docs/ARCHITECTURE.md, docs/FUNCTIONAL.md (+_ru), platform/telegram/README.md,
  the deploy compose comment and the PRERELEASE tracker. The live test proved a
  per-user grant cannot exceed a deny-by-default group (Telegram intersects the
  chat default with the per-user permission), so the chat allows sending by
  default and the bot restricts the ineligible instead of granting the eligible.
- Lower the per-event chat_member trace and eligibility evaluation to Debug;
  keep the actual mute/unmute actions, the startup self-check and warnings at
  Info, so prod logs only what the bot did.
- Update game.searchingForOpponent (Searching -> Waiting for opponent / Поиск ->
  Ждём соперника) and the quickmatch e2e assertions to match.
2026-06-21 17:15:10 +02:00
Ilia Denisov bdd1cc7d85 fix(telegram): invert the chat gate — mute the ineligible (default-allow)
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 19s
CI / ui (pull_request) Successful in 57s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m17s
Telegram intersects the chat default with each user's permissions, so a per-user
grant can never exceed a deny-by-default group: the original default-deny +
grant design could not let any user write (can_send=true was AND-ed with the
denying default). Invert it — the chat allows sending by default and the bot
MUTES an ineligible member (unregistered, admin-suspended, or chat_muted) and
restores an eligible one it had muted, acting only when the current state
differs (idempotent, no self-loop). The block/unblock/chat_muted/registration
path already sets can_send to the eligibility, so it is unchanged.
2026-06-21 16:50:44 +02:00
Ilia Denisov 0ab1719ee9 fix(telegram): grant in-chat members regardless of reported can_send
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 11s
CI / integration (pull_request) Successful in 16s
CI / ui (pull_request) Successful in 56s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m18s
The CanSendMessages loop-guard skipped exactly the stuck case — a restricted
member whose chat_member event reports can_send=true yet who cannot actually
write. Replace it with a precise loop guard (skip only the bot's own restrict
action, i.e. the update whose performer is the bot) and grant any eligible
in-chat member (member or restricted) otherwise. Also log the new member's
can_send, is_member and the actor id for full visibility.
2026-06-21 16:25:57 +02:00
Ilia Denisov 380f82438c fix(telegram): grant write to restricted members in default-deny chats
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 11s
CI / integration (pull_request) Successful in 15s
CI / ui (pull_request) Successful in 57s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m22s
A default-deny discussion group reports a present or freshly joined member as
`restricted` (no send right), not `member`. The join filter required `member`,
so the real case never matched and a registered user stayed muted. Grant any
eligible in-chat member (member or restricted) that still lacks the send right,
with a loop guard (skip when send is already allowed) so the bot's own grant
does not re-fire. Revoking a now-ineligible user stays the chat-gate path's job,
so this never fights a chat_muted/block.
2026-06-21 16:12:58 +02:00
Ilia Denisov a404513037 feat(telegram): chat-gate observability + grant on first registration
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 11s
CI / integration (pull_request) Successful in 18s
CI / ui (pull_request) Successful in 56s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m0s
Two follow-ups from a contour test where a user joined the chat, then
registered, and got no write access — with silent logs.

Observability: log every chat_member update (chat id, configured id, user,
old->new status), the eligibility result and the grant outcome; plus a startup
self-check that warns loudly when the bot is not an administrator in the chat
with the restrict-members ("Ban users") right — the common misconfiguration,
previously invisible in the logs.

Grant on first registration: a user who joins the moderated chat BEFORE
registering is covered by no chat_member event, so the join-time grant never
fires for them. ProvisionTelegram now reports first contact, and the Telegram
auth handler emits chat_access_changed on it, so the gateway re-evaluates and
grants write access if the user is already in the chat.
2026-06-21 15:19:21 +02:00
Ilia Denisov b22b624d28 fix(telegram): keep a failed promo-bot construction non-fatal
CI / changes (pull_request) Successful in 3s
CI / unit (pull_request) Successful in 11s
CI / integration (pull_request) Successful in 17s
CI / ui (pull_request) Successful in 57s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m17s
tgbot.New validates the token with getMe, so a bad or unreachable promo token
would otherwise return an error from run() and crash-loop the whole bot process
— taking the main game bot down with it, since they share the container. Log it
and skip the promo bot instead; the main bot and bot-link are unaffected.
2026-06-21 14:50:01 +02:00
Ilia Denisov e71e40eef5 feat(telegram): promo bot + channel-chat moderation gate
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 11s
CI / integration (pull_request) Successful in 19s
CI / ui (pull_request) Successful in 57s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m39s
Add a second standalone promo bot to the bot container (answers /start with a
localized message + a URL button into the main bot's Mini App) and gate write
access in a channel's linked discussion chat: grant on join when the Telegram
user is registered and neither admin-suspended nor holding a new chat_muted
role, and revoke/grant on the matching moderation change for a member currently
in the chat.

Eligibility (registered AND NOT suspended AND NOT chat_muted; the game
suspension dominates) is resolved once in the backend and reached two ways: the
bot's join-time unary ResolveChatEligibility over the existing mTLS bot-link,
and a backend chat_access_changed event -> gateway -> ChatGate command
(idempotent; a temporary-block-expiry sweeper may over-emit). The bot guards the
block/unblock path with getChatMember, since bots cannot list members.

A web_app button cannot open another bot's Mini App (it signs initData with the
sending bot's token), so the promo button is a t.me ?startapp URL reusing the
UI's VITE_TELEGRAM_LINK. The bot must be a chat admin with the restrict-members
right and chat_member in its allowed updates.

No schema change: chat_muted reuses the data-driven account_roles table.
2026-06-21 14:46:51 +02:00
developer 41d21f3f6f Merge pull request 'feat(ui): tile-crossword loading splash for cold lobby open' (#98) from feature/lobby-splash-tiles into development
CI / changes (push) Successful in 2s
CI / unit (push) Has been skipped
CI / integration (push) Has been skipped
CI / ui (push) Successful in 57s
CI / gate (push) Successful in 1s
CI / deploy (push) Successful in 1m15s
2026-06-21 10:19:15 +00:00
Ilia Denisov 9642cafc1f fix(ui): hold each splash word for the pause before the readiness check
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Has been skipped
CI / integration (pull_request) Has been skipped
CI / ui (pull_request) Successful in 57s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m18s
The dismiss check fired at the instant a word finished laying, so a word
was never held: ЭРУДИТ fell straight into the lobby (too fast) and
ЗАГРУЗКА got no readable pause. The pause was a *leading* gap before the
next word, not a hold after the current one.

Move the hold to after each word and run the check after it: every word
(ЭРУДИТ included) now stays up for PAUSE_MS before the splash either
dismisses or lays the next word. prefixMs = WORD_MS + PAUSE_MS,
cycleMs = 2*(WORD_MS + PAUSE_MS).
2026-06-21 10:42:31 +02:00
Ilia Denisov ba6ee90278 feat(ui): tile-crossword loading splash for cold lobby open
CI / changes (pull_request) Successful in 3s
CI / unit (pull_request) Has been skipped
CI / integration (pull_request) Has been skipped
CI / ui (pull_request) Successful in 57s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m17s
On a cold app open the lobby's game list arrives over the network; on a
slow link the empty "no games yet" line flashed before the games loaded.
Add a full-screen tile splash that lays a Scrabble crossword of ЭРУДИТ /
ЗАГРУЗКА / ОЖИДАНИЕ (Эрудит point values, hardcoded since the alphabet
table is not cached at boot) until the lobby's first load settles, then
removes itself to reveal the populated list.

- lib/splash.ts: pure layout + reveal schedule (unit-tested).
- components/Splash.svelte: App-level overlay; per-tile drop-in; loops
  ЗАГРУЗКА → ОЖИДАНИЕ until ready, dismisses on a word boundary. Static
  ЭРУДИТ under reduced motion / the mock build.
- app state: lobbyReady (set by Lobby on first settle) + splashDone,
  reset on logout.
- App.svelte: overlay while routeIsLobby && !splashDone; the plain text
  splash now only covers non-lobby deep-links during bootstrap.
- docs: UI_DESIGN + FUNCTIONAL (+ _ru).
2026-06-21 10:29:39 +02:00
developer e79c1ea891 Merge pull request 'feat(gateway): temporary IP ban (fail2ban) + honeypot/honeytoken (PRERELEASE AG)' (#97) from feature/abuse-ip-ban-honeypot into development
CI / changes (push) Successful in 1s
CI / unit (push) Successful in 11s
CI / integration (push) Successful in 15s
CI / ui (push) Successful in 56s
CI / gate (push) Successful in 0s
CI / deploy (push) Successful in 1m15s
2026-06-21 07:28:30 +00:00
Ilia Denisov cf9fa75d62 fix(deploy): honeypot tag dropped — Caddy applies header_up delete after set
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 14s
CI / ui (pull_request) Successful in 53s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m3s
The @honeypot block both deleted and set X-Scrabble-Honeypot in one reverse_proxy.
Caddy applies header_up deletions *after* sets, so the tag we set was immediately
stripped: the gateway never saw it, and a decoy hit (e.g. GET /.env) fell through
to the gateway's /app redirect (308) instead of tripping the honeypot. Drop the
delete — the bare set already replaces any client-supplied value. The real
endpoints keep stripping the header in the @gateway block (delete-only, no
conflicting set). Caught on the live test contour (no caddy locally).
2026-06-21 09:02:23 +02:00
Ilia Denisov 81b44c2b02 docs(prerelease): mark phase AG done (code + test contour)
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 14s
CI / ui (pull_request) Successful in 53s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m6s
2026-06-21 08:58:10 +02:00
Ilia Denisov 041106d623 feat(gateway): temporary IP ban (fail2ban) fed by rejections + honeypot/honeytoken
CI / changes (pull_request) Successful in 3s
CI / unit (pull_request) Successful in 11s
CI / integration (pull_request) Successful in 17s
CI / ui (pull_request) Successful in 53s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m11s
Add a prod-only, in-memory IP ban enforced at the edge, fed by three signals:
sustained rate-limiter rejections (the IP-keyed public/email/admin classes — the
user class stays the backend soft-flag's concern), a honeypot decoy-path hit (the
contour caddy tags decoys with X-Scrabble-Honeypot and routes them to the gateway),
and a honeytoken (a planted bearer, GATEWAY_HONEYTOKEN). A banned IP is refused with
429 by the abuseGuard middleware before any work — covering the Connect edge, the
live stream and the static SPA/landing the per-op limiter never gated.

The ban is off by default: it keys by the real client IP the shared-NAT test contour
does not expose, so a ban there would be self-inflicted; detection still logs in the
contour, only the ban action is gated (GATEWAY_ABUSE_BAN_ENABLED). Rejection bans last
GATEWAY_ABUSE_BAN_DURATION; tripwire/honeytoken hits are near-zero-false-positive and
earn longer fixed bans. Each ban increments gateway_abuse_banned_total{reason}.

Operators see and lift active bans on the admin console's Throttled page; the gateway
syncs its active set to the backend every 30s (POST /api/v1/internal/bans/sync,
backend/internal/banview) and applies the operator unbans the response returns.

PRERELEASE phase AG. Docs baked into ARCHITECTURE / FUNCTIONAL (+ru) / both READMEs.
2026-06-21 08:54:20 +02:00
developer 3fffee7817 Merge pull request 'feat(telegram): split connector into home validator + remote bot (mTLS bot-link)' (#96) from feature/telegram-egress-botlink into development
CI / changes (push) Successful in 3s
CI / unit (push) Successful in 10s
CI / integration (push) Successful in 14s
CI / ui (push) Successful in 53s
CI / gate (push) Successful in 0s
CI / deploy (push) Successful in 1m24s
2026-06-21 05:35:02 +00:00
Ilia Denisov 860cfeb30f fix(deploy): make bot-link cert leaves readable by the distroless nonroot UID
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 13s
CI / ui (pull_request) Successful in 54s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 53s
The gateway and bot run on distroless nonroot (UID 65532) and bind-mount the
cert dir read-only, but gen-certs.sh wrote the keys 0600 (owner-only, the deploy
user), so both crash-looped at boot with "open /certs/*.key: permission denied"
and the deploy probe correctly failed (the contour's gateway was down).

The .crt files were already 0644 (openssl default); make the leaf keys 0644 too
so UID 65532 can read them. These are ephemeral TEST certificates regenerated
every deploy on the trusted runner; prod keys come from PROD_ secrets. The CA
private key stays 0600 (containers never read it).
2026-06-21 00:27:43 +02:00
Ilia Denisov 6aeb529f13 feat(telegram): split connector into home validator + remote bot
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 15s
CI / ui (pull_request) Successful in 53s
CI / gate (pull_request) Successful in 1s
CI / deploy (pull_request) Failing after 2m6s
Move all Telegram egress off the main host. The single connector held the
bot token, long-polled Telegram and answered the gateway/backend over the
trusted internal network, so the whole component (including login validation)
shared fate with its VPN sidecar. Split it into two binaries that share the
token:

- cmd/validator (home, no VPN): Mini App initData + Login Widget HMAC only,
  never calls the Bot API. The gateway dials it for Telegram auth, so game
  login is now independent of Telegram reachability.
- cmd/bot (remote): Bot API long-poll + sendMessage, the only component
  reaching Telegram. It holds no inbound port — it dials the gateway over a
  new reverse mTLS bot-link (pkg/proto/botlink/v1) and executes the send
  commands the gateway pushes.

The gateway funnels sends to the bot-link: out-of-app push is fire-and-forget
(at-most-once, dropped if no bot is connected); the backend admin broadcasts
reach a gateway-served relay that forwards them and awaits the bot's ack
(SendToUser/SendToGameChannel contract preserved). mTLS (pkg/mtls) is the one
inter-service link that leaves the trusted segment; validator<->gateway and
the relay stay plaintext internal. The bot is Telegram-rate-limited.

One bot now; the gateway bot registry, an owns_updates flag and per-command
ids leave seams for N later. Webhook rejected (one URL per token, adds inbound
+ a static address).

The unified test contour runs the split (the bot keeps its VPN sidecar and
dials the gateway by its internal name; bot-link certs from deploy/gen-certs.sh,
generated in CI). The prod wiring — the bot on a separate host (no VPN), the
gateway bot-link port published, PROD_ certs with scheduled rotation, an SSH
deploy of both hosts together — is the deferred final stage (PRERELEASE.md TX,
Stage 18).

Docs: ARCHITECTURE, PRERELEASE (phase TX), platform/telegram + gateway +
backend + deploy READMEs, FUNCTIONAL(+ru), CLAUDE.md, .env.example.
2026-06-21 00:19:07 +02:00
developer 2a8717c930 Merge pull request 'fix(ui): green both lobby scores on a tie, mute a 0:0 board' (#95) from feature/lobby-equal-score-color into development
CI / changes (push) Successful in 2s
CI / unit (push) Has been skipped
CI / integration (push) Has been skipped
CI / ui (push) Successful in 53s
CI / gate (push) Successful in 0s
CI / deploy (push) Successful in 58s
2026-06-20 19:33:06 +00:00
Ilia Denisov 264097bbf6 fix(ui): green both lobby scores on a tie, mute a 0:0 board
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Has been skipped
CI / integration (pull_request) Has been skipped
CI / ui (pull_request) Successful in 54s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m7s
The lobby tinted only the viewer's own number, and a tie counted as
"leading" — so an even score showed only the viewer's number green,
reading as if the viewer were ahead. A fresh 0:0 board did the same,
accenting the start of a game where nobody has scored.

scoreStanding is now per-seat: the viewer's seat stays green when
leading or tied and red when trailing; an opponent's seat greens only
when it ties the viewer for the lead, so an equal non-zero score paints
both numbers green. When the top score is 0 (nobody has moved) every
number is left muted, like a finished game.
2026-06-20 21:28:31 +02:00
developer 9824214fd7 ci(deploy): probe backend /readyz so a dead backend fails the deploy (#94)
CI / changes (push) Successful in 2s
CI / unit (push) Successful in 10s
CI / integration (push) Successful in 15s
CI / ui (push) Successful in 53s
CI / gate (push) Successful in 0s
CI / deploy (push) Successful in 1m11s
The post-deploy probe checked only the static landing and the gateway-served SPA shell, so a crash-looping backend passed the deploy green. Add an http://backend:8080/readyz probe on the internal network (and dump backend logs on failure) so an unready backend fails the deploy loudly.
2026-06-20 19:04:52 +00:00
developer c72adddb91 Merge pull request 'fix(engine): make .seed_version marker authoritative (no boot refusal)' (#93) from feature/dict-seed-marker-wins into development
CI / changes (push) Successful in 2s
CI / unit (push) Successful in 10s
CI / integration (push) Successful in 14s
CI / ui (push) Successful in 54s
CI / gate (push) Successful in 0s
CI / deploy (push) Failing after 21s
2026-06-20 18:25:12 +00:00
Ilia Denisov 95f5703372 fix(engine): make .seed_version marker authoritative (no boot refusal)
CI / changes (pull_request) Successful in 1s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 15s
CI / ui (pull_request) Successful in 53s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m2s
The seed-drift guard shipped as refuse-boot: the backend exited when
BACKEND_DICT_VERSION disagreed with the flat dir's recorded .seed_version. On
the test contour that turned a harmless-in-intent action — bumping the
TEST_DICT_VERSION variable to the active release (v1.2.1) on a volume seeded as
v1.0.0 — into a crash loop, because DICT_VERSION is the *seed* of a fresh
volume, not the active version (which the admin console drives).

Make the marker authoritative instead: OpenWithVersions resolves the flat dir's
version from .seed_version when present and ignores bootVersion on an
already-seeded volume; bootVersion only seeds a fresh volume's marker. So a
bumped build seed on a live volume is a no-op (it can't relabel live bytes and
can't void games pinned to the prior label), and it correctly seeds the next
fresh volume. The subdirectory scan now skips the resolved seed, so a version
also present as a subdir (e.g. v1.2.1 uploaded via the console while the build
seed is bumped to v1.2.1) is still loaded rather than shadowed by the flat bytes.

Tests: marker-wins over a bumped boot version; a bumped boot keeps the matching
subdir resident (the live-contour case). Docs updated (ARCHITECTURE §5, READMEs,
compose/.env, PRERELEASE DV) from "refuses to boot" to "marker wins / ignored".

Verified locally against v1.2.1: gofmt, build, vet, unit, integration green.
2026-06-20 20:06:57 +02:00
developer d40fe1edec Merge pull request 'feat(engine,deploy): seed-drift guard + track current dictionary release (v1.2.1)' (#92) from feature/dict-version-track-release into development
CI / changes (push) Successful in 2s
CI / unit (push) Successful in 9s
CI / integration (push) Successful in 16s
CI / ui (push) Successful in 53s
CI / gate (push) Successful in 0s
CI / deploy (push) Successful in 1m4s
2026-06-20 17:39:02 +00:00
Ilia Denisov a5db10c46e feat(engine,deploy): seed-drift guard + track current dictionary release
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 14s
CI / ui (pull_request) Successful in 54s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m12s
The dictionary release moved to v1.2.1 while DICT_VERSION stayed pinned at
v1.0.0 in CI and the image/compose seed defaults. Two problems:

1. CI validated against a stale dictionary.
2. The contour seed could be bumped on a live volume, which silently relabels
   the already-seeded bytes — voiding games pinned to the prior label and
   serving the wrong dictionary for new ones. The flat DAWGs carry no embedded
   version, so this drift was undetectable.

Changes:

- Seed-drift guard: OpenWithVersions records the flat dir's version in a
  .seed_version marker on first boot and refuses to start when a later
  BACKEND_DICT_VERSION disagrees. DICT_VERSION is now the seed for a *fresh*
  volume only; a live contour migrates through the admin console (old versions
  stay resident, in-progress games keep replaying).
- Track the current release: CI's DICT_VERSION centralised to one workflow-level
  env (v1.2.1); image/compose/.env seed defaults bumped to v1.2.1. The deploy
  job keeps reading the per-contour vars.TEST_DICT_VERSION.
- Docs: ARCHITECTURE §5 (decision record), backend/deploy READMEs, PRERELEASE
  tracker (DV row).

Verified locally against the v1.2.1 artifact: gofmt, build, vet, unit and
integration (-tags=integration) all green.
2026-06-20 19:26:32 +02:00
152 changed files with 9673 additions and 3616 deletions
+61 -31
View File
@@ -26,6 +26,13 @@ on:
push: push:
branches: [development] branches: [development]
# The dictionary release the test suite validates against — the current
# scrabble-dictionary release. Centralised here so a release bump is one edit; the
# unit/integration jobs inherit it. The deploy job overrides it per contour with
# vars.TEST_DICT_VERSION (the seed for a fresh volume), see deploy/README.md.
env:
DICT_VERSION: v1.2.1
jobs: jobs:
# changes detects which areas a PR/push touched, so the test jobs can skip when # changes detects which areas a PR/push touched, so the test jobs can skip when
# irrelevant. It defaults to running everything when the diff cannot be computed. # irrelevant. It defaults to running everything when the diff cannot be computed.
@@ -86,7 +93,6 @@ jobs:
# The engine consumes the published scrabble-solver module from this Gitea; # The engine consumes the published scrabble-solver module from this Gitea;
# GOPRIVATE makes go fetch it directly (skipping the public proxy/checksum DB). # GOPRIVATE makes go fetch it directly (skipping the public proxy/checksum DB).
GOPRIVATE: gitea.iliadenisov.ru/* GOPRIVATE: gitea.iliadenisov.ru/*
DICT_VERSION: v1.0.0
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
@@ -134,7 +140,6 @@ jobs:
# suite's TestMain terminates its own container, so disable it. # suite's TestMain terminates its own container, so disable it.
TESTCONTAINERS_RYUK_DISABLED: "true" TESTCONTAINERS_RYUK_DISABLED: "true"
GOPRIVATE: gitea.iliadenisov.ru/* GOPRIVATE: gitea.iliadenisov.ru/*
DICT_VERSION: v1.0.0
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
@@ -255,11 +260,16 @@ jobs:
GM_BASICAUTH_HASH: ${{ secrets.TEST_GM_BASICAUTH_HASH }} GM_BASICAUTH_HASH: ${{ secrets.TEST_GM_BASICAUTH_HASH }}
GRAFANA_ADMIN_PASSWORD: ${{ secrets.TEST_GRAFANA_ADMIN_PASSWORD }} GRAFANA_ADMIN_PASSWORD: ${{ secrets.TEST_GRAFANA_ADMIN_PASSWORD }}
TELEGRAM_BOT_TOKEN: ${{ secrets.TEST_TELEGRAM_BOT_TOKEN }} TELEGRAM_BOT_TOKEN: ${{ secrets.TEST_TELEGRAM_BOT_TOKEN }}
TELEGRAM_PROMO_BOT_TOKEN: ${{ secrets.TEST_TELEGRAM_PROMO_BOT_TOKEN }}
GM_BASICAUTH_USER: ${{ vars.TEST_GM_BASICAUTH_USER }} GM_BASICAUTH_USER: ${{ vars.TEST_GM_BASICAUTH_USER }}
GRAFANA_ROOT_URL: ${{ vars.TEST_GRAFANA_ROOT_URL }} GRAFANA_ROOT_URL: ${{ vars.TEST_GRAFANA_ROOT_URL }}
CADDY_SITE_ADDRESS: ${{ vars.TEST_CADDY_SITE_ADDRESS }} CADDY_SITE_ADDRESS: ${{ vars.TEST_CADDY_SITE_ADDRESS }}
TELEGRAM_MINIAPP_URL: ${{ vars.TEST_TELEGRAM_MINIAPP_URL }} TELEGRAM_MINIAPP_URL: ${{ vars.TEST_TELEGRAM_MINIAPP_URL }}
TELEGRAM_GAME_CHANNEL_ID: ${{ vars.TEST_TELEGRAM_GAME_CHANNEL_ID }} TELEGRAM_GAME_CHANNEL_ID: ${{ vars.TEST_TELEGRAM_GAME_CHANNEL_ID }}
TELEGRAM_CHAT_ID: ${{ vars.TEST_TELEGRAM_CHAT_ID }}
TELEGRAM_BOT_USERNAME: ${{ vars.TEST_TELEGRAM_BOT_USERNAME }}
# The promo button reuses the UI's Mini App link variable.
TELEGRAM_BOT_LINK: ${{ vars.TEST_VITE_TELEGRAM_LINK }}
# The test contour always uses Telegram's test environment — pinned here, # The test contour always uses Telegram's test environment — pinned here,
# not an operator variable. The prod workflow leaves it false. # not an operator variable. The prod workflow leaves it false.
TELEGRAM_TEST_ENV: "true" TELEGRAM_TEST_ENV: "true"
@@ -283,62 +293,82 @@ jobs:
mkdir -p "$conf" mkdir -p "$conf"
cp -r caddy otelcol prometheus tempo grafana "$conf"/ cp -r caddy otelcol prometheus tempo grafana "$conf"/
export SCRABBLE_CONFIG_DIR="$conf" export SCRABBLE_CONFIG_DIR="$conf"
# Bot-link mTLS material for the test contour: a private CA + gateway/bot
# leaves (CN=gateway, the service name the bot dials). Prod supplies these
# from PROD_ secrets instead. Regenerated each deploy; both ends redeploy
# together so they always share the fresh CA (see deploy/gen-certs.sh).
bash "$GITHUB_WORKSPACE/deploy/gen-certs.sh" "$conf/certs"
# App version for the About screen: the git tag if present, else the short SHA # App version for the About screen: the git tag if present, else the short SHA
# (the test checkout is shallow/untagged, so this is the SHA here — fine). # (the test checkout is shallow/untagged, so this is the SHA here — fine).
export APP_VERSION="$(git -C "$GITHUB_WORKSPACE" describe --tags --always 2>/dev/null || echo dev)" export APP_VERSION="$(git -C "$GITHUB_WORKSPACE" describe --tags --always 2>/dev/null || echo dev)"
docker compose --ansi never build --progress plain # The telegram-local profile brings the bot + its VPN sidecar; prod runs the
docker compose --ansi never up -d --remove-orphans # bot on its own host instead (deploy/docker-compose.bot.yml), and the prod
# main host omits both. Without the profile they would not start here.
docker compose --ansi never --profile telegram-local build --progress plain
docker compose --ansi never --profile telegram-local up -d --remove-orphans
# The config-only services bind-mount the reseeded config dir. A plain `up -d` # The config-only services bind-mount the reseeded config dir. A plain `up -d`
# leaves them on the previous bind mount (the dir was rm'd + recreated), so a # leaves them on the previous bind mount (the dir was rm'd + recreated), so a
# changed Caddyfile or Grafana dashboard is ignored — force-recreate them to # changed Caddyfile or Grafana dashboard is ignored — force-recreate them to
# pick up the fresh config. # pick up the fresh config.
docker compose --ansi never up -d --force-recreate --no-deps caddy otelcol prometheus tempo grafana docker compose --ansi never up -d --force-recreate --no-deps caddy otelcol prometheus tempo grafana
- name: Probe the landing and the gateway through caddy - name: Probe the landing, gateway and backend
run: | run: |
set -u set -u
# Two probes through the contour caddy: "/" is the static # Three probes. "/" is the static landing container and "/app/" the
# landing container, "/app/" is the gateway-served SPA shell. # gateway-served SPA shell (both through the contour caddy on the edge net).
# The backend /readyz is probed on the internal net as well: the caddy probes
# are blind to a crash-looping backend (the landing is static and the SPA
# shell is served without it), which let a bad deploy go green while the
# backend was down — so check it directly here.
for i in $(seq 1 20); do for i in $(seq 1 20); do
if docker run --rm --network edge alpine:3.20 wget -q -T 5 -O /dev/null http://scrabble/ && if docker run --rm --network edge alpine:3.20 wget -q -T 5 -O /dev/null http://scrabble/ &&
docker run --rm --network edge alpine:3.20 wget -q -T 5 -O /dev/null http://scrabble/app/; then docker run --rm --network edge alpine:3.20 wget -q -T 5 -O /dev/null http://scrabble/app/ &&
echo "healthy: GET http://scrabble/ (landing) + /app/ (gateway)" docker run --rm --network scrabble-internal alpine:3.20 wget -q -T 5 -O /dev/null http://backend:8080/readyz; then
echo "healthy: GET / (landing) + /app/ (gateway) + backend /readyz"
exit 0 exit 0
fi fi
sleep 3 sleep 3
done done
echo "probe failed; recent landing + gateway logs:" echo "probe failed; recent landing + gateway + backend logs:"
docker logs --tail 50 scrabble-landing || true docker logs --tail 50 scrabble-landing || true
docker logs --tail 50 scrabble-gateway || true docker logs --tail 50 scrabble-gateway || true
docker logs --tail 50 scrabble-backend || true
exit 1 exit 1
- name: Probe the Telegram connector liveness - name: Probe the Telegram validator and bot liveness
run: | run: |
set -u set -u
# The gateway probe cannot see a crash-looping connector (it long-polls and # The gateway/backend probes cannot see a crash-looping validator or bot
# egresses through the VPN sidecar, with no public ingress). Inspect the # (the validator answers only internal gRPC; the bot long-polls + egresses
# container directly: it must be running, not restarting, with a stable # through the VPN sidecar with no public ingress). Inspect the containers
# restart count. A grace period lets the VPN handshake settle (the connector # directly: each must be running, not restarting, with a stable restart
# may restart a few times first). # count. A grace period lets the VPN handshake and the bot-link dial settle.
sleep 20 sleep 20
for i in $(seq 1 20); do for name in scrabble-telegram-validator scrabble-telegram-bot; do
status="$(docker inspect -f '{{.State.Status}}' scrabble-telegram 2>/dev/null || echo missing)" ok=
restarting="$(docker inspect -f '{{.State.Restarting}}' scrabble-telegram 2>/dev/null || echo true)" for i in $(seq 1 20); do
if [ "$status" = "running" ] && [ "$restarting" = "false" ]; then status="$(docker inspect -f '{{.State.Status}}' "$name" 2>/dev/null || echo missing)"
c1="$(docker inspect -f '{{.RestartCount}}' scrabble-telegram)" restarting="$(docker inspect -f '{{.State.Restarting}}' "$name" 2>/dev/null || echo true)"
sleep 5 if [ "$status" = "running" ] && [ "$restarting" = "false" ]; then
c2="$(docker inspect -f '{{.RestartCount}}' scrabble-telegram)" c1="$(docker inspect -f '{{.RestartCount}}' "$name")"
if [ "$c1" = "$c2" ]; then sleep 5
echo "connector healthy: status=$status restarts=$c2" c2="$(docker inspect -f '{{.RestartCount}}' "$name")"
exit 0 if [ "$c1" = "$c2" ]; then
echo "$name healthy: status=$status restarts=$c2"
ok=1
break
fi
echo "$name still restarting ($c1 -> $c2); waiting"
fi fi
echo "connector still restarting ($c1 -> $c2); waiting" sleep 3
done
if [ -z "$ok" ]; then
echo "$name not healthy; recent logs:"
docker logs --tail 80 "$name" || true
exit 1
fi fi
sleep 3
done done
echo "connector not healthy; recent logs:"
docker logs --tail 80 scrabble-telegram || true
exit 1
- name: Prune dangling images - name: Prune dangling images
if: always() if: always()
+266
View File
@@ -0,0 +1,266 @@
# Manual production rollout. Runs ONLY from master, ONLY on workflow_dispatch with
# confirm=deploy (development->master is merged + green first; this is the separate,
# deliberate prod step). Visible sequential jobs from most to least significant:
# build -> deploy-main -> deploy-bot -> verify
# The per-service rolling (postgres->backend->gateway->landing->validator->caddy),
# health-gating and auto-rollback live in deploy/prod-deploy.sh on the main host and
# show in the deploy-main log. Manual post-deploy rollback is prod-rollback.yaml.
# See deploy/README.md (prod runbook).
name: prod-deploy
run-name: "prod deploy ${{ github.sha }}"
on:
workflow_dispatch:
inputs:
confirm:
description: 'Type "deploy" to confirm a production rollout from master.'
required: true
default: ""
permissions:
contents: read
env:
NO_COLOR: "1"
DOCKER_CLI_HINTS: "false"
REGISTRY: docker.iliadenisov.ru/developer
jobs:
build:
if: ${{ github.ref == 'refs/heads/master' && inputs.confirm == 'deploy' }}
runs-on: ubuntu-latest
defaults:
run:
shell: bash
outputs:
tag: ${{ steps.ver.outputs.tag }}
env:
PROD_REGISTRY_USER: ${{ vars.PROD_REGISTRY_USER }}
PROD_REGISTRY_PASSWORD: ${{ secrets.PROD_REGISTRY_PASSWORD }}
VITE_TELEGRAM_BOT_ID: ${{ vars.PROD_VITE_TELEGRAM_BOT_ID }}
VITE_TELEGRAM_LINK: ${{ vars.PROD_VITE_TELEGRAM_LINK }}
VITE_TELEGRAM_GAME_CHANNEL_NAME: ${{ vars.PROD_VITE_TELEGRAM_GAME_CHANNEL_NAME }}
VITE_GATEWAY_URL: ${{ vars.PROD_VITE_GATEWAY_URL }}
POSTGRES_PASSWORD: ${{ secrets.PROD_POSTGRES_PASSWORD }}
GM_BASICAUTH_HASH: ${{ secrets.PROD_GM_BASICAUTH_HASH }}
TELEGRAM_MINIAPP_URL: ${{ vars.PROD_TELEGRAM_MINIAPP_URL }}
DICT_VERSION: ${{ vars.PROD_DICT_VERSION }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Compute version tag
id: ver
run: echo "tag=$(git describe --tags --always)" >> "$GITHUB_OUTPUT"
- name: Registry login
run: echo "$PROD_REGISTRY_PASSWORD" | docker login "${REGISTRY%%/*}" -u "$PROD_REGISTRY_USER" --password-stdin
- name: Build and push images
working-directory: deploy
run: |
export TAG="${{ steps.ver.outputs.tag }}" APP_VERSION="${{ steps.ver.outputs.tag }}" SCRABBLE_CONFIG_DIR=.
# The four main-stack images via compose (reuses the build args, incl. VERSION);
# the bot separately, since it is profiled out of the prod compose.
docker compose -f docker-compose.yml -f docker-compose.prod.yml build
docker compose -f docker-compose.yml -f docker-compose.prod.yml push backend gateway landing validator
docker build -f ../platform/telegram/Dockerfile --target bot --build-arg VERSION="$TAG" -t "$REGISTRY/scrabble-telegram-bot:$TAG" ..
docker push "$REGISTRY/scrabble-telegram-bot:$TAG"
deploy-main:
needs: build
runs-on: ubuntu-latest
defaults:
run:
shell: bash
env:
TAG: ${{ needs.build.outputs.tag }}
PROD_REGISTRY_USER: ${{ vars.PROD_REGISTRY_USER }}
PROD_REGISTRY_PASSWORD: ${{ secrets.PROD_REGISTRY_PASSWORD }}
PROD_SSH_KEY: ${{ secrets.PROD_SSH_KEY }}
PROD_SSH_KNOWN_HOSTS: ${{ secrets.PROD_SSH_KNOWN_HOSTS }}
MAIN_HOST: ${{ vars.PROD_MAIN_HOST }}
POSTGRES_PASSWORD: ${{ secrets.PROD_POSTGRES_PASSWORD }}
GM_BASICAUTH_HASH: ${{ secrets.PROD_GM_BASICAUTH_HASH }}
GRAFANA_ADMIN_PASSWORD: ${{ secrets.PROD_GRAFANA_ADMIN_PASSWORD }}
TELEGRAM_BOT_TOKEN: ${{ secrets.PROD_TELEGRAM_BOT_TOKEN }}
PROD_BOTLINK_CA: ${{ secrets.PROD_BOTLINK_CA }}
PROD_BOTLINK_GATEWAY_CERT: ${{ secrets.PROD_BOTLINK_GATEWAY_CERT }}
PROD_BOTLINK_GATEWAY_KEY: ${{ secrets.PROD_BOTLINK_GATEWAY_KEY }}
GM_BASICAUTH_USER: ${{ vars.PROD_GM_BASICAUTH_USER }}
GRAFANA_ROOT_URL: ${{ vars.PROD_GRAFANA_ROOT_URL }}
CADDY_SITE_ADDRESS: ${{ vars.PROD_CADDY_SITE_ADDRESS }}
LOG_LEVEL: ${{ vars.PROD_LOG_LEVEL }}
DICT_VERSION: ${{ vars.PROD_DICT_VERSION }}
POSTGRES_DB: ${{ vars.PROD_POSTGRES_DB }}
POSTGRES_USER: ${{ vars.PROD_POSTGRES_USER }}
TELEGRAM_MINIAPP_URL: ${{ vars.PROD_TELEGRAM_MINIAPP_URL }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up SSH
run: |
mkdir -p ~/.ssh && chmod 700 ~/.ssh
printf '%s\n' "$PROD_SSH_KEY" > ~/.ssh/id_deploy && chmod 600 ~/.ssh/id_deploy
printf '%s\n' "$PROD_SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
- name: Determine previous tag and migration
run: |
ssh_main() { ssh -i ~/.ssh/id_deploy -o BatchMode=yes "deploy@$MAIN_HOST" "$@"; }
PREV_TAG="$(ssh_main 'cat /opt/scrabble/DEPLOYED_TAG 2>/dev/null || echo none')"
MIGRATION=0
if [ "$PREV_TAG" != none ]; then
if ! git cat-file -e "$PREV_TAG^{commit}" 2>/dev/null; then
MIGRATION=1
elif git diff --name-only "$PREV_TAG..$TAG" -- backend/internal/postgres/migrations/ | grep -q .; then
MIGRATION=1
fi
fi
{ echo "PREV_TAG=$PREV_TAG"; echo "MIGRATION=$MIGRATION"; } >> "$GITHUB_ENV"
echo "prev=$PREV_TAG migration=$MIGRATION"
- name: Render main env + certs
run: |
umask 077
mkdir -p stage/certs-main
cat > stage/env.sh <<EOF
export REGISTRY='$REGISTRY'
export SCRABBLE_CONFIG_DIR='/opt/scrabble'
export POSTGRES_DB='${POSTGRES_DB:-scrabble}'
export POSTGRES_USER='${POSTGRES_USER:-scrabble}'
export POSTGRES_PASSWORD='$POSTGRES_PASSWORD'
export GM_BASICAUTH_USER='${GM_BASICAUTH_USER:-gm}'
export GM_BASICAUTH_HASH='$GM_BASICAUTH_HASH'
export GRAFANA_ADMIN_PASSWORD='$GRAFANA_ADMIN_PASSWORD'
export GRAFANA_ROOT_URL='$GRAFANA_ROOT_URL'
export CADDY_SITE_ADDRESS='$CADDY_SITE_ADDRESS'
export LOG_LEVEL='${LOG_LEVEL:-info}'
export DICT_VERSION='$DICT_VERSION'
export APP_VERSION='$TAG'
export TELEGRAM_BOT_TOKEN='$TELEGRAM_BOT_TOKEN'
export TELEGRAM_MINIAPP_URL='$TELEGRAM_MINIAPP_URL'
export GATEWAY_ABUSE_BAN_ENABLED='true'
EOF
printf '%s\n' "$PROD_BOTLINK_CA" > stage/certs-main/ca.crt
printf '%s\n' "$PROD_BOTLINK_GATEWAY_CERT" > stage/certs-main/gateway.crt
printf '%s\n' "$PROD_BOTLINK_GATEWAY_KEY" > stage/certs-main/gateway.key
chmod 644 stage/certs-main/*
- name: Deploy the main host
run: |
ssh_main() { ssh -i ~/.ssh/id_deploy -o BatchMode=yes "deploy@$MAIN_HOST" "$@"; }
ssh_main 'mkdir -p /opt/scrabble/compose'
tar -C deploy -czf - docker-compose.yml docker-compose.prod.yml prod-deploy.sh \
| ssh_main 'tar -C /opt/scrabble/compose -xzf -'
tar -C deploy -czf - caddy otelcol prometheus tempo grafana \
| ssh_main 'tar -C /opt/scrabble -xzf -'
tar -C stage -czf - certs-main \
| ssh_main 'rm -rf /opt/scrabble/certs && mkdir -p /opt/scrabble/certs && tar -C /opt/scrabble/certs --strip-components=1 -xzf -'
scp -i ~/.ssh/id_deploy -o BatchMode=yes stage/env.sh "deploy@$MAIN_HOST:/opt/scrabble/env.sh"
echo "$PROD_REGISTRY_PASSWORD" | ssh_main "docker login ${REGISTRY%%/*} -u $PROD_REGISTRY_USER --password-stdin"
ssh_main "TAG='$TAG' PREV_TAG='$PREV_TAG' MIGRATION='$MIGRATION' bash /opt/scrabble/compose/prod-deploy.sh"
deploy-bot:
needs: [build, deploy-main]
runs-on: ubuntu-latest
defaults:
run:
shell: bash
env:
TAG: ${{ needs.build.outputs.tag }}
PROD_REGISTRY_USER: ${{ vars.PROD_REGISTRY_USER }}
PROD_REGISTRY_PASSWORD: ${{ secrets.PROD_REGISTRY_PASSWORD }}
PROD_SSH_KEY: ${{ secrets.PROD_SSH_KEY }}
PROD_SSH_KNOWN_HOSTS: ${{ secrets.PROD_SSH_KNOWN_HOSTS }}
TG_HOST: ${{ vars.PROD_TG_HOST }}
MAIN_HOST: ${{ vars.PROD_MAIN_HOST }}
TELEGRAM_BOT_TOKEN: ${{ secrets.PROD_TELEGRAM_BOT_TOKEN }}
TELEGRAM_PROMO_BOT_TOKEN: ${{ secrets.PROD_TELEGRAM_PROMO_BOT_TOKEN }}
PROD_BOTLINK_CA: ${{ secrets.PROD_BOTLINK_CA }}
PROD_BOTLINK_BOT_CERT: ${{ secrets.PROD_BOTLINK_BOT_CERT }}
PROD_BOTLINK_BOT_KEY: ${{ secrets.PROD_BOTLINK_BOT_KEY }}
LOG_LEVEL: ${{ vars.PROD_LOG_LEVEL }}
TELEGRAM_MINIAPP_URL: ${{ vars.PROD_TELEGRAM_MINIAPP_URL }}
TELEGRAM_GAME_CHANNEL_ID: ${{ vars.PROD_TELEGRAM_GAME_CHANNEL_ID }}
TELEGRAM_CHAT_ID: ${{ vars.PROD_TELEGRAM_CHAT_ID }}
TELEGRAM_BOT_USERNAME: ${{ vars.PROD_TELEGRAM_BOT_USERNAME }}
TELEGRAM_BOT_LINK: ${{ vars.PROD_VITE_TELEGRAM_LINK }}
steps:
- uses: actions/checkout@v4
- name: Set up SSH
run: |
mkdir -p ~/.ssh && chmod 700 ~/.ssh
printf '%s\n' "$PROD_SSH_KEY" > ~/.ssh/id_deploy && chmod 600 ~/.ssh/id_deploy
printf '%s\n' "$PROD_SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
- name: Render bot env + certs
run: |
umask 077
mkdir -p stage/certs-bot
cat > stage/env.bot.sh <<EOF
export SCRABBLE_CONFIG_DIR='/opt/scrabble'
export BOT_IMAGE='$REGISTRY/scrabble-telegram-bot:$TAG'
export BOTLINK_GATEWAY_ADDR='$MAIN_HOST:9443'
export TELEGRAM_BOT_TOKEN='$TELEGRAM_BOT_TOKEN'
export TELEGRAM_MINIAPP_URL='$TELEGRAM_MINIAPP_URL'
export TELEGRAM_GAME_CHANNEL_ID='$TELEGRAM_GAME_CHANNEL_ID'
export TELEGRAM_CHAT_ID='$TELEGRAM_CHAT_ID'
export TELEGRAM_PROMO_BOT_TOKEN='$TELEGRAM_PROMO_BOT_TOKEN'
export TELEGRAM_BOT_USERNAME='$TELEGRAM_BOT_USERNAME'
export TELEGRAM_BOT_LINK='$TELEGRAM_BOT_LINK'
export LOG_LEVEL='${LOG_LEVEL:-info}'
EOF
printf '%s\n' "$PROD_BOTLINK_CA" > stage/certs-bot/ca.crt
printf '%s\n' "$PROD_BOTLINK_BOT_CERT" > stage/certs-bot/bot.crt
printf '%s\n' "$PROD_BOTLINK_BOT_KEY" > stage/certs-bot/bot.key
chmod 644 stage/certs-bot/*
- name: Deploy the bot host
run: |
ssh_tg() { ssh -i ~/.ssh/id_deploy -o BatchMode=yes "deploy@$TG_HOST" "$@"; }
ssh_tg 'mkdir -p /opt/scrabble/compose'
tar -C deploy -czf - docker-compose.bot.yml | ssh_tg 'tar -C /opt/scrabble/compose -xzf -'
tar -C stage -czf - certs-bot \
| ssh_tg 'rm -rf /opt/scrabble/certs && mkdir -p /opt/scrabble/certs && tar -C /opt/scrabble/certs --strip-components=1 -xzf -'
scp -i ~/.ssh/id_deploy -o BatchMode=yes stage/env.bot.sh "deploy@$TG_HOST:/opt/scrabble/env.bot.sh"
echo "$PROD_REGISTRY_PASSWORD" | ssh_tg "docker login ${REGISTRY%%/*} -u $PROD_REGISTRY_USER --password-stdin"
ssh_tg 'set -a; . /opt/scrabble/env.bot.sh; set +a; cd /opt/scrabble/compose;
docker compose -f docker-compose.bot.yml pull;
docker compose -f docker-compose.bot.yml up -d'
ssh_tg 'for i in $(seq 1 20); do
s=$(docker inspect -f "{{.State.Status}}" scrabble-telegram-bot 2>/dev/null || echo missing)
r=$(docker inspect -f "{{.State.Restarting}}" scrabble-telegram-bot 2>/dev/null || echo true)
if [ "$s" = running ] && [ "$r" = false ]; then
c1=$(docker inspect -f "{{.RestartCount}}" scrabble-telegram-bot); sleep 5
c2=$(docker inspect -f "{{.RestartCount}}" scrabble-telegram-bot)
[ "$c1" = "$c2" ] && { echo "bot healthy"; exit 0; }
fi
sleep 3
done
echo "bot not healthy:"; docker logs --tail 80 scrabble-telegram-bot; exit 1'
verify:
needs: [deploy-main, deploy-bot]
runs-on: ubuntu-latest
defaults:
run:
shell: bash
env:
PROD_SSH_KEY: ${{ secrets.PROD_SSH_KEY }}
PROD_SSH_KNOWN_HOSTS: ${{ secrets.PROD_SSH_KNOWN_HOSTS }}
MAIN_HOST: ${{ vars.PROD_MAIN_HOST }}
CADDY_SITE_ADDRESS: ${{ vars.PROD_CADDY_SITE_ADDRESS }}
steps:
- name: Set up SSH
run: |
mkdir -p ~/.ssh && chmod 700 ~/.ssh
printf '%s\n' "$PROD_SSH_KEY" > ~/.ssh/id_deploy && chmod 600 ~/.ssh/id_deploy
printf '%s\n' "$PROD_SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
- name: Verify the public site
run: |
domain="${CADDY_SITE_ADDRESS%% *}"
ssh -i ~/.ssh/id_deploy -o BatchMode=yes "deploy@$MAIN_HOST" "for i in \$(seq 1 20); do
if curl -fsS -k --resolve $domain:443:127.0.0.1 https://$domain/ -o /dev/null &&
curl -fsS -k --resolve $domain:443:127.0.0.1 https://$domain/app/ -o /dev/null &&
docker run --rm --network scrabble-internal alpine:3.20 wget -q -T 5 -O /dev/null http://backend:8080/readyz; then
echo 'public site + /app/ + backend healthy'; exit 0
fi
sleep 5
done
echo 'public verify failed; recent caddy + gateway + backend logs:'
docker logs --tail 40 scrabble-caddy; docker logs --tail 40 scrabble-gateway; docker logs --tail 40 scrabble-backend
exit 1"
+223
View File
@@ -0,0 +1,223 @@
# Manual production rollback. Runs ONLY from master, ONLY on workflow_dispatch with
# confirm=rollback. Re-deploys an already-published image tag (no build): leave
# target_version blank to roll back to the previously deployed version (read from the
# main host), or set it to a specific release tag from the Releases page. The
# re-deploy is the same rolling, health-gated path as prod-deploy (TAG=target,
# MIGRATION=0 — rollback is image-only and never migrates the DB; image rollback is
# DB-safe under the expand-contract rule). See deploy/README.md (prod runbook).
name: prod-rollback
run-name: "prod rollback ${{ inputs.target_version || 'previous' }}"
on:
workflow_dispatch:
inputs:
confirm:
description: 'Type "rollback" to confirm a production rollback.'
required: true
default: ""
target_version:
description: "Release tag to roll back to (blank = the previous deployed version)."
required: false
default: ""
permissions:
contents: read
env:
NO_COLOR: "1"
DOCKER_CLI_HINTS: "false"
REGISTRY: docker.iliadenisov.ru/developer
jobs:
rollback-main:
if: ${{ github.ref == 'refs/heads/master' && inputs.confirm == 'rollback' }}
runs-on: ubuntu-latest
defaults:
run:
shell: bash
outputs:
target: ${{ steps.resolve.outputs.target }}
env:
PROD_REGISTRY_USER: ${{ vars.PROD_REGISTRY_USER }}
PROD_REGISTRY_PASSWORD: ${{ secrets.PROD_REGISTRY_PASSWORD }}
PROD_SSH_KEY: ${{ secrets.PROD_SSH_KEY }}
PROD_SSH_KNOWN_HOSTS: ${{ secrets.PROD_SSH_KNOWN_HOSTS }}
MAIN_HOST: ${{ vars.PROD_MAIN_HOST }}
POSTGRES_PASSWORD: ${{ secrets.PROD_POSTGRES_PASSWORD }}
GM_BASICAUTH_HASH: ${{ secrets.PROD_GM_BASICAUTH_HASH }}
GRAFANA_ADMIN_PASSWORD: ${{ secrets.PROD_GRAFANA_ADMIN_PASSWORD }}
TELEGRAM_BOT_TOKEN: ${{ secrets.PROD_TELEGRAM_BOT_TOKEN }}
PROD_BOTLINK_CA: ${{ secrets.PROD_BOTLINK_CA }}
PROD_BOTLINK_GATEWAY_CERT: ${{ secrets.PROD_BOTLINK_GATEWAY_CERT }}
PROD_BOTLINK_GATEWAY_KEY: ${{ secrets.PROD_BOTLINK_GATEWAY_KEY }}
GM_BASICAUTH_USER: ${{ vars.PROD_GM_BASICAUTH_USER }}
GRAFANA_ROOT_URL: ${{ vars.PROD_GRAFANA_ROOT_URL }}
CADDY_SITE_ADDRESS: ${{ vars.PROD_CADDY_SITE_ADDRESS }}
LOG_LEVEL: ${{ vars.PROD_LOG_LEVEL }}
DICT_VERSION: ${{ vars.PROD_DICT_VERSION }}
POSTGRES_DB: ${{ vars.PROD_POSTGRES_DB }}
POSTGRES_USER: ${{ vars.PROD_POSTGRES_USER }}
TELEGRAM_MINIAPP_URL: ${{ vars.PROD_TELEGRAM_MINIAPP_URL }}
INPUT_TARGET: ${{ inputs.target_version }}
steps:
- uses: actions/checkout@v4
- name: Set up SSH
run: |
mkdir -p ~/.ssh && chmod 700 ~/.ssh
printf '%s\n' "$PROD_SSH_KEY" > ~/.ssh/id_deploy && chmod 600 ~/.ssh/id_deploy
printf '%s\n' "$PROD_SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
- name: Resolve rollback target
id: resolve
run: |
ssh_main() { ssh -i ~/.ssh/id_deploy -o BatchMode=yes "deploy@$MAIN_HOST" "$@"; }
CURRENT="$(ssh_main 'cat /opt/scrabble/DEPLOYED_TAG 2>/dev/null || echo none')"
if [ -n "$INPUT_TARGET" ]; then
TARGET="$INPUT_TARGET"
else
TARGET="$(ssh_main 'cat /opt/scrabble/PREVIOUS_TAG 2>/dev/null || echo none')"
fi
if [ -z "$TARGET" ] || [ "$TARGET" = none ]; then
echo "no rollback target (no PREVIOUS_TAG on the host and no target_version input)"; exit 1
fi
if [ "$TARGET" = "$CURRENT" ]; then
echo "target $TARGET is already the deployed version; nothing to do"; exit 1
fi
echo "rolling back: current=$CURRENT -> target=$TARGET"
echo "target=$TARGET" >> "$GITHUB_OUTPUT"
{ echo "TARGET=$TARGET"; echo "CURRENT=$CURRENT"; } >> "$GITHUB_ENV"
- name: Render main env + certs
run: |
umask 077
mkdir -p stage/certs-main
cat > stage/env.sh <<EOF
export REGISTRY='$REGISTRY'
export SCRABBLE_CONFIG_DIR='/opt/scrabble'
export POSTGRES_DB='${POSTGRES_DB:-scrabble}'
export POSTGRES_USER='${POSTGRES_USER:-scrabble}'
export POSTGRES_PASSWORD='$POSTGRES_PASSWORD'
export GM_BASICAUTH_USER='${GM_BASICAUTH_USER:-gm}'
export GM_BASICAUTH_HASH='$GM_BASICAUTH_HASH'
export GRAFANA_ADMIN_PASSWORD='$GRAFANA_ADMIN_PASSWORD'
export GRAFANA_ROOT_URL='$GRAFANA_ROOT_URL'
export CADDY_SITE_ADDRESS='$CADDY_SITE_ADDRESS'
export LOG_LEVEL='${LOG_LEVEL:-info}'
export DICT_VERSION='$DICT_VERSION'
export APP_VERSION='$TARGET'
export TELEGRAM_BOT_TOKEN='$TELEGRAM_BOT_TOKEN'
export TELEGRAM_MINIAPP_URL='$TELEGRAM_MINIAPP_URL'
export GATEWAY_ABUSE_BAN_ENABLED='true'
EOF
printf '%s\n' "$PROD_BOTLINK_CA" > stage/certs-main/ca.crt
printf '%s\n' "$PROD_BOTLINK_GATEWAY_CERT" > stage/certs-main/gateway.crt
printf '%s\n' "$PROD_BOTLINK_GATEWAY_KEY" > stage/certs-main/gateway.key
chmod 644 stage/certs-main/*
- name: Roll the main host back
run: |
ssh_main() { ssh -i ~/.ssh/id_deploy -o BatchMode=yes "deploy@$MAIN_HOST" "$@"; }
ssh_main 'mkdir -p /opt/scrabble/compose'
tar -C deploy -czf - docker-compose.yml docker-compose.prod.yml prod-deploy.sh \
| ssh_main 'tar -C /opt/scrabble/compose -xzf -'
tar -C deploy -czf - caddy otelcol prometheus tempo grafana \
| ssh_main 'tar -C /opt/scrabble -xzf -'
tar -C stage -czf - certs-main \
| ssh_main 'rm -rf /opt/scrabble/certs && mkdir -p /opt/scrabble/certs && tar -C /opt/scrabble/certs --strip-components=1 -xzf -'
scp -i ~/.ssh/id_deploy -o BatchMode=yes stage/env.sh "deploy@$MAIN_HOST:/opt/scrabble/env.sh"
echo "$PROD_REGISTRY_PASSWORD" | ssh_main "docker login ${REGISTRY%%/*} -u $PROD_REGISTRY_USER --password-stdin"
# Image-only rollback: no migration window (TAG=target, MIGRATION=0). A failed
# rollback's auto-revert returns to the current version (PREV_TAG=$CURRENT).
ssh_main "TAG='$TARGET' PREV_TAG='$CURRENT' MIGRATION=0 bash /opt/scrabble/compose/prod-deploy.sh"
rollback-bot:
needs: rollback-main
runs-on: ubuntu-latest
defaults:
run:
shell: bash
env:
TARGET: ${{ needs.rollback-main.outputs.target }}
PROD_REGISTRY_USER: ${{ vars.PROD_REGISTRY_USER }}
PROD_REGISTRY_PASSWORD: ${{ secrets.PROD_REGISTRY_PASSWORD }}
PROD_SSH_KEY: ${{ secrets.PROD_SSH_KEY }}
PROD_SSH_KNOWN_HOSTS: ${{ secrets.PROD_SSH_KNOWN_HOSTS }}
TG_HOST: ${{ vars.PROD_TG_HOST }}
MAIN_HOST: ${{ vars.PROD_MAIN_HOST }}
TELEGRAM_BOT_TOKEN: ${{ secrets.PROD_TELEGRAM_BOT_TOKEN }}
TELEGRAM_PROMO_BOT_TOKEN: ${{ secrets.PROD_TELEGRAM_PROMO_BOT_TOKEN }}
PROD_BOTLINK_CA: ${{ secrets.PROD_BOTLINK_CA }}
PROD_BOTLINK_BOT_CERT: ${{ secrets.PROD_BOTLINK_BOT_CERT }}
PROD_BOTLINK_BOT_KEY: ${{ secrets.PROD_BOTLINK_BOT_KEY }}
LOG_LEVEL: ${{ vars.PROD_LOG_LEVEL }}
TELEGRAM_MINIAPP_URL: ${{ vars.PROD_TELEGRAM_MINIAPP_URL }}
TELEGRAM_GAME_CHANNEL_ID: ${{ vars.PROD_TELEGRAM_GAME_CHANNEL_ID }}
TELEGRAM_CHAT_ID: ${{ vars.PROD_TELEGRAM_CHAT_ID }}
TELEGRAM_BOT_USERNAME: ${{ vars.PROD_TELEGRAM_BOT_USERNAME }}
TELEGRAM_BOT_LINK: ${{ vars.PROD_VITE_TELEGRAM_LINK }}
steps:
- uses: actions/checkout@v4
- name: Set up SSH
run: |
mkdir -p ~/.ssh && chmod 700 ~/.ssh
printf '%s\n' "$PROD_SSH_KEY" > ~/.ssh/id_deploy && chmod 600 ~/.ssh/id_deploy
printf '%s\n' "$PROD_SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
- name: Render bot env + certs
run: |
umask 077
mkdir -p stage/certs-bot
cat > stage/env.bot.sh <<EOF
export SCRABBLE_CONFIG_DIR='/opt/scrabble'
export BOT_IMAGE='$REGISTRY/scrabble-telegram-bot:$TARGET'
export BOTLINK_GATEWAY_ADDR='$MAIN_HOST:9443'
export TELEGRAM_BOT_TOKEN='$TELEGRAM_BOT_TOKEN'
export TELEGRAM_MINIAPP_URL='$TELEGRAM_MINIAPP_URL'
export TELEGRAM_GAME_CHANNEL_ID='$TELEGRAM_GAME_CHANNEL_ID'
export TELEGRAM_CHAT_ID='$TELEGRAM_CHAT_ID'
export TELEGRAM_PROMO_BOT_TOKEN='$TELEGRAM_PROMO_BOT_TOKEN'
export TELEGRAM_BOT_USERNAME='$TELEGRAM_BOT_USERNAME'
export TELEGRAM_BOT_LINK='$TELEGRAM_BOT_LINK'
export LOG_LEVEL='${LOG_LEVEL:-info}'
EOF
printf '%s\n' "$PROD_BOTLINK_CA" > stage/certs-bot/ca.crt
printf '%s\n' "$PROD_BOTLINK_BOT_CERT" > stage/certs-bot/bot.crt
printf '%s\n' "$PROD_BOTLINK_BOT_KEY" > stage/certs-bot/bot.key
chmod 644 stage/certs-bot/*
- name: Roll the bot host back
run: |
ssh_tg() { ssh -i ~/.ssh/id_deploy -o BatchMode=yes "deploy@$TG_HOST" "$@"; }
ssh_tg 'mkdir -p /opt/scrabble/compose'
tar -C deploy -czf - docker-compose.bot.yml | ssh_tg 'tar -C /opt/scrabble/compose -xzf -'
tar -C stage -czf - certs-bot \
| ssh_tg 'rm -rf /opt/scrabble/certs && mkdir -p /opt/scrabble/certs && tar -C /opt/scrabble/certs --strip-components=1 -xzf -'
scp -i ~/.ssh/id_deploy -o BatchMode=yes stage/env.bot.sh "deploy@$TG_HOST:/opt/scrabble/env.bot.sh"
echo "$PROD_REGISTRY_PASSWORD" | ssh_tg "docker login ${REGISTRY%%/*} -u $PROD_REGISTRY_USER --password-stdin"
ssh_tg 'set -a; . /opt/scrabble/env.bot.sh; set +a; cd /opt/scrabble/compose;
docker compose -f docker-compose.bot.yml pull;
docker compose -f docker-compose.bot.yml up -d'
verify:
needs: [rollback-main, rollback-bot]
runs-on: ubuntu-latest
defaults:
run:
shell: bash
env:
PROD_SSH_KEY: ${{ secrets.PROD_SSH_KEY }}
PROD_SSH_KNOWN_HOSTS: ${{ secrets.PROD_SSH_KNOWN_HOSTS }}
MAIN_HOST: ${{ vars.PROD_MAIN_HOST }}
CADDY_SITE_ADDRESS: ${{ vars.PROD_CADDY_SITE_ADDRESS }}
steps:
- name: Set up SSH
run: |
mkdir -p ~/.ssh && chmod 700 ~/.ssh
printf '%s\n' "$PROD_SSH_KEY" > ~/.ssh/id_deploy && chmod 600 ~/.ssh/id_deploy
printf '%s\n' "$PROD_SSH_KNOWN_HOSTS" > ~/.ssh/known_hosts
- name: Verify the public site
run: |
domain="${CADDY_SITE_ADDRESS%% *}"
ssh -i ~/.ssh/id_deploy -o BatchMode=yes "deploy@$MAIN_HOST" "for i in \$(seq 1 20); do
if curl -fsS -k --resolve $domain:443:127.0.0.1 https://$domain/ -o /dev/null &&
docker run --rm --network scrabble-internal alpine:3.20 wget -q -T 5 -O /dev/null http://backend:8080/readyz; then
echo 'rolled-back site healthy'; exit 0
fi
sleep 5
done
echo 'verify failed'; docker logs --tail 40 scrabble-caddy; docker logs --tail 40 scrabble-backend; exit 1"
+4
View File
@@ -17,5 +17,9 @@
**/.env.local **/.env.local
**/.env.*.local **/.env.*.local
# Bot-link mTLS material: private keys never belong in the repo. The test contour
# generates them with deploy/gen-certs.sh; prod supplies them from PROD_ secrets.
deploy/certs/
# Claude Code harness runtime artifacts # Claude Code harness runtime artifacts
.claude/scheduled_tasks.lock .claude/scheduled_tasks.lock
+89 -90
View File
@@ -1,96 +1,97 @@
# scrabble-game — project guide # scrabble-game — project guide
Multiplatform Scrabble game. Read this first every session. The owner drives the Multiplatform Scrabble game, **in production** at `https://erudit-game.ru`. Read this
project **one stage per session** (tariff constraint), so the repository — not first every session. The repository — not conversation memory — is the source of
conversation memory — is the source of continuity. Keep it that way. continuity; keep it that way.
## Sources of truth (read before changing behaviour) ## Sources of truth (read before changing behaviour)
- [`PLAN.md`](PLAN.md) — staged plan + **stage tracker** + per-stage *open - [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) — architecture, transport, security,
details to interview*. the decision record. Always describes the current state.
- [`PRERELEASE.md`](PRERELEASE.md) — pre-release hardening tracker (phases R1R7 - [`docs/FUNCTIONAL.md`](docs/FUNCTIONAL.md) (+ [`_ru`](docs/FUNCTIONAL_ru.md) mirror)
before Stage 18); same per-phase *interview + bake-back* discipline as `PLAN.md`. — per-domain user stories. English authoritative.
- [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) — architecture, transport, - [`docs/TESTING.md`](docs/TESTING.md) — test layers + the CI gate.
security, the decision record. Always describes current state.
- [`docs/FUNCTIONAL.md`](docs/FUNCTIONAL.md) (+ [`_ru`](docs/FUNCTIONAL_ru.md)
mirror) — per-domain user stories. English authoritative.
- [`docs/TESTING.md`](docs/TESTING.md) — test layers + the per-stage CI gate.
- [`docs/UI_DESIGN.md`](docs/UI_DESIGN.md) — the `ui` visual/interaction design system. - [`docs/UI_DESIGN.md`](docs/UI_DESIGN.md) — the `ui` visual/interaction design system.
- [`deploy/README.md`](deploy/README.md) — the deploy contour + the production
rollout / rollback runbook.
## Mandatory per-stage workflow ## How we work
**Start of a stage** - Inspect the relevant code path and the docs above before changing behaviour.
1. Read `PLAN.md` (the stage's scope + *open details*) and the relevant `docs/`. - **Interview the owner on every fork** — do not silently pick borderline decisions;
2. Analyse what the stage actually requires against the current code. offer options with brief pros/cons.
3. **Interview the owner** on every open detail and any fork not already fixed - Smallest correct diff. Prefer compact code; reuse before adding; do not add deps,
in the plan — do not silently pick borderline decisions. Offer options with seams or knobs until they are needed.
brief pros/cons. - **Update or add tests for every functional change**, at the layers
4. Only then implement, strictly within the stage's scope. `docs/TESTING.md` calls out.
- **Bake docs in the same PR**: update `docs/ARCHITECTURE.md`, `docs/FUNCTIONAL.md`
**End of a stage** (+`_ru`), the affected service `README` and Go Doc comments alongside the change.
1. Bake every new agreement back into `PLAN.md`, `docs/ARCHITECTURE.md`, - Document added packages, types, funcs, consts and vars with Go Doc comments.
`docs/FUNCTIONAL.md` (+ `_ru`), the affected service `README`, and Go Doc
comments — in the **same** PR. Correct earlier stages' docs/code if a new
decision changes them.
2. Update the stage tracker; add a line under *Refinements logged during
implementation* for any plan deviation.
3. Get CI green, then mark the stage done.
(The `stage-implementation` skill encodes this same loop and can be invoked.)
## Conventions ## Conventions
- All code, comments, identifiers, commits, docs, filenames in **English**. - All code, comments, identifiers, commits, docs, filenames in **English**.
- Chat with the owner follows the user-level `~/.claude/CLAUDE.md` (Russian, - Chat with the owner follows the user-level `~/.claude/CLAUDE.md` (Russian, the
the agreed persona and translation rules). agreed persona and translation rules).
- Mirror every point edit of `docs/FUNCTIONAL.md` into `docs/FUNCTIONAL_ru.md` - Mirror every point edit of `docs/FUNCTIONAL.md` into `docs/FUNCTIONAL_ru.md` in the
in the same patch (translate only the touched paragraphs). same patch (translate only the touched paragraphs).
- Prefer compact code; do not add deps, seams or knobs until a stage needs them.
Reuse before adding. Document added packages/types/funcs with Go Doc comments.
- Update or add tests for every functional change.
## Branching & CI ## Branching, CI & production
- **Two long-lived branches** (Stage 16 onward): **`development`** is the - **Two long-lived branches**: **`development`** is the integration branch; **`master`**
integration branch; **`master`** is the production trunk. Cut `feature/*` is the production trunk. Cut `feature/*` from `development` and PR back into it;
branches **from `development`** and PR them back into it. (Stages 015 used promote `development → master` via PR when ready to release. Both branches require
`master` as the trunk with `feature/* → master`; the genesis Stage 0 commit is one approval + the `CI / gate` check.
on `master` by necessity.) - A commit to a `feature/*` branch triggers nothing. The single workflow
- A commit to a `feature/*` branch triggers **nothing**. The single workflow `.gitea/workflows/ci.yaml` runs the full suite (`unit` + `integration` + `ui`) on a
`.gitea/workflows/ci.yaml` runs the full suite (`unit` + `integration` + `ui`) PR into `development` or `master`, and the gated **`deploy`** job auto-rolls the
on a PR into `development` or `master`, and the gated **`deploy`** job auto-rolls **test contour** on a PR into — or a push to — `development`
the **test contour** on a PR into — or a push to — `development` (`docker compose up -d --build` on the runner host + landing/SPA/backend probes). A
(`docker compose up -d --build` on the runner host + a `GET /` probe). A PR into PR into `master` is test-only.
`master` is test-only. - **Production is live on two hosts** (main + the Telegram bot host) and deploys
- Merge `development → master` only when CI is green; the **prod** deploy is then a **only manually** (`workflow_dispatch`), never automatically:
**manual** workflow (Stage 18), never automatic. Secrets/variables are prefixed - **`.gitea/workflows/prod-deploy.yaml`** (`confirm=deploy`, from `master`) builds +
`TEST_` / `PROD_` per contour (Gitea 1.26 has no deployment environments). pushes the images to the registry, then SSH-deploys both hosts — rolling per
- After any push, watch the run to green before declaring a stage done — use the service in dependency order, health-gated, **auto-rollback to the previous tag**;
ready-made watcher, never an inline poll loop: a schema migration adds a maintenance window + a consistent `pg_dump`. Four visible
`python3 ~/.claude/bin/gitea-ci-watch.py` (background). It reads `$GITEA_URL` jobs: build → deploy-main → deploy-bot → verify.
/ `$GITEA_TOKEN`; `gitea.iliadenisov.ru` is allow-listed in - **`.gitea/workflows/prod-rollback.yaml`** (`confirm=rollback`) re-deploys a prior
`.claude/settings.json`. Remote: `origin git@gitea.iliadenisov.ru:developer/scrabble-game.git`. release (blank `target_version` = the previous deployed version) — image-only,
rolling, health-gated.
- **Releases are git tags `vX.Y.Z` on `master`**; the deploy stamps `git describe
--tags` into the image tag, every binary (`pkg/version` via `-ldflags` → the
`service.version` telemetry attribute) and the SPA About screen. Tag the release
before deploying.
- Hosts are provisioned idempotently by **`deploy/ansible/`**. Per-contour
secrets/variables use the `TEST_` / `PROD_` prefix (Gitea 1.26 has no deployment
environments). Migrations must be **expand-contract** (backward-compatible) so
image rollback stays DB-safe. Full runbook + variable list in `deploy/README.md`.
- After any push, merge or deploy, **watch the run to green** before declaring done —
use the ready-made watcher (run it in the background), never an inline poll loop:
`python3 ~/.claude/bin/gitea-ci-watch.py`. It reads `$GITEA_URL` / `$GITEA_TOKEN`;
`gitea.iliadenisov.ru` is allow-listed in `.claude/settings.json`. Remote:
`origin git@gitea.iliadenisov.ru:developer/scrabble-game.git`.
## Stack ## Stack
Go 1.26.3, `go.work` monorepo, module paths `scrabble/<name>`. Dependencies are Go 1.26.3, `go.work` monorepo, module paths `scrabble/<name>`. Backend uses `gin` +
added **when first used** (incremental): backend uses `gin` + `zap` + `zap` + `pgx`/`go-jet`/`goose`/OTel. Client↔gateway is Connect-RPC + FlatBuffers
`pgx`/`go-jet`/`goose`/OTel (added in Stage 1). Client↔gateway is Connect-RPC + (h2c); gateway↔backend is REST/JSON + `X-User-ID` plus a gRPC server-stream for live
FlatBuffers (h2c); gateway↔backend is REST/JSON + `X-User-ID` plus a gRPC events. UI is pure HTML5/CSS on plain Svelte + Vite, packaged to native with
server-stream for live events. UI is pure HTML5/CSS on plain Svelte + Vite, Capacitor. No Redis.
packaged to native with Capacitor. Likely no Redis.
## Reused engine: `../scrabble-solver` (module `scrabble-solver`, Go 1.26.3) ## Reused engine: `../scrabble-solver` (module `scrabble-solver`, Go 1.26.3)
Embedded **in-process as a library** — there is no per-game container. Public Embedded **in-process as a library** (`replace scrabble-solver => ../scrabble-solver`
API to reuse (do not reimplement): in `go.work`; CI checks out the sibling from
`https://gitea.iliadenisov.ru/.../scrabble-solver.git`). There is no per-game
container. Public API to reuse (do not reimplement):
- `scrabble.NewSolver(rs, finder)``GenerateMoves(b, r, mode)` (ranked, - `scrabble.NewSolver(rs, finder)` → `GenerateMoves(b, r, mode)` (ranked, highest
highest score first), `ValidatePlay(b, dir, tiles)`, `ScorePlay(...)`; score first), `ValidatePlay(b, dir, tiles)`, `ScorePlay(...)`; `scrabble.Apply(b, m)`;
`scrabble.Apply(b, m)`; types `Move/Word/Placement/Direction/Mode` types `Move/Word/Placement/Direction/Mode`
(`scrabble-solver/scrabble/{solver,move,apply}.go`). (`scrabble-solver/scrabble/{solver,move,apply}.go`).
- `rules.English() / RussianScrabble() / Erudit()` - `rules.English() / RussianScrabble() / Erudit()` (`scrabble-solver/rules/rules.go`).
(`scrabble-solver/rules/rules.go`).
- `board.New / Parse / Clone / Transpose`; `rack.New / Add / Remove / Clone`; - `board.New / Parse / Clone / Transpose`; `rack.New / Add / Remove / Clone`;
`selfplay.NewBag / Draw / Len` (bag pattern). `selfplay.NewBag / Draw / Len` (bag pattern).
- Load committed dictionaries with `dawg.Load(path)` from - Load committed dictionaries with `dawg.Load(path)` from
@@ -99,20 +100,17 @@ API to reuse (do not reimplement):
Constraints: Constraints:
- Words/tiles are **alphabet-index bytes**, meaningful only with the matching - Words/tiles are **alphabet-index bytes**, meaningful only with the matching
`rules.Ruleset` (`Alphabet.Decode`); blank flag carried separately. **Decode `rules.Ruleset` (`Alphabet.Decode`); the blank flag is carried separately. **Decode
to real characters before persisting history** (history must be to real characters before persisting history** (history must be
dictionary-independent — see `docs/ARCHITECTURE.md` §9.1). dictionary-independent — see `docs/ARCHITECTURE.md` §9.1).
- The solver's `internal/*` is NOT importable from this sibling module. - The solver's `internal/*` is NOT importable from this sibling module.
- **GCG is test-only** in the solver (no public writer) — we ship our own. - **GCG is test-only** in the solver (no public writer) — we ship our own.
- Wiring: add `replace scrabble-solver => ../scrabble-solver` to `go.work` in - The solver uses published `github.com/iliadenisov/{alphabet,dafsa}` (no local replace).
**Stage 2** (when `internal/engine` first imports it), and make CI check out
the solver sibling (`https://gitea.iliadenisov.ru/.../scrabble-solver.git`).
It uses published `github.com/iliadenisov/{alphabet,dafsa}` (no local replace).
## Repository layout ## Repository layout
``` ```
go.work # use the existing modules; grows per stage go.work # the go.work monorepo
backend/ # module scrabble/backend backend/ # module scrabble/backend
cmd/backend/ # main: telemetry -> db+migrate -> cache -> server cmd/backend/ # main: telemetry -> db+migrate -> cache -> server
cmd/jetgen/ # dev tool: regenerate go-jet code (throwaway container) cmd/jetgen/ # dev tool: regenerate go-jet code (throwaway container)
@@ -123,12 +121,14 @@ backend/ # module scrabble/backend
internal/session/ # opaque tokens, sessions store, cache, service internal/session/ # opaque tokens, sessions store, cache, service
internal/server/ # gin engine, /api/v1 groups, X-User-ID, probes internal/server/ # gin engine, /api/v1 groups, X-User-ID, probes
internal/inttest/ # //go:build integration Postgres-backed tests internal/inttest/ # //go:build integration Postgres-backed tests
docs/ .gitea/workflows/ PLAN.md CLAUDE.md README.md gateway/ # module scrabble/gateway: Connect-RPC edge, embeds the SPA
gateway/ ui/ pkg/ # added by their stages ui/ # Svelte + Vite SPA + landing (Node project, not in go.work)
platform/telegram/ # Telegram connector side-service (Stage 9): bot + gRPC API pkg/ # shared: telemetry, version, wire/FlatBuffers, proto, mtls
loadtest/ # module scrabble/loadtest: the pre-release stress harness (R2) platform/telegram/ # Telegram side-service: cmd/validator (HMAC, no VPN) + cmd/bot (Bot API; dials gateway over reverse mTLS bot-link)
backend/Dockerfile gateway/Dockerfile platform/telegram/Dockerfile loadtest/Dockerfile # multi-stage distroless (Stage 16; loadtest R2); gateway/Dockerfile also has the `landing` target (R3) loadtest/ # module scrabble/loadtest: the load/stress harness
deploy/ # docker-compose (per-service limits, R7) + caddy + landing + otelcol (OTLP + docker_stats per-container metrics) + prometheus/tempo/grafana + postgres_exporter docs/ .gitea/workflows/ CLAUDE.md README.md
backend/Dockerfile gateway/Dockerfile platform/telegram/Dockerfile loadtest/Dockerfile # multi-stage distroless; gateway/Dockerfile has the `landing` target, platform/telegram/Dockerfile has `validator`+`bot` targets
deploy/ # docker-compose (+ prod overlay + bot host) + ansible provisioning + caddy + landing + otelcol (OTLP + docker_stats) + prometheus/tempo/grafana + node_exporter + postgres_exporter; prod-deploy.sh
``` ```
## Build & test ## Build & test
@@ -138,20 +138,19 @@ go build ./backend/... # per module ('./...' from the root won't span t
go vet ./backend/... go vet ./backend/...
gofmt -l . # must print nothing gofmt -l . # must print nothing
go test -count=1 ./backend/... go test -count=1 ./backend/...
go build ./platform/telegram/... && go test ./platform/telegram/... # Telegram connector (Stage 9) go build ./platform/telegram/... && go test ./platform/telegram/... # Telegram validator + bot
go run ./backend/cmd/backend # /healthz, /readyz on :8080 go run ./backend/cmd/backend # /healthz, /readyz on :8080
cd ui && pnpm install && pnpm check && pnpm test:unit && pnpm build # the UI (Stage 7+) cd ui && pnpm install && pnpm check && pnpm test:unit && pnpm build # the UI
pnpm start # UI mock mode: lobby -> game, no backend pnpm start # UI mock mode: lobby -> game, no backend
docker build -f backend/Dockerfile -t scrabble-backend . # images (Stage 16); gateway embeds the SPA docker build -f backend/Dockerfile -t scrabble-backend . # images; gateway embeds the SPA
docker build -f gateway/Dockerfile --target gateway -t scrabble-gateway . docker build -f gateway/Dockerfile --target gateway -t scrabble-gateway .
docker build -f gateway/Dockerfile --target landing -t scrabble-landing . # static landing (R3) docker build -f gateway/Dockerfile --target landing -t scrabble-landing . # static landing
docker compose -f deploy/docker-compose.yml config # validate the full contour docker compose -f deploy/docker-compose.yml config # validate the full contour
``` ```
The `ui` module is a Node project (pnpm), **not** in `go.work`; it is the `ui` job The `ui` module is a Node project (pnpm), **not** in `go.work`; it is the `ui` job of
of the single `.gitea/workflows/ci.yaml` (Stage 16 folded the former go-unit / the single `.gitea/workflows/ci.yaml`. Committed edge codegen under `ui/src/gen/`
integration / ui-test workflows into it). Committed edge codegen under `ui/src/gen/`
(regenerate with `pnpm codegen`); pnpm build-script approval lives in (regenerate with `pnpm codegen`); pnpm build-script approval lives in
`ui/pnpm-workspace.yaml` (`allowBuilds: esbuild: true`). `ui/pnpm-workspace.yaml` (`allowBuilds: esbuild: true`).
-1603
View File
File diff suppressed because it is too large Load Diff
-551
View File
@@ -1,551 +0,0 @@
# Pre-release plan — hardening before Stage 18
Living tracker for the pre-release hardening pass that runs **before Stage 18** (the
prod cutover). Same discipline as [`PLAN.md`](PLAN.md): one phase per session,
**interview the owner on the open details** at the start of each phase, bake every
decision back into `PLAN.md` / `docs/` / the affected `README`s / Go Doc comments in
the **same** PR, get CI green, then mark the phase done. Phases run as
`feature/* → development` PRs (the Stage 16 branch model); the owner approves+merges.
**Why now:** the system is feature-complete through Stage 17 and the test contour is
green, but there is **no prod data yet** — schema, wire labels and the dictionary
layout can still change for free. These phases spend that one-time freedom and harden
the edge before prod. Each phase maps back to the owner's raw pre-release TODO list
(numbers in the tracker).
## Phase tracker
| # | Phase | Raw TODOs | Status |
|---|-------|-----------|--------|
| R1 | Schema & naming reset | 1 + 10 | **done** |
| R2 | Stress harness + contour observability + early run | 9a | **done** |
| R3 | Edge hardening | 2 + 8 + 3 | **done** |
| R4 | Push enrichment + kill the last poll | 4 + 5 | **done** |
| R5 | Bundle slimming | 6 | **done** |
| R6 | Refactor + docs reconciliation + de-staging | 7 | **done** |
| R7 | Final stress run + tuning | 9b | **done** |
| UI | Tab-bar navigation redesign (drop the hamburger) | owner ad-hoc | **done** |
| MW | "Multiple words per turn" rule for Russian games (engine v1.1.0) | owner ad-hoc | **done** |
| MW2 | Single-word rule connectivity fix: the word must run along its own line through an existing tile (perpendicular-only contact no longer connects); single-tile direction picks the best legal word (engine v1.1.1) | owner ad-hoc | **done** |
| MW3 | Graceful replay degradation: a game whose journalled move became illegal under MW2 is closed as a draw (`end_reason='aborted'`) on open instead of erroring, with an impersonal organizer note in the history + GCG (migration `00002`) | owner ad-hoc | **done** |
| OW | Open auto-match: enter the game at once and wait inside it (robot after 90180 s) | owner ad-hoc | **done** |
| DA | Dictionary admin: online release-archive upload → word-diff preview → install/activate; versioned dict volume; active version persisted in DB; resident label = release tag | owner ad-hoc | **done** |
| AB | Manual account block (admin suspension): permanent/temporary with an editable en+ru reason picklist; a block forfeits the player's active games + cancels their open ones; a backend gate refuses a blocked account with **403 `account_blocked`**; the UI shows a terminal blocked screen and stops all push/poll; manual unblock; temporary blocks self-expire (migration `00003`) | owner ad-hoc | **done** |
| AI | Honest AI opponent in quick game: an explicit 🤖 AI / 👤 random selector (AI default); the robot is seated and moves at once; 7-day inactivity loss (the per-turn timeout reused); chat/nudge disabled, no statistics; the opponent is shown as 🤖 everywhere | owner ad-hoc | **done** |
| AD | Advertising banner ("ad network"): server-driven weighted campaigns (percent weight + validity window; the perpetual default fills the remainder up to 100%), bilingual messages shown by bot (`service_language`); eligibility = free account + empty hint wallet + no `no_banner` role (guests included); the resolved feed rides `profile.get` with a `notify` `banner` re-poll on eligibility change; `/_gm/banners` admin + global display timings; client smooth-weighted-round-robin rotation + fade-out/gap/fade-in UX. A single `app.load` bootstrap aggregator was considered and **deferred** (see ARCHITECTURE §10). | owner ad-hoc | **done** (PR1 backend+admin, PR2 UI rotation) |
| GL | Simultaneous quick-game cap (10): grey "New Game" + a lobby notice at the cap; backend gate on quick enqueue + invitation creation (409 `game_limit_reached`), accepting invitations exempt; `at_game_limit` rides `games.list` | owner ad-hoc | **done** |
| CR | In-game chat read receipts: per-message `unread_seats` bitmask (migration `00008`); a per-viewer unread **dot** in the lobby + game header (a nudge counts and clears when its recipient moves); reading = opening the move history (the 💬 fade-blinks twice) or the chat, acked (`chat.read`) only when unread; `chat_read_duration` + `chat_unread_messages` metrics + tracing + the **Scrabble — Messages** Grafana dashboard (follow-up PR); a message to a disguised robot opponent is born read; admin unread-only filter / read column / per-seat read card | owner ad-hoc | **done** |
| BX | Asymmetric per-user block + in-game controls: a block now silently suppresses everything **from** the blocked user (chat, nudge, friend requests, invitations are kept but never delivered/surfaced, born-read) while they notice nothing, **without** deleting the friendship (unblock restores it); auto-match excludes a block-related pair (either direction); in-game opponent card gains a ✖️ **block** control (mirroring 🤝, red "Block?" confirm, mutual-hide, struck name + hidden chat composer when blocked); optimistic apply + `user_blocked`/`user_unblocked` event confirm + rollback; admin user card gains **blocks / blocked-by / friends** cross-linked lists. Blocking a disguised-robot opponent is recorded per-game in a separate **`robot_blocks`** table (migration `00011`), keyed on game+seat with the seen name — never the shared robot account — so the matchmaker keeps giving robots; it shows in the blocked list and re-marks the in-game card | owner ad-hoc | **done** |
| FM | First-move tile draw (official rules): each seated player draws a tile, the one closest to "A" leads (a blank beats every letter), ties re-drawing until a single leader; **honest per-draw `crypto/rand` entropy**, not the bag seed, so the **record** (`game_setup_draws`, migration `00013`) — not a seed — is the only account of the outcome, kept for future **tournaments** (designed as a discrete per-tile "player N draws" step). Friend/AI draws at create; **auto-match draws at *open*** against a synthetic `uuid.Nil` opponent whose draw rows are back-filled on join, so the opener's seat is fixed up front and the existing open-game pre-move is preserved (no reseating, no play-gating). Admin `/_gm/games/:id` gains the recorded draw list + a simple **step-by-step board replay** (`ReplayTimeline`). | owner ad-hoc | **done** |
| SB | Single Telegram bot + per-user variant preferences: the two per-language bots collapse into **one** (drop `accounts.service_language`, `supported_languages`, the `*_EN`/`*_RU` env vars and game-language push routing — the single bot renders in the recipient's `preferred_language`); New Game variant gating moves to a profile **`variant_preferences`** set (default Erudit only, Erudit-first, server-enforced on the caller's auto-match/vs-AI/invitation-create paths, an invited friend may accept any variant); env vars collapse to unsuffixed `TELEGRAM_BOT_TOKEN`/`TELEGRAM_GAME_CHANNEL_ID`/`VITE_TELEGRAM_LINK`/`VITE_TELEGRAM_GAME_CHANNEL_NAME` and `GATEWAY_DEFAULT_SUPPORTED_LANGUAGES` is removed; wire drops `service_language`/`supported_languages` (Session, ValidateInitDataResponse) + the push `language` routing field and adds `variant_preferences` to Profile/UpdateProfile. | owner ad-hoc | **done** |
| → | Stage 18 — prod contour deploy | — | see [`PLAN.md`](PLAN.md) |
## Key findings (these reshaped the raw list — read before starting a phase)
- **R1 (TODO 1 + 10) is one cheap moment, now.** Squashing the 12 goose migrations is
safe precisely because there is no prod data and the contour DB is wiped. Folding the
new variant labels (`scrabble_ru`/`scrabble_en`/`erudit_ru`) into that single baseline
makes the rename need **no data migration and no back-compat mapping**. Today's labels
(`english`/`russian_scrabble`/`erudit`) are persisted in `games.variant`,
`game_invitations.variant`, in `pkg/fbs` and the UI — ~100 files, but a mechanical sweep
on a clean DB.
- **R4 (TODO 4 + 5): the app is already push-first.** Game state refreshes on
`your_turn`/`opponent_moved`, the lobby on `notify`, chat on `chat_message`. The **only**
genuine periodic server poll is `lobby.poll` (matchmaking, 2.5 s,
`ui/src/screens/NewGame.svelte`). What remains is killing that one poll **and** enriching
push events to carry payloads so the UI stops re-fetching after each signal.
- **R3 (TODO 2): identity forgery is already mitigated.** Identity is always derived from
the session (`Authorization: Bearer``X-User-ID`); the client cannot inject identity,
the backend re-validates resource ownership, Telegram initData is HMAC-checked. The real
gaps are a missing **request-body size limit** (cheap DoS) and **invisible rate-limit
rejections** (no log/metric/admin view — that is TODO 8). Static landing serving is **not**
covered by the gateway token bucket (it only guards `Execute`).
- **R6 (TODO 7) scale:** ~431 `Stage N` references across ~104 files (incl. the file name
`backend/internal/inttest/stage6_test.go`). Code is the source of truth; `docs/` describe
current state; `PLAN.md` keeps the decision history.
## Locked decisions (owner interview)
- **Stress test (TODO 9):** **early + final** runs. Driver = **edge protocol** (Connect/FB
through the gateway, moves generated by the solver) **plus a separate gateway-hammer**
saturation test. Pacing = **realistic (under limits) + saturation (ramp to the knee)**.
Resource metrics = **add cAdvisor + postgres_exporter to the contour** (today only
Go-runtime metrics exist). The harness stays in the repo for repeats.
- **Push (TODO 4 + 5):** **both** — kill `lobby.poll` (use the existing `match_found`, keep
poll as the ws-down fallback) **and** enrich push events with payloads.
- **Refactor (TODO 7):** **hygiene + structural changes by a reviewed list**
behaviour-preserving, test-gated, contentious items surfaced to the owner before applying.
- **Landing (TODO 3):** **separate static container** behind the project caddy
(`/` → landing, `/app/` + `/telegram/` → gateway); drop `landing.html` from the gateway
`go:embed`.
- **Rate-abuse (TODO 8):** metric + Grafana + admin view **plus a conservative auto-flag**
a *soft, reversible* "suspected high-rate" marker for operator review, tunable threshold,
**no auto-ban**.
- **Open auto-match (owner ad-hoc):** a quick game **enters a real game at once and waits inside
it** (status `open`, the opponent seat empty); a second human searching the same variant+rule
joins it, or a robot fills it after a **90 s + random 090 s** wait, pushing the in-app
**opponent_joined** event. While open, the starter may move on their turn but resign, chat and
nudge are disabled, and the lobby + opponent card read "searching for opponent". Matchmaking is
now **DB-backed open games** — the in-memory pool, `lobby.poll` and `lobby.cancel` are gone. The
schema is edited in the baseline (no prod data); `game_players.account_id` is nullable for the
empty seat.
## Phases
Each phase: read this tracker + the relevant `docs/`, **interview the owner on the open
details below**, implement within scope, then update the tracker + docs/code and get CI
green before marking it done.
### R1 — Schema & naming reset *(TODO 1 + 10)* — first
Squash `backend/internal/postgres/migrations/00001..00012` into one `00001_baseline.sql`
(method: `pg_dump --schema-only` from a fully-migrated DB → wrap as the goose baseline →
prove a fresh migrate yields a schema identical to the 12-migration chain via the
integration suite → delete the old files; keep goose). Bake the new variant labels into the
baseline. Propagate `scrabble_ru`/`scrabble_en`/`erudit_ru` through the backend
(`engine.Variant`/`ParseVariant`, `registry.dictFiles`, the CHECK values), the wire
(`pkg/fbs` `variant:string`, regenerate FB) and the UI (`lib/model.ts` union, `variants.ts`,
fixtures, premium/alphabet keys, tests); i18n display keys stay display-only. Tidy
`../scrabble-dictionary` to a single source→dawg build point and align the dawg artifact
names to the new labels (crosses into `../scrabble-solver`'s committed fixtures — keep them
byte-identical). After merge, **wipe the contour DB** (drop the volume) so it re-provisions
on the next deploy.
- Critical files: `backend/internal/postgres/migrations/`,
`backend/internal/engine/{engine,registry}.go`, `pkg/fbs/scrabble.fbs`,
`ui/src/lib/{model,variants}.ts`, `../scrabble-dictionary/{Makefile,cmd/builddict,…}`.
- Open details to interview: the exact dawg filename scheme; whether the dict-repo tidy is
one PR or split; how to script the contour DB wipe in the deploy.
### R2 — Stress harness + contour observability + early run *(TODO 9, part 1)*
Build the reusable load harness as a new `loadtest` module in `go.work` (reuses `pkg/fbs`,
`connect-go`, and `scrabble-solver` for legal-move generation): a seeder that inserts
**1000 guest + 10000 durable** accounts with pre-created sessions (token hashes) directly in
the DB and hands the plaintext tokens to the client; a driver that runs N virtual users,
each in 35 concurrent 24-player games, exercising submit-play / pass / exchange / nudge /
chat / check-word / draft-move / profile-save through the **edge protocol**, in
**realistic** (under rate limits) and **saturation** (ramp) modes; plus a separate
**gateway-hammer** that deliberately exceeds limits to verify the limiter holds and measure
its cost. Add **cAdvisor + postgres_exporter** to `deploy/docker-compose.yml` and a Grafana
resource dashboard. Run the **early pass** against the freshly-wiped contour; produce a
**trip report** (logic/concurrency bugs + a resource baseline) that feeds R3 and R6.
- Critical files: new `loadtest/`, `deploy/docker-compose.yml`, `deploy/observability/*`,
`docs/TESTING.md`.
- Open details: the scale ramp steps; the move-selection policy (a mid-ranked solver move
for realistic game progress); run duration; the pass/fail bar.
### R3 — Edge hardening *(TODO 2 + 8 + 3)*
Add a **request-body size cap** at the gateway h2c mux / `Execute` (e.g. ~1 MB). Add
**rate-limit observability**: a `gateway_rate_limited_total{class}` counter + a structured
log per rejection; an **aggregate** Grafana panel (request rate + rejection rate — spikes
visible without per-user label cardinality, honouring the Stage 12/17 discipline); an
**admin-console view** of recently throttled users/IPs (in-memory ring buffer, single-
instance, reset-on-restart, like the `active_users` gauge). Add the **conservative
auto-flag**: when a user is *sustained*-throttled past a tunable threshold, set a soft,
reversible `account.flagged_high_rate_at` marker (baked into the R1 baseline) surfaced in the
admin user list/detail — **no auto-ban**; the operator clears it. Split the **landing** into
its own static container (`deploy/` + a Caddyfile route `/` → landing) and drop
`landing.html` from the gateway `go:embed`.
- Critical files: `gateway/internal/connectsrv/server.go`, `gateway/internal/ratelimit/`,
`gateway/internal/connectsrv/metrics.go`, `backend/internal/adminconsole/`,
`deploy/caddy/Caddyfile`, `deploy/docker-compose.yml`, `gateway/internal/webui/`.
- Open details: the auto-flag threshold/window + whether the marker is persisted vs
in-memory; the landing image base (caddy vs nginx).
### R4 — Push enrichment + kill the last poll *(TODO 4 + 5)*
Replace `lobby.poll` with the existing `match_found` push (keep the poll as a ws-down
fallback). Enrich `your_turn`/`opponent_moved`/`notify` to carry the state payload so the UI
renders from the event without a follow-up `game.state` (removes the lobby↔game nav latency
the owner noticed). Wire-contract change: `pkg/fbs` event payloads → backend `notify` emit →
UI stream consumers (`ui/src/lib/app.svelte.ts`), with the per-game cache as the landing
spot; regenerate FB.
- Critical files: `pkg/fbs/scrabble.fbs`, `backend/internal/notify/events.go`,
`ui/src/lib/{app.svelte,transport}.ts`, `ui/src/screens/NewGame.svelte`.
- Open details: which events carry full vs delta payloads; the fallback-poll cadence when the
stream is down.
### R5 — Bundle slimming *(TODO 6)* — done
Analysed the bundle against the 100 KB-gzip budget; **no code slimming was warranted**, and the
budget metric was retargeted to measure the app correctly. The build already minifies +
tree-shakes; the dominant cost is the Connect/FlatBuffers transport runtime + generated bindings
+ the Svelte runtime (≈⅔ of `main`'s source is third-party/generated) — irreducible within scope.
**Lazy-loading was rejected**: `bundle-size.mjs` sums every emitted chunk, so code-splitting yields
no total-size win and adds request latency (+N gateway fetches on first navigation to a split
screen). i18n lazy-load was skipped (the catalogs are a sliver of a Svelte-runtime-dominated shared
chunk, and `en` must stay bundled as the `MessageKey` type source + fallback). Instead,
`bundle-size.mjs` now measures **per HTML entry**, with three independent gates on the natural chunk
boundaries — **app entry ≤ 100 KB, the Svelte+i18n shared chunk ≤ 30 KB, the landing's own chunk
≤ 5 KB** — since the app's real payload is its entry chunk plus the shared chunk (≈97 KB), while the
landing (≈24 KB) is reported separately and kept minimal. Same CLI + exit-code contract, so the CI
step is unchanged.
- Critical files: `ui/scripts/bundle-size.mjs`; no app code changed.
### R6 — Refactor + docs reconciliation + de-staging *(TODO 7)* — done
Behaviour-preserving only. Three separable, separately-committed passes: (a) mechanical
**de-staging** — remove `Stage N`/`TODO-N` references from code, comments and service
READMEs (rename `stage6_test.go`); (b) **docs↔code reconciliation** — reconcile
`docs/ARCHITECTURE.md` / `docs/FUNCTIONAL.md`(+`_ru`) against the code-as-truth, fixing drift
and Go Doc comments; (c) **structural changes by a reviewed list** — surface a list of
proposed optimizations / test-suite consolidations to the owner, apply only the approved,
behaviour-preserving, test-gated ones. The full suite + the final stress run (R7) are the
regression gate. Incorporates the early-run (R2) bug fixes not already shipped.
- Open details: the structural-changes list itself (owner-approved before applying); the test
consolidation targets.
### R7 — Final stress run + tuning *(TODO 9, part 2)* — done
Re-run the R2 harness against the final, refactored system on a clean contour; analyse
resource consumption across **all** components (gateway, backend, Postgres, the
metrics/observability stack, docker log volume) and agree the tuning (pool sizes, rate
limits, cache TTLs, container limits, GOMAXPROCS, log levels). Apply the agreed tuning; record
the methodology + results in the repo.
**Stage 18** (prod contour) then proceeds per [`PLAN.md`](PLAN.md).
## Sequencing rationale
`R1` first (cheapest now; everything builds on the final schema/naming and the stress test
must run against it). `R2` builds the harness and runs the **early** pass to surface bugs and
a resource baseline that feed `R3` and `R6`. `R3`/`R4`/`R5` harden and improve the system.
`R6` (de-stage + reconcile + structural) runs near the end so it sweeps settled code once and
benefits from all accumulated bug knowledge. `R7` validates the final system and tunes it.
Then Stage 18.
## Regression-safety discipline (cross-cutting)
- Every phase is a `feature/* → development` PR; CI (`unit` + `integration` + `ui` behind the
`CI / gate` check) must be green before the owner merges; watch the post-merge contour
deploy with `gitea-ci-watch.py`.
- `R6` structural changes are behaviour-preserving, test-gated, and split from the mechanical
sweeps; contentious items are owner-approved first.
- The two stress runs (`R2` early, `R7` final) are the system-level regression gate.
## Verification (per phase)
- `go build ./<module>/...`, `go vet`, `gofmt -l .` clean, `go test -count=1 ./<module>/...`;
UI: `pnpm check && pnpm test:unit && pnpm build`; the integration suite
(`-tags integration`) for DB/schema changes; `docker compose config` for deploy changes;
green CI on the PR + a healthy contour deploy.
- `R1`: prove the squashed baseline yields a schema identical to the 12-migration chain
(integration suite on a fresh DB) **before** deleting the old files.
- `R2`/`R7`: the harness runs end-to-end against the contour; the trip report lists concrete
defects + a resource profile from the Grafana cAdvisor/postgres_exporter panels.
## Refinements logged during implementation
- **R1** (interview + implementation):
- **Variant labels** `english`/`russian_scrabble`/`erudit`**`scrabble_en`/`scrabble_ru`/`erudit_ru`**
across the backend (`engine.Variant.String`/`ParseVariant`; the `games`/`game_invitations` `variant`
CHECK in the baseline; GCG `#lexicon` and the `variant` metric attribute both flow from `String`),
the wire (`pkg/fbs` `variant` is a `string` field — values change with **no FlatBuffers regen**) and
the UI (`model.ts` union, `variants.ts` records, `codec`/`premiums`/mocks/tests, the admin
`dictionary.gohtml`). **Kept:** the Go enum identifiers (`VariantEnglish`…, internal) and the i18n
display keys (`new.english`/`new.russian`/`new.erudit`, display-only). `complaints.variant` stays
free-text (no CHECK, as before).
- **dawg filenames kept descriptive** (`en_sowpods`/`ru_scrabble`/`ru_erudit`) — only the registry's
`Variant` key carries the rename, so `registry.go`, the published `scrabble-solver` fixtures and the
dictionary release artifact are untouched (decouples the three repos).
- **Migrations squashed** 12 → one hand-written `00001_baseline.sql`. Verified by a
`pg_dump --schema-only` diff (the chain vs the baseline are **identical** but for the two intended
variant-CHECK values) plus the green integration suite. **No data migration** (no production data).
- **Done (cross-repo + contour):** the **`scrabble-dictionary` tidy** merged (PR #2) and was re-cut as
the **byte-identical `v1.0.1`** release for clean provenance (the backend stays on `v1.0.0` — same
bytes, no rewire; the backend pulls a version-pinned release artifact, not master). Post-merge the
contour `backend` schema was wiped (`DROP SCHEMA backend CASCADE` + restart, not a volume drop) and
re-migrated to the baseline — verified the new variant CHECK (`scrabble_en/scrabble_ru/erudit_ru`),
`games`=0 and a clean boot.
- **R2** (interview + implementation):
- **Locked decisions:** game assembly via **invitations** (real path, no robots; not direct game-row
inserts); **moderate** ramp **50 → 200 → 500** at 10 min/step; **diagnostic** pass bar (no SLO gate);
run as a **one-shot container on `scrabble-internal`** in this PR.
- **Harness** = new `scrabble/loadtest` module (`use ./loadtest` + a `replace scrabble/gateway` for the
dot-free edge-proto import). It seeds 1000 guest + 10000 durable accounts + sessions **directly in
Postgres** (token hash mirrors `backend/internal/session`), drives players over the **edge protocol**,
generates **mid-ranked legal moves locally** with the embedded `scrabble-solver` by replaying
`game.history` (the edge carries no board — mirrors `engine.ReplayBoard` via the public API), and a
**gateway-hammer**. Compact CLI (`run` / `cleanup`), distroless Dockerfile (DAWGs baked), Go unit tests.
- **Adding the module broke the other images' builds** — backend/gateway/telegram Dockerfiles reduce the
workspace but still referenced `./loadtest` (not in their context); each now also
`-dropuse=./loadtest` (backend/telegram additionally `-dropreplace` the gateway replace). Caught by the
first deploy run; verified by building all four images.
- **Harness payload fixes found by the smoke pass:** the draft DTO's `rack_order` is a string (was sent
as `[]``bad_request`); the display-name validator forbids digits/colons, so the cleanup marker
became a letters-only `Zzloadtest` so `profile.update` resends the seeded name. `chat_not_your_turn` /
`nudge_own_turn` are **by-design** turn gates, correctly exercised.
- **Observability:** added **cAdvisor + postgres_exporter** + the **Scrabble — Resources** dashboard +
two Prometheus jobs. **Finding:** cAdvisor yields only the root cgroup on the contour host (separate
XFS `/var/lib/docker` breaks its layer-ID resolution — the existing galaxy deploy has the same limit),
so per-container CPU/RSS for the early pass was captured via `docker stats`. **R7:** adopt the otelcol
`docker_stats` receiver (already the contrib image) for per-container metrics in Grafana.
- **Early run (2026-06-09):** ramped clean to 500 players, no crash/deadlock, cleanup removed all 11000
accounts. 1.2 M edge calls, 48 870 plays, 2 798 games finished; the per-user limiter held under the
hammer (99.97 % rejected, p99 2 ms). **Top finding:** ~14 % `transport_error` on `game.state` at 500
players, under CPU saturation (backend/gateway/Postgres each ~1 core) and amplified by the harness's
single shared `http2.Transport`; the harness itself peaked at 86 % of a core on the same host, so the
figures are pessimistic. Full trip report in [`../loadtest/REPORT-R2.md`](../loadtest/REPORT-R2.md);
it feeds R3 (h2c `MaxConcurrentStreams`/timeouts, body-size cap), R6 and R7 (per-player transports,
separate hardware, pool/limit sizing).
- **CI:** `./loadtest/...` added to the path filter + vet/build/test; `go.work.sum` carries the new deps.
- **R3** (interview + implementation):
- **Locked decisions:** the flag column lands by **editing the R1 baseline** (+ a contour schema
wipe after merge — no migration chain accrues before prod); auto-flag defaults **1000 rejected /
10 min** (`BACKEND_HIGHRATE_FLAG_THRESHOLD`/`_WINDOW`, rolling window, set-once, operator clears,
no auto-ban); landing image = **caddy:2-alpine**; throttle data flows **gateway → backend** (a
30 s per-key summary POST to the new `/api/v1/internal/ratelimit/report`, the existing trusted
direction) with the episode window + flag rule in the backend (`internal/ratewatch`); rejection
logging = **Warn summary per key per window + Debug per rejection** — a deliberate deviation from
the phase's "structured log per rejection" (the R2 hammer would have logged ~522k lines in
minutes); all three R2-report tails included (explicit h2c sizing, the session-resolve failure
cause at Warn, reviving the admin limiter).
- **Body cap:** `GATEWAY_MAX_BODY_BYTES` (default 1 MiB) as both the Connect per-message read limit
and an `http.MaxBytesReader` wrap of the public mux; an oversized Execute is `resource_exhausted`.
- **Dead config found:** `AdminPerMinute`/`AdminBurst` were never wired — the gateway `/_gm` mount is
now 429-guarded per IP ahead of its Basic-Auth. The caddy-fronted contour path stays unlimited
(stock caddy has no limiter) — an accepted gap, recorded in `docs/ARCHITECTURE.md` §12.
- **Landing split:** a `landing` target in `gateway/Dockerfile` (the UI build stage is shared;
identical compose build args keep it one cached build); the gateway drops `landing.html` from the
embed and 308-redirects `/``/app/`; the contour caddy routes `/app/`, `/telegram/` and the
Connect path to the gateway and the catch-all to the landing container; the CI deploy probe now
checks both `/` (landing) and `/app/` (gateway).
- **Observability:** `gateway_rate_limited_total{class}` (user/public/email/admin, aggregate-only)
+ a rate-vs-rejections panel on the Edge/UX dashboard; the admin console gains the **Throttled**
page (the in-memory episode window, reset-on-restart like `active_users`, plus the flagged-account
queue) and the flag badge / clear action on the user list / card.
- The jet regen also restored the previously missing `game_drafts`/`game_hidden` generated models
(their tables were added after the last jetgen run; no behaviour change).
- **R4** (interview + implementation):
- **Locked decisions:** **delta-first**, not full snapshots — an event carries only the new move and
the UI applies it to its per-game cache, keyed on `move_count` (idempotent + gap-safe: a gap or the
actor's own move falls back to a `game.state` + `game.history` refetch). `match_found` /
`game_started` carry the recipient's **initial `StateView`** (instant lobby→game); the fallback
refetch stays the existing two calls (no merged endpoint); the matchmaking poll runs **only while
the stream is down** (2.5 s); **all** UI-state-changing events carry their payload (incl. lobby `notify`).
- **Enriched events** (`pkg/fbs` trailing fields — backward-compatible, no FB regen of *values*, only
the schema): `opponent_moved` (+`move`/`game`/`bag_len`), `your_turn` (+`move_count`), `match_found`
(+`state`), `game_over` (+`game`), `notify` (+`account`/`invitation`/`state`). The pre-R4
`opponent_moved` scalars (`seat`/`action`/`score`/`total`) stay for wire back-compat, now redundant
with `move`/`game` — slated for the R6 de-stage.
- **Encoding placement:** the `notify` package keeps ownership of the FlatBuffers encoding (a new
`encode.go` mirrors the gateway transcode but reads wire-agnostic `notify.*` input structs +
`engine.MoveRecord`); the game/lobby/social services map their domain types to those structs, so the
wire schema stays out of the domain. **Flagged for R6:** this partly duplicates the gateway encoders
(different source types) — a candidate consolidation.
- **Actor self-fetch killed too** (beyond literal "push"): the `submit_play`/`pass`/`exchange`/`resign`
**response** (`MoveResult`) now returns the actor's refilled rack + bag size, so the mover renders the
next turn from the response — `Game.svelte`'s `commit`/`pass`/`exchange`/`resign` drop their `await load()`.
- **`match_found` enrichment** needs a per-seat initial state: `lobby.GameCreator` gained `InitialState`,
and `game.Service.InitialState` builds the `notify.PlayerState` (rack re-encoded to wire indices, the
variant alphabet embedded for a first-seen variant).
- **UI:** a pure `lib/gamedelta.ts` reducer (`applyMoveDelta` / `applyGameOver` / `seedInitialState`,
unit-tested) advances the cache; `app.svelte` seeds it on `match_found` / `game_started`; `Game.svelte`
applies the delta (falling back to `load()` while composing, on a gap, or on its own move's new rack);
`NewGame.svelte` polls only when `app.streamAlive` is false and guards its teardown so a push-delivered
match is not cancelled.
- **notify (friends/invitations) scope:** the backend carries the full account / invitation payload on the
wire (per "all events → push"); the UI seeds the game cache from `game_started` but keeps its lightweight
**authoritative** badge refresh (`refreshNotifications`, on the rare `notify` event + on foreground) rather
than adding client-side friend/invitation caches — the per-move hot path is fully de-fetched, which was the
goal. Deeper lobby-cache consumption is an easy follow-up.
- **No schema change** (no migration); the contour needs no DB wipe. Tests: `notify` FB round-trips +
`emitMove` delta + the `gamedelta` reducer; the e2e mock now emits the enriched delta.
- **R5** (interview + implementation):
- **No code slimming — by analysis.** A gzip measure + sourcemap attribution of the real `dist` showed
the app bundle is already minified + tree-shaken and dominated by the Connect/FlatBuffers transport
runtime + generated FB/PB bindings (≈⅔ of `main`'s source) and the Svelte runtime — all
third-party/generated, irreducible within R5's scope. App-authored code carries no hand-trimmable fat.
- **Lazy-load rejected** (screens *and* i18n): `bundle-size.mjs` sums every emitted chunk, so
code-splitting moves bytes between chunks for **zero total-size win** while adding request latency (+N
gateway fetches on first navigation to a split screen). i18n lazy-load additionally buys ≤3 KB (en-only
users) at the cost of an async `t()`, and `en` must stay bundled (it is the `MessageKey` type source +
fallback). **Chunk-collapsing rejected** too — keeping the near-static Svelte runtime in its own
cacheable chunk is the recommended practice (an app deploy then re-busts only `main`, not the runtime),
and HTTP/2 makes the extra preload request negligible.
- **Metric retargeted to the app.** The two-entry build (`index.html` app + `landing.html`) makes Rollup
hoist the code shared by both (Svelte runtime + i18n + `aboutContent`) into one preloaded chunk, so the
app actually loads its entry chunk **+ the shared chunk** (≈74 + ≈23 = **≈97 KB**), never `landing.js`
(≈1.6 KB). The old script summed all three chunks (98.8 KB), over-counting the app by `landing.js`.
`bundle-size.mjs` now parses each built HTML for the JS it eagerly loads and gates three parts
independently — **app entry ≤ 100 KB, shared (Svelte+i18n) ≤ 30 KB, landing-own ≤ 5 KB** — reporting the
app total (≈97) and landing total (≈24.5). Same CLI + exit-code contract, so the CI step is unchanged.
- **No app/source/build change** (`App.svelte`, `lib/i18n/`, `vite.config.ts` untouched); no schema
change, no contour wipe. The stale "~82 KB" figure was corrected in `bundle-size.mjs` and `ui/README.md`.
- **R6** (interview + implementation):
- **Locked decisions:** apply **both** wire/code structural changes (**B** + **A**) and **only C1+C2** of
the test consolidation (not C3/C5); strip the `*(Stage N)*` tags from **all current-state docs**
(ARCHITECTURE / FUNCTIONAL+`_ru` / TESTING / UI_DESIGN), keeping PLAN.md / PRERELEASE.md / CLAUDE.md as
history; **split `stage6_test.go`** by domain. The `h2cMaxConcurrentStreams` sizing stays an **R7**
concern (tuning, not behaviour-preserving); the R2 early run forced no code fix, so nothing was carried in.
- **(a) De-staging:** removed the `Stage N` / `TODO-N` / `(RN)` references across code, comments, service
READMEs and the current-state docs, rewording narratives to present tense (no technical content lost).
Renamed the only stage-named identifiers (`registerStage8``registerSocialOps`,
`registerStage11``registerLinkOps`) and split `stage6_test.go` (`TestEmailLoginFlow``email_test.go`;
`TestGuestAutoMatchLeavesNoStats`+`provisionGuest``account_test.go`). De-staged the `.fbs`/`.proto`
comments and regenerated: only the `.proto`-derived Go docstrings (`*_grpc.pb.go`, `push.pb.go`) changed —
flatc strips schema comments, so the FB Go/TS bindings were untouched.
- **(b) Reconciliation:** the docs were accurate (each R-phase baked its own); the one drift was a stale
"guest-reaping deferred (TODO-3)" note in `ARCHITECTURE.md` §3 — guest reaping is implemented, so the
note was replaced with the current behaviour (FUNCTIONAL/TESTING already described it).
- **(c) B — dead `opponent_moved` scalars:** removed `seat/action/score/total` from `OpponentMovedEvent`
(`pkg/fbs/scrabble.fbs` + the `notify` emit + the round-trip test); regenerated FB Go + TS. No reader
used them (the UI codec/mock take `move`/`game`/`bag_len`; the gateway forwards the payload verbatim).
A pre-release wire-slot renumber — free with no prod data, no DB change.
- **(c) A — shared FB builders:** new `scrabble/pkg/wire` holds the single definition of the nested wire
tables (GameView / MoveRecord / StateView / AccountRef / Invitation) shared by the backend `notify`
encoder and the gateway `transcode`; both map their own source types to neutral `wire.*` structs and
delegate. **Honest tradeoff:** the verbose `Start/Add/End` + reverse-prepend boilerplate is now written
once, but the field *set* is still mapped per side, and the new package makes the change net **+~145 LOC**
— a single-source / anti-drift win for the fiddly mechanics rather than a line-count cut. Behaviour-
preserving: the two sides' field sets were verified identical and the round-trip tests pass unchanged.
- **(c) C1+C2 — inttest fixtures:** moved the cross-file service/game fixtures (`newGameService` was used by
10 files) into `backend/internal/inttest/helpers.go`; single-file helpers stay local. Pure relocation.
- **No schema change → no contour DB wipe.** Regression gate: the full unit + integration + UI suites plus
the R7 stress run.
- **R7** (interview + implementation):
- **Locked decisions:** run the harness **same-host** (one-shot container on `scrabble-internal`, capped
`--cpus=3` so the contour keeps spare cores); **apply container limits + `GOMAXPROCS` now** (not just a
prod recommendation); **replace cAdvisor with the otelcol `docker_stats` receiver** (it resolved only the
root cgroup on this host); keep rate-limit / h2c knobs **compiled-in** (change values only if the data
demands — it did not).
- **Harness refinements (pre-run):** each virtual player builds its **own `edge.Client`** (its own h2c
connection for its Subscribe stream + Execute calls) instead of all players sharing one `http2.Transport`
the R2 `transport_error` artifact; and `playTurn` now reports a **finished** game so the player drops it
from rotation. Effect, measured: `game.state` `transport_error` 14 % (R2) → **2.49 %**; `game_finished` on
chat ≈ 3 900 → **35**.
- **Observability:** added the `docker_stats` receiver to `otelcol` (`api_version: "1.44"` — the daemon's
minimum is 1.40; the receiver defaults to 1.25 and crash-looped until pinned), mounted the docker socket
read-only with `group_add` (the contrib image runs as UID 10001), dropped the cAdvisor service + its
Prometheus job, and retargeted the **Scrabble — Resources** dashboard to the docker_stats metric names
(`container_cpu_utilization`/100 == cores). Cross-checked against `docker stats` within sampling error.
- **Profile (final run, 500 players, limits in force):** the **gateway is the binding constraint** — with
one connection per player it bursts into its 2-core cap (the residual 2.49 % `transport_error`); backend
~0.85 core and postgres ~1.4 cores had headroom; **tempo reached its 1 GiB cap**; the backend pool sat at
its `MaxOpenConns=25` cap (28 backends); docker logs were unbounded (~14 MiB / 30 min on the backend at
info). Full write-up in [`../loadtest/REPORT-R7.md`](../loadtest/REPORT-R7.md).
- **Round-2 tuning (owner-agreed, all in `deploy/docker-compose.yml`, no code change):** gateway **2 → 3
cores + `GOMAXPROCS=3`**; tempo memory **1 → 2 GiB**; backend `MAX_OPEN_CONNS` **25 → 40**; a json-file
**log-rotation** default (10m × 3) applied contour-wide via a YAML anchor (level stays info).
backend/postgres kept at 2 cores / 512 MiB (headroom is cheap on the shared host).
- **Validation:** the same gradual ramp on the tuned contour cut `game.state` `transport_error` to **0.72 %**
(gateway ~2 cores, now under the 3-core cap, no throttle; tempo ~1.27 GiB, under 2 GiB). A separate
**burst** run (a single 100 → 500 jump) pegged the gateway at 3 cores (≈296 % sustained, 9.27 % error),
confirming it is **connection-CPU-bound** — a true arrival spike is a **horizontal-scaling** lever, not
more cores per node (recorded in the prod-sizing recommendation).
- **No schema change → no contour DB wipe.** Bake-back: `loadtest/REPORT-R7.md` (new), `loadtest/README.md`,
`docs/TESTING.md`, the telemetry/observability section of `docs/ARCHITECTURE.md`, the repo-layout line in `CLAUDE.md`.
- **UI — Tab-bar navigation redesign** (owner ad-hoc, not on the raw TODO list): drop the hamburger
`Menu.svelte` everywhere (it fought the Telegram-fullscreen layout, where it had to be re-centred).
- **Locked decisions (interview):** the in-Settings sub-nav is a **bottom TabBar with the active tab
highlighted** (icon-only); **Export GCG** moves to the left slot of the move-history header (free in a
finished game, where 🏁 *leave* does not apply); the lobby **⚙️ badge counts incoming friend requests
only** (invitations keep their own lobby section); unread chat is badged on **the score bar and the 💬**.
- **What shipped:** a ⚙️ **Settings hub** (`screens/SettingsHub.svelte`) over the existing
Settings/Profile/Friends/About bodies and an in-game **comms hub** (`game/CommsHub.svelte`) over
chat + dictionary, both with in-place tabs and a fixed back target; the game's menu items relocate into
the open move history (🏁 leave / 📤 export + 💬 comms header) and the player cards (🤝 add-friend); a
shared **TapConfirm** (`components/TapConfirm.svelte`, `lib/tapconfirm.ts`) — tap → fading ✅ → tap —
replaces the Skip/Hint press-and-hold popovers and drives the add-friend confirm. Fixed the move-history
"jump" bug (the slid board is now inert and the stage can't scroll, so a swipe up genuinely closes it).
`Menu.svelte` + `HoldConfirm.svelte` removed.
- **No schema/wire change → no contour DB wipe.** Bake-back: `docs/UI_DESIGN.md`, `docs/FUNCTIONAL.md`
(+`_ru`). Regression gate: UI `check` + unit (`tapconfirm`) + build + bundle budget + e2e (Chromium &
WebKit), all green.
- **UI — Merge Exchange/Pass; drop the dead Tournaments tab** (owner ad-hoc, not on the raw TODO
list): the lobby's 🏆 *Tournaments* tab was an inert `lobby.soon` toast — removed (the lobby is back
to three tabs, matching `docs/FUNCTIONAL.md`). In-game the separate 🥺 *Skip* (pass) tab folds into
the 🔄 tab, now **Exchange/Pass**, whose dialog passes when no tile is selected and exchanges when
tiles are.
- **Decision — a pass is NOT an exchange of zero (verified against the rules + GCG):** the merge is
**UI-only**. Pass and exchange stay distinct game actions end-to-end — wire (`GameActionRequest` vs
`ExchangeRequest`), engine (`ActionPass` vs `ActionExchange`), and the GCG Poslfit dialect (a pass is
a bare `-`, an exchange is `-TILES`). The engine forbids a zero-tile exchange (`ErrNothingToExchange`)
and allows an exchange only with a full rack left in the bag (`ErrNotEnoughTilesToExchange`), while a
pass is always legal — collapsing them would lose a real distinction. The dialog dispatches the
existing `gateway.pass` / `gateway.exchange`.
- **What shipped:** `Lobby.svelte` (tab removed); `Game.svelte` (one 🔄 Exchange/Pass tab no longer
gated on an empty bag; the dialog disables tile selection while the bag is below a full rack
(`bagLen >= RACK_SIZE`), its confirm button reading **Pass without exchanging** / **Exchange N**);
i18n (`game.draw` → Exchange/Pass, new `game.passNoExchange`, dropped `game.skip` /
`lobby.tournaments` / `lobby.soon`). No backend/wire/history/GCG change.
- **No schema/wire change → no contour DB wipe.** Bake-back: `docs/UI_DESIGN.md`, `docs/FUNCTIONAL.md`
(+`_ru`). Regression gate: UI `check` + unit + build + bundle budget + e2e (Chromium & WebKit).
- **AI — Honest AI opponent in quick game** (owner ad-hoc, not on the raw TODO list): a second quick-game
opponent the player *knowingly* chooses, distinct from the disguised robot of the random/open path
(which is kept as-is). New Game's quick-game mode replaces the "auto-match" subtitle with a two-button
selector **🤖 AI / 👤 Random player** (the `.seg`/`.opt` segmented style, AI the default); for AI the
move-clock line reads "Loss after 7 days of inactivity" and the "searching" hint is hidden.
- **Locked decisions (interview):** AI move is **event-driven** (the robot replies the instant the
player's move commits; the 30 s driver is the fallback); AI games **do not touch `account_stats`**
(practice, like guests); the **Stage 5 strength logic is reused unchanged** (`playToWin` 40 % from the
seed + margin band); **no per-move timeout — a 7-day inactivity loss** instead; the 7-day line lives on
the New Game screen (the in-game screen has no move-clock line); chat + nudge **disabled**, word-check
kept, add-friend never drawn, opponent shown as **🤖** everywhere.
- **The 7-day rule reuses the existing per-turn timeout:** an AI game is created with
`turn_timeout_secs = AIInactivityTimeout` (7 days) and the existing timeout sweeper resigns the overdue
seat — since the robot moves at once, only the human is ever on the clock, so the per-turn timeout *is*
the abandon rule (no new column, no new sweeper).
- **One game flag drives everything:** `games.vs_ai` (edited into the R1 baseline — pre-release, so a
contour DB wipe after merge). It is set **only** on AI-started games, so a robot-filled random game keeps
`vs_ai=false` and the disguised opponent is never revealed; the UI derives 🤖 / the gates **from the flag,
never from the opponent account**. New backend path `Matchmaker.StartVsAI` (picks a pooled robot via the
existing `Pick`, creates an **active** seated game via `game.Service.Create`, random seat order) — the AI
request never enters the open pool, so the open-game reaper never touches it. The robot driver gains a
`vs_ai` branch (no sleep, no proactive nudge, zero delay) and a focused `DriveGame`/`TriggerMove` fast
path wired from the game service's after-create/after-commit hook (`SetAITrigger`, a func value so the
game package never imports the robot package). Chat/nudge gated by a new `social` `VsAI` check
(`ErrGameVsAI` → 409 `ai_game`); statistics skipped in `commit` when `vs_ai`.
- **Wire:** `EnqueueRequest` += `vs_ai`, `GameView` += `vs_ai` (trailing FB fields, regenerated Go + TS),
threaded through the backend DTO, the gateway transcode and the `pkg/wire` + `notify` builders.
- **Tests:** `lobby` unit (StartVsAI seats a robot + flags the game; empty pool leaves no game); backend
integration (`ai_game_test.go`: active+seated+vs_ai+7-day clock, robot moves immediately, stats skipped,
7-day timeout resigns the human, chat/nudge rejected); UI codec round-trip (`vs_ai` on enqueue + game
view); e2e (an AI game shows 🤖, no "searching", chat disabled, the dictionary still works) + the
existing quick-match e2e updated to pick **Random player** (the default is now AI).
- **Schema/wire change → a contour DB wipe** after merge (`DROP SCHEMA backend CASCADE` + restart, the
R1/R3 pattern). Bake-back: `docs/ARCHITECTURE.md`, `docs/FUNCTIONAL.md` (+`_ru`), `docs/UI_DESIGN.md`,
`backend/README.md`, Go Doc comments.
- **Post-review refinements (owner, same PR):** (1) the **GCG export labels the robot seat "AI"** rather
than its human-like pool name (`ExportGCG` overrides the name via `accounts.IsRobot`; the in-app 🤖 is
unchanged); (2) honest-AI games **emit no `your_turn`** — the robot replies instantly, so the signal
would arrive with the move and be pointless; `opponent_moved` still advances the UI; (3) the **admin
console surfaces the AI flag** — a **🤖 column** in `/games` and an "AI game" line on the game card
(`GameRow`/`GameDetailView` gain `VsAI`); (4) `games_started_total` / `games_abandoned_total` gain a
**`vs_ai`** attribute and the Grafana *Game domain* dashboard splits started/abandoned into **human**
and **AI** panels.
- **Follow-up (separate PR — strategy deviation):** the robot now plays **≈20%** of opening/midgame moves
*against* its per-game `playToWin` intent (toward the opposite margin band — a winning robot eases off, a
losing one surges ahead), tapering linearly to **0 over the last 14 bag tiles** and **0 once the bag is
empty**, so the endgame follows the chosen strategy strictly while earlier outcomes can swing the human's
way. Deterministic from the seed (`mix(seed,"deviate",moveCount)`), applied to **both** robot paths via
the shared `selectMove`; the per-game intent (and the admin card) is unchanged. Tests: `robot` unit
(taper bounds + monotonicity, never-in-endgame, determinism, ~20% distribution). Bake-back:
`docs/ARCHITECTURE.md` §7, `docs/FUNCTIONAL.md` (+`_ru`), `backend/README.md`, `PLAN.md` Stage 5.
- **GL — Simultaneous quick-game cap** (owner ad-hoc, not on the raw TODO list): a player may hold at
most **10** active quick games; at the cap the lobby greys **New Game** and shows a plain notice
"Вы достигли лимита одновременных партий", both clearing automatically when an active game finishes.
- **Locked decisions (interview):** what counts = active **+** open (searching) quick games, **including
AI** (`vs_ai`); friend games (invitation-linked) **never** count. The backend gate refuses **all** new-game
creation at the cap — `lobby/enqueue` **and** `invitations` — with **409 `game_limit_reached`**; **accepting**
an invitation is never gated, so friend games are capped "from the other end". Delivery = a boolean
**`at_game_limit`** on the existing `games.list` (no per-event payload: a turn change does not move the count,
and the lobby already re-fetches `games.list` on entry + every game event); the first uncached lobby frame
defaults the button **enabled** (the backend gate is the authority).
- **What shipped:** `game.MaxActiveQuickGames` + `Store/Service.CountActiveQuickGames` (active/open seats, no
`game_invitations` row; hidden games still count → a dedicated count, not a filter over the lobby list);
`Server.atGameLimit`/`ensureUnderGameLimit` gating `handleEnqueue` + `handleCreateInvitation`;
`gameListDTO.at_game_limit`; the FB `GameList` trailing `at_game_limit` (regenerated Go + TS) threaded through
the gateway transcode + UI codec; `lib/model` + `lobbycache` snapshot + `Lobby.svelte` (disabled tab + a muted
`.limit` notice); i18n `lobby.limitReached` (en authoritative + ru).
- **Caveat (logged):** the gate is a pre-check, not transaction-atomic — concurrent creates from one account could
momentarily exceed by 12 (harmless soft cap; the UI disables the button regardless). Strict atomicity was judged
a disproportionate diff across the two create paths.
- **No schema change → no contour DB wipe** (only a trailing FB field, no migration). Tests: backend integration
(`game_limit_test.go`: count rule + HTTP gate 409 + accept bypass), server unit (error mapping), gateway
transcode round-trip, UI codec + lobbycache unit, e2e (`gamelimit.spec.ts`). Bake-back: `docs/FUNCTIONAL.md`
(+`_ru`), `docs/ARCHITECTURE.md` §8, `docs/UI_DESIGN.md`, `backend/README.md`.
+2 -3
View File
@@ -22,9 +22,8 @@ supports English Scrabble, Russian Scrabble and Эрудит.
security, cross-service contracts. security, cross-service contracts.
- [`docs/FUNCTIONAL.md`](docs/FUNCTIONAL.md) (+ [`_ru`](docs/FUNCTIONAL_ru.md)) — - [`docs/FUNCTIONAL.md`](docs/FUNCTIONAL.md) (+ [`_ru`](docs/FUNCTIONAL_ru.md)) —
per-domain user stories. per-domain user stories.
- [`docs/TESTING.md`](docs/TESTING.md) — test layers and the per-stage CI gate. - [`docs/TESTING.md`](docs/TESTING.md) — test layers and the CI gate.
- [`PLAN.md`](PLAN.md) — the staged implementation plan and stage tracker. - [`CLAUDE.md`](CLAUDE.md) — project guide and development workflow.
- [`CLAUDE.md`](CLAUDE.md) — project guide and the mandatory per-stage workflow.
## Build & test ## Build & test
+5 -3
View File
@@ -12,7 +12,7 @@
# --- dictionary artifact ----------------------------------------------------- # --- dictionary artifact -----------------------------------------------------
FROM alpine:3.20 AS dawg FROM alpine:3.20 AS dawg
ARG DICT_VERSION=v1.0.0 ARG DICT_VERSION=v1.2.1
RUN apk add --no-cache curl tar RUN apk add --no-cache curl tar
RUN mkdir -p /dawg \ RUN mkdir -p /dawg \
&& curl -fsSL -o /tmp/dawg.tar.gz \ && curl -fsSL -o /tmp/dawg.tar.gz \
@@ -33,14 +33,16 @@ COPY backend ./backend
# Reduce the workspace to what the backend needs: backend + pkg. loadtest and the # Reduce the workspace to what the backend needs: backend + pkg. loadtest and the
# gateway replace it requires are not in this context, so drop both. # gateway replace it requires are not in this context, so drop both.
RUN go work edit -dropuse=./gateway -dropuse=./platform/telegram -dropuse=./loadtest -dropreplace=scrabble/gateway@v0.0.0 RUN go work edit -dropuse=./gateway -dropuse=./platform/telegram -dropuse=./loadtest -dropreplace=scrabble/gateway@v0.0.0
RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -o /out/backend ./backend/cmd/backend # VERSION (the deploy passes the git tag) is stamped into the binary via the linker.
ARG VERSION=dev
RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags "-X scrabble/pkg/version.Version=${VERSION}" -o /out/backend ./backend/cmd/backend
# --- runtime ----------------------------------------------------------------- # --- runtime -----------------------------------------------------------------
FROM gcr.io/distroless/static-debian12:nonroot FROM gcr.io/distroless/static-debian12:nonroot
# Re-declare the build arg in this stage so it labels the seed dictionary. One # Re-declare the build arg in this stage so it labels the seed dictionary. One
# DICT_VERSION drives both the artifact the dawg stage downloads and the version # DICT_VERSION drives both the artifact the dawg stage downloads and the version
# label the binary pins, so the resident version equals the release tag. # label the binary pins, so the resident version equals the release tag.
ARG DICT_VERSION=v1.0.0 ARG DICT_VERSION=v1.2.1
COPY --from=build /out/backend /usr/local/bin/backend COPY --from=build /out/backend /usr/local/bin/backend
# Own the seed dictionary as the nonroot runtime user (UID 65532): a named volume # Own the seed dictionary as the nonroot runtime user (UID 65532): a named volume
# mounted at /opt/dawg inherits this ownership on first use, so the admin console # mounted at /opt/dawg inherits this ownership on first use, so the admin console
+31 -8
View File
@@ -103,9 +103,16 @@ second listener — `internal/pushgrpc`, a gRPC server (`BACKEND_GRPC_ADDR`) str
live events (your-turn, opponent-moved, chat, nudge, match-found, notify) to the live events (your-turn, opponent-moved, chat, nudge, match-found, notify) to the
gateway. The gateway-only `POST /api/v1/internal/push-target` (a user's gateway. The gateway-only `POST /api/v1/internal/push-target` (a user's
Telegram `external_id`, language and `notifications_in_app_only` flag) lets the gateway Telegram `external_id`, language and `notifications_in_app_only` flag) lets the gateway
route out-of-app push to the Telegram connector; the Telegram login route out-of-app push to the Telegram bot over the gateway bot-link; the Telegram login
seeds a new account's language and display name from the launch fields, and the seeds a new account's language and display name from the launch fields, and the
`accounts.notifications_in_app_only` flag (default true). `accounts.notifications_in_app_only` flag (default true).
The gateway-only `POST /api/v1/internal/chat-access` resolves a Telegram identity (the
bot's join-time query) or an account id (a `chat_access_changed` event) to its
**moderated-chat write eligibility**`registered AND NOT suspended AND NOT chat_muted`.
That event is emitted on an admin block/unblock, a `chat_muted` role grant/revoke, or — via
the `account.SuspensionSweeper` started in `cmd/backend` — a temporary block lapsing;
`chat_muted` is an `account.KnownRoles` entry, a chat-only mute distinct from the game
suspension (which dominates it).
`accounts.is_guest` marks an ephemeral guest — a durable row `accounts.is_guest` marks an ephemeral guest — a durable row
with no identity, excluded from statistics. The server-rendered with no identity, excluded from statistics. The server-rendered
**admin console** at `/_gm` (`internal/adminconsole` + `internal/server/handlers_admin_console.go`; **admin console** at `/_gm` (`internal/adminconsole` + `internal/server/handlers_admin_console.go`;
@@ -116,8 +123,9 @@ pipeline, the online **dictionary update** (upload the `scrabble-dawg-vX.Y.Z.tar
archive, preview the per-variant word diff, then install + activate — `internal/dictadmin` + archive, preview the per-variant word diff, then install + activate — `internal/dictadmin` +
`engine.DiffWords` / `Registry.LoadAvailable`, written to per-version subdirectories of the `engine.DiffWords` / `Registry.LoadAvailable`, written to per-version subdirectories of the
`BACKEND_DICT_DIR` volume with the active version persisted in `dictionary_state`), and operator **broadcasts** via a `BACKEND_DICT_DIR` volume with the active version persisted in `dictionary_state`), and operator **broadcasts** via a
backend Telegram-connector client (`internal/connector`, `BACKEND_CONNECTOR_ADDR`) — each backend client (`internal/connector`, `BACKEND_CONNECTOR_ADDR`) that calls the gateway's
broadcast renders through the single bot in an operator-chosen language. There is one bot, **bot-link relay** — each broadcast renders through the bot in an operator-chosen language
and the relay awaits the bot's delivery ack. There is one bot,
so `/internal/push-target` returns the recipient's `preferred_language` as the render so `/internal/push-target` returns the recipient's `preferred_language` as the render
language for out-of-app push; no per-bot routing remains. The console also manages the **advertising banner** (`/_gm/banners` + language for out-of-app push; no per-bot routing remains. The console also manages the **advertising banner** (`/_gm/banners` +
`/_gm/banner-settings`, `internal/ads`): operator campaigns with a percent weight, an optional `/_gm/banner-settings`, `internal/ads`): operator campaigns with a percent weight, an optional
@@ -147,6 +155,13 @@ rejected calls within `BACKEND_HIGHRATE_FLAG_WINDOW` gets the soft, reversible
`accounts.flagged_high_rate_at` marker (set-once; a badge in the user list and a `accounts.flagged_high_rate_at` marker (set-once; a badge in the user list and a
**Clear** action on the user card; never an automatic ban). **Clear** action on the user card; never an automatic ban).
The gateway also syncs its active IP bans (prod-only — see ARCHITECTURE §11) to
`POST /api/v1/internal/bans/sync`; `internal/banview` mirrors them for the console's
**Throttled** page (an **Active IP bans** panel with an **Unban** action) and returns
the operator's pending unbans in the response, which the gateway applies on its next
sync. Like `ratewatch` it is in-memory and resets on restart — the enforced ban lives
in the gateway, not here.
## Package layout ## Package layout
``` ```
@@ -170,8 +185,9 @@ internal/lobby/ # auto-match (DB-backed open games + robot substitution) +
internal/robot/ # human-like robot opponent: account pool, seed-derived strategy, move driver internal/robot/ # human-like robot opponent: account pool, seed-derived strategy, move driver
internal/adminconsole/ # server-rendered admin console (Go templates + embedded CSS, view models), served at /_gm internal/adminconsole/ # server-rendered admin console (Go templates + embedded CSS, view models), served at /_gm
internal/ads/ # advertising banner: campaigns + bilingual messages + display timings, weighted-rotation feed (ActiveSet) internal/ads/ # advertising banner: campaigns + bilingual messages + display timings, weighted-rotation feed (ActiveSet)
internal/connector/ # backend gRPC client to the Telegram connector (operator broadcasts) internal/connector/ # backend gRPC client to the gateway bot-link relay (operator broadcasts)
internal/ratewatch/ # gateway rate-limit reports: episode window for the console + the high-rate auto-flag internal/ratewatch/ # gateway rate-limit reports: episode window for the console + the high-rate auto-flag
internal/banview/ # gateway active-ban mirror: the console's Active IP bans panel + the operator unban backchannel
``` ```
## Configuration (environment) ## Configuration (environment)
@@ -190,7 +206,7 @@ internal/ratewatch/ # gateway rate-limit reports: episode window for the consol
| `BACKEND_OTEL_TRACES_EXPORTER` | `none` | `none`, `stdout` or `otlp` (gRPC; endpoint from the standard `OTEL_EXPORTER_OTLP_*`). | | `BACKEND_OTEL_TRACES_EXPORTER` | `none` | `none`, `stdout` or `otlp` (gRPC; endpoint from the standard `OTEL_EXPORTER_OTLP_*`). |
| `BACKEND_OTEL_METRICS_EXPORTER` | `none` | `none`, `stdout` or `otlp`. | | `BACKEND_OTEL_METRICS_EXPORTER` | `none` | `none`, `stdout` or `otlp`. |
| `BACKEND_DICT_DIR` | — | **Required.** Directory of committed `.dawg` dictionaries. | | `BACKEND_DICT_DIR` | — | **Required.** Directory of committed `.dawg` dictionaries. |
| `BACKEND_DICT_VERSION` | `v1` | Dictionary version new games pin. | | `BACKEND_DICT_VERSION` | `v1` | Version label for the flat dictionary dir. Recorded in a `.seed_version` marker on first boot and authoritative after: on a seeded volume a changed value is ignored (it seeds only a fresh volume) — the seed-drift guard (ARCHITECTURE.md §5). |
| `BACKEND_GAME_TIMEOUT_SWEEP_INTERVAL` | `1m` | How often the turn-timeout sweeper runs. | | `BACKEND_GAME_TIMEOUT_SWEEP_INTERVAL` | `1m` | How often the turn-timeout sweeper runs. |
| `BACKEND_GAME_CACHE_TTL` | `24h` | Idle window before a live game is evicted from cache. | | `BACKEND_GAME_CACHE_TTL` | `24h` | Idle window before a live game is evicted from cache. |
| `BACKEND_LOBBY_ROBOT_WAIT` | `10s` | Auto-match wait before a robot is substituted for a missing human. | | `BACKEND_LOBBY_ROBOT_WAIT` | `10s` | Auto-match wait before a robot is substituted for a missing human. |
@@ -201,7 +217,7 @@ internal/ratewatch/ # gateway rate-limit reports: episode window for the consol
| `BACKEND_SMTP_USERNAME` | — | SMTP user; empty relays without authentication. | | `BACKEND_SMTP_USERNAME` | — | SMTP user; empty relays without authentication. |
| `BACKEND_SMTP_PASSWORD` | — | SMTP password. | | `BACKEND_SMTP_PASSWORD` | — | SMTP password. |
| `BACKEND_SMTP_FROM` | `no-reply@localhost` | Envelope/From address for confirm-codes. | | `BACKEND_SMTP_FROM` | `no-reply@localhost` | Envelope/From address for confirm-codes. |
| `BACKEND_CONNECTOR_ADDR` | — | Telegram connector gRPC address for admin-console operator broadcasts. Empty disables broadcasts. | | `BACKEND_CONNECTOR_ADDR` | — | the gateway bot-link relay gRPC address for admin-console operator broadcasts. Empty disables broadcasts. |
| `BACKEND_GUEST_REAP_INTERVAL` | `1h` | How often the abandoned-guest reaper sweeps. | | `BACKEND_GUEST_REAP_INTERVAL` | `1h` | How often the abandoned-guest reaper sweeps. |
| `BACKEND_GUEST_RETENTION` | `720h` | Account age past which a guest with no game seat is deleted. | | `BACKEND_GUEST_RETENTION` | `720h` | Account age past which a guest with no game seat is deleted. |
| `BACKEND_HIGHRATE_FLAG_THRESHOLD` | `1000` | Gateway-reported rejected calls within the window past which an account is soft-flagged. | | `BACKEND_HIGHRATE_FLAG_THRESHOLD` | `1000` | Gateway-reported rejected calls within the window past which an account is soft-flagged. |
@@ -212,7 +228,7 @@ internal/ratewatch/ # gateway rate-limit reports: episode window for the consol
```sh ```sh
docker run -d --name scrabble-pg -e POSTGRES_PASSWORD=dev -p 5432:5432 postgres:17-alpine docker run -d --name scrabble-pg -e POSTGRES_PASSWORD=dev -p 5432:5432 postgres:17-alpine
# DAWGs: extract the dictionary release artifact (or point at a local scrabble-solver/dawg): # DAWGs: extract the dictionary release artifact (or point at a local scrabble-solver/dawg):
mkdir -p /tmp/dawg && curl -fsSL https://gitea.iliadenisov.ru/developer/scrabble-dictionary/releases/download/v1.0.0/scrabble-dawg-v1.0.0.tar.gz | tar xz -C /tmp/dawg mkdir -p /tmp/dawg && curl -fsSL https://gitea.iliadenisov.ru/developer/scrabble-dictionary/releases/download/v1.2.1/scrabble-dawg-v1.2.1.tar.gz | tar xz -C /tmp/dawg
BACKEND_POSTGRES_DSN='postgres://postgres:dev@localhost:5432/postgres?search_path=backend&sslmode=disable' \ BACKEND_POSTGRES_DSN='postgres://postgres:dev@localhost:5432/postgres?search_path=backend&sslmode=disable' \
BACKEND_DICT_DIR=/tmp/dawg \ BACKEND_DICT_DIR=/tmp/dawg \
GOPRIVATE='gitea.iliadenisov.ru/*' \ GOPRIVATE='gitea.iliadenisov.ru/*' \
@@ -251,7 +267,14 @@ local solver co-development you may add a temporary replace — see `go.work`).
from the [`scrabble-dictionary`](https://gitea.iliadenisov.ru/developer/scrabble-dictionary) from the [`scrabble-dictionary`](https://gitea.iliadenisov.ru/developer/scrabble-dictionary)
repo (one semver per set); the engine loads them by `(variant, dict_version)` from repo (one semver per set); the engine loads them by `(variant, dict_version)` from
`BACKEND_DICT_DIR`. The backend loads them at startup as a hard dependency `BACKEND_DICT_DIR`. The backend loads them at startup as a hard dependency
(a missing dictionary aborts the boot). (a missing dictionary aborts the boot). The flat directory is the seed version,
labelled `BACKEND_DICT_VERSION`; uploaded versions live in `<version>/`
subdirectories the admin console writes and a restart re-loads. Because the DAWGs
carry no embedded version, the first boot records the seed in a `.seed_version`
marker that is authoritative after: on a seeded volume a changed `BACKEND_DICT_VERSION`
is ignored (it seeds only a fresh volume) — the seed-drift guard — so a live contour's
dictionary is changed through the console, never by bumping the build seed
(ARCHITECTURE.md §5).
## Tests ## Tests
+18 -2
View File
@@ -3,8 +3,8 @@
// loads the dictionaries into the engine registry, warms the session cache, // loads the dictionaries into the engine registry, warms the session cache,
// constructs the game domain and starts its turn-timeout sweeper, constructs the // constructs the game domain and starts its turn-timeout sweeper, constructs the
// lobby and social domains, then serves the HTTP listener with the infrastructure // lobby and social domains, then serves the HTTP listener with the infrastructure
// probes and the /api/v1 route-group skeleton. Domain HTTP endpoints are added // probes and the /api/v1 route group, behind which the domains expose their HTTP
// with the gateway in a later stage described in PLAN.md. // endpoints to the gateway.
package main package main
import ( import (
@@ -15,11 +15,13 @@ import (
"syscall" "syscall"
"time" "time"
"github.com/google/uuid"
"go.uber.org/zap" "go.uber.org/zap"
"scrabble/backend/internal/account" "scrabble/backend/internal/account"
"scrabble/backend/internal/accountmerge" "scrabble/backend/internal/accountmerge"
"scrabble/backend/internal/ads" "scrabble/backend/internal/ads"
"scrabble/backend/internal/banview"
"scrabble/backend/internal/config" "scrabble/backend/internal/config"
"scrabble/backend/internal/connector" "scrabble/backend/internal/connector"
"scrabble/backend/internal/engine" "scrabble/backend/internal/engine"
@@ -159,6 +161,15 @@ func run(ctx context.Context, cfg config.Config, logger *zap.Logger) error {
zap.Duration("interval", cfg.GuestReapInterval), zap.Duration("interval", cfg.GuestReapInterval),
zap.Duration("retention", cfg.GuestRetention)) zap.Duration("retention", cfg.GuestRetention))
// Re-evaluate moderated-chat write access when a temporary block self-expires:
// no operator action fires then, so the sweeper emits the chat-access-changed
// event for lapsed blocks and the gateway re-pushes the chat-gate command.
chatSweeper := account.NewSuspensionSweeper(accounts, func(id uuid.UUID) {
hub.Publish(notify.ChatAccessChanged(id))
}, logger)
go chatSweeper.Run(ctx)
logger.Info("suspension expiry sweeper started", zap.Duration("interval", chatSweeper.Interval()))
// Lobby & social domains. Their REST and stream surface lives in the gateway, // Lobby & social domains. Their REST and stream surface lives in the gateway,
// so they are handed to the server (like the route groups) for the handlers. // so they are handed to the server (like the route groups) for the handlers.
mailer := newMailer(cfg.SMTP, logger) mailer := newMailer(cfg.SMTP, logger)
@@ -211,6 +222,10 @@ func run(ctx context.Context, cfg config.Config, logger *zap.Logger) error {
zap.Int("flag_threshold", cfg.RateWatch.FlagThreshold), zap.Int("flag_threshold", cfg.RateWatch.FlagThreshold),
zap.Duration("flag_window", cfg.RateWatch.FlagWindow)) zap.Duration("flag_window", cfg.RateWatch.FlagWindow))
// Ban observability: mirror the gateway's active IP bans for the admin console's
// active-bans panel and collect operator unban requests.
banView := banview.New()
// Advertising-banner domain: campaign rotation feeding the profile.get banner // Advertising-banner domain: campaign rotation feeding the profile.get banner
// block and the banner admin console section. // block and the banner admin console section.
adsSvc := ads.NewService(ads.NewStore(db)) adsSvc := ads.NewService(ads.NewStore(db))
@@ -233,6 +248,7 @@ func run(ctx context.Context, cfg config.Config, logger *zap.Logger) error {
DictDir: cfg.Game.DictDir, DictDir: cfg.Game.DictDir,
Connector: conn, Connector: conn,
RateWatch: rateWatch, RateWatch: rateWatch,
BanView: banView,
Ads: adsSvc, Ads: adsSvc,
Notifier: hub, Notifier: hub,
}) })
+40 -13
View File
@@ -151,14 +151,26 @@ func (s *Store) ProvisionRobot(ctx context.Context, externalID, displayName stri
return modelToAccount(row), nil return modelToAccount(row), nil
} }
// ProvisionTelegram provisions (or finds) the account bound to a Telegram // ProvisionTelegram provisions (or finds) the account bound to a Telegram identity,
// identity. On first contact only, it seeds the new account's preferred language // reporting whether this call created it (first contact). On first contact only, it
// from the Telegram client languageCode (when it maps to a supported language) and // seeds the new account's preferred language from the Telegram client languageCode
// its display name sanitized from firstName (falling back to username, then to a // (when it maps to a supported language) and its display name sanitized from firstName
// generated placeholder when neither yields any letters); an already-existing // (falling back to username, then to a generated placeholder when neither yields any
// account is returned unchanged, so a later profile edit is never overwritten. // letters); an already-existing account is returned unchanged, so a later profile edit
func (s *Store) ProvisionTelegram(ctx context.Context, externalID, languageCode, username, firstName string) (Account, error) { // is never overwritten. The created flag lets the auth handler re-evaluate moderated-
return s.provision(ctx, KindTelegram, externalID, telegramSeed(languageCode, username, firstName)) // chat write access on first registration — the path of a user who joined the chat
// before registering, whom no chat_member event covers.
func (s *Store) ProvisionTelegram(ctx context.Context, externalID, languageCode, username, firstName string) (Account, bool, error) {
// Pre-check whether the identity already exists so the caller can act on first
// contact. A race with a concurrent create only over- or under-reports created for
// that one call, which the idempotent chat-access re-evaluation tolerates.
_, err := s.findByIdentity(ctx, KindTelegram, externalID)
created := errors.Is(err, ErrNotFound)
if err != nil && !created {
return Account{}, false, err
}
acc, err := s.provision(ctx, KindTelegram, externalID, telegramSeed(languageCode, username, firstName))
return acc, created, err
} }
// provision finds the account for (kind, externalID) or creates it with seed, // provision finds the account for (kind, externalID) or creates it with seed,
@@ -195,10 +207,11 @@ type provisionSeed struct {
// telegramSeed derives the create-time seed from Telegram launch fields: a // telegramSeed derives the create-time seed from Telegram launch fields: a
// supported preferred language from languageCode (an ISO-639 code, possibly // supported preferred language from languageCode (an ISO-639 code, possibly
// region-tagged like "ru-RU"), and a display name sanitized from firstName or, // region-tagged like "ru-RU"), and a display name. The name precedence is the real
// failing that, username (sanitizeDisplayName strips disallowed characters to the // name (firstName, sanitized to the editable format) → the @username taken verbatim
// editable format). When neither yields any letters, it falls back to a generated // (already a valid handle, only trimmed and length-capped, never character-stripped)
// placeholder in the seeded language (placeholderDisplayName). // → a generated placeholder in the seeded language (placeholderDisplayName), reached
// only when firstName has no usable letters and no username is set.
func telegramSeed(languageCode, username, firstName string) provisionSeed { func telegramSeed(languageCode, username, firstName string) provisionSeed {
var seed provisionSeed var seed provisionSeed
if lang, _, _ := strings.Cut(strings.ToLower(strings.TrimSpace(languageCode)), "-"); lang == "en" || lang == "ru" { if lang, _, _ := strings.Cut(strings.ToLower(strings.TrimSpace(languageCode)), "-"); lang == "en" || lang == "ru" {
@@ -206,7 +219,13 @@ func telegramSeed(languageCode, username, firstName string) provisionSeed {
} }
name := sanitizeDisplayName(firstName) name := sanitizeDisplayName(firstName)
if name == "" { if name == "" {
name = sanitizeDisplayName(username) // The real name yielded nothing usable: fall back to the @username verbatim
// (Telegram guarantees a valid handle), only trimmed and capped to the column
// width — never character-stripped like the real name.
name = strings.TrimSpace(username)
if r := []rune(name); len(r) > maxDisplayName {
name = strings.TrimRight(string(r[:maxDisplayName]), " ")
}
} }
if name == "" { if name == "" {
name = placeholderDisplayName(seed.preferredLanguage) name = placeholderDisplayName(seed.preferredLanguage)
@@ -303,6 +322,14 @@ func (s *Store) CountAccounts(ctx context.Context) (int, error) {
return int(dest.Count), nil return int(dest.Count), nil
} }
// AccountByIdentity returns the account bound to (kind, externalID), or ErrNotFound
// when none exists. Unlike ProvisionByIdentity it never creates one: the chat-access
// resolver uses it to tell a registered Telegram user (eligible to be granted chat
// write access) from an unregistered one (left muted).
func (s *Store) AccountByIdentity(ctx context.Context, kind, externalID string) (Account, error) {
return s.findByIdentity(ctx, kind, externalID)
}
// findByIdentity joins identities to accounts and returns the matching account, // findByIdentity joins identities to accounts and returns the matching account,
// or ErrNotFound. // or ErrNotFound.
func (s *Store) findByIdentity(ctx context.Context, kind, externalID string) (Account, error) { func (s *Store) findByIdentity(ctx context.Context, kind, externalID string) (Account, error) {
+8 -6
View File
@@ -9,8 +9,9 @@ import (
// TestTelegramSeed covers the pure mapping from Telegram launch fields to the // TestTelegramSeed covers the pure mapping from Telegram launch fields to the
// create-time account seed: supported-language detection (bare and region-tagged), // create-time account seed: supported-language detection (bare and region-tagged),
// the first-name / username display-name precedence, and the sanitization that // the real-name → @username (verbatim) → placeholder display-name precedence, and
// strips disallowed characters (emoji, digits, punctuation) to the editable format. // the sanitization of the real name (emoji, digits, punctuation stripped to the
// editable format). The username, when used, is kept verbatim.
func TestTelegramSeed(t *testing.T) { func TestTelegramSeed(t *testing.T) {
cases := map[string]struct { cases := map[string]struct {
languageCode, username, firstName string languageCode, username, firstName string
@@ -28,6 +29,7 @@ func TestTelegramSeed(t *testing.T) {
"punct to space": {"en", "user", "John❤Doe", "en", "John Doe"}, "punct to space": {"en", "user", "John❤Doe", "en", "John Doe"},
"digits dropped": {"ru", "user", "Маша123", "ru", "Маша"}, "digits dropped": {"ru", "user", "Маша123", "ru", "Маша"},
"garbage to username": {"en", "good", "123!@#", "en", "good"}, "garbage to username": {"en", "good", "123!@#", "en", "good"},
"username verbatim": {"en", "co_ol99", "🎮🎮", "en", "co_ol99"},
} }
for name, tc := range cases { for name, tc := range cases {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
@@ -49,10 +51,10 @@ func TestTelegramSeedPlaceholder(t *testing.T) {
languageCode, username, firstName string languageCode, username, firstName string
wantRe string wantRe string
}{ }{
"en empty": {"en", "", "", `^Player-\d{5}$`}, "en empty": {"en", "", "", `^Player-\d{5}$`},
"ru empty": {"ru", "", "", `^Игрок-\d{5}$`}, "ru empty": {"ru", "", "", `^Игрок-\d{5}$`},
"default en": {"fr", "", "", `^Player-\d{5}$`}, "default en": {"fr", "", "", `^Player-\d{5}$`},
"both garbage": {"ru", "123", "!!!", `^Игрок-\d{5}$`}, "name garbage, no username": {"ru", "", "!!!", `^Игрок-\d{5}$`},
} }
for name, tc := range cases { for name, tc := range cases {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
+9 -1
View File
@@ -24,11 +24,19 @@ const (
// unconditionally, overriding the usual eligibility (a free account with an // unconditionally, overriding the usual eligibility (a free account with an
// empty hint wallet otherwise sees it). See internal/ads. // empty hint wallet otherwise sees it). See internal/ads.
RoleNoBanner = "no_banner" RoleNoBanner = "no_banner"
// RoleChatMuted forbids the account from writing in the moderated Telegram
// discussion chat, without otherwise restricting the game (the chat-only
// counterpart to a full account suspension). It is one input to the chat-access
// gate; an active admin suspension mutes the player regardless, so this role only
// matters for an account that is not suspended. Granting or revoking it re-pushes
// the chat-gate command for a member currently in the chat.
RoleChatMuted = "chat_muted"
) )
// KnownRoles is the set of roles the console may grant or revoke; an operator // KnownRoles is the set of roles the console may grant or revoke; an operator
// cannot assign an unrecognised role. // cannot assign an unrecognised role.
var KnownRoles = []string{RoleFeedbackBanned, RoleNoBanner} var KnownRoles = []string{RoleFeedbackBanned, RoleNoBanner, RoleChatMuted}
// IsKnownRole reports whether role is a recognised account role. // IsKnownRole reports whether role is a recognised account role.
func IsKnownRole(role string) bool { func IsKnownRole(role string) bool {
+25
View File
@@ -161,6 +161,31 @@ func (s *Store) queryCurrentSuspension(ctx context.Context, accountID uuid.UUID,
return modelToSuspension(row), true, nil return modelToSuspension(row), true, nil
} }
// SuspensionsExpiredBetween returns the distinct account ids whose temporary block lapsed in the
// half-open window (since, until]: a non-lifted suspension with a blocked_until in that range. The
// chat-access sweeper uses it to re-evaluate chat write access when a temporary block self-expires,
// since no operator action fires then. An account that still has another active block may be
// included; the eligibility resolver returns the true state, so emitting for it is harmless.
func (s *Store) SuspensionsExpiredBetween(ctx context.Context, since, until time.Time) ([]uuid.UUID, error) {
rows, err := s.db.QueryContext(ctx,
`SELECT DISTINCT account_id FROM backend.account_suspensions
WHERE lifted_at IS NULL AND blocked_until > $1 AND blocked_until <= $2`,
since.UTC(), until.UTC())
if err != nil {
return nil, fmt.Errorf("account: suspensions expired between: %w", err)
}
defer rows.Close()
var out []uuid.UUID
for rows.Next() {
var id uuid.UUID
if err := rows.Scan(&id); err != nil {
return nil, fmt.Errorf("account: scan expired suspension: %w", err)
}
out = append(out, id)
}
return out, rows.Err()
}
// invalidateSuspension drops the account's cached block so the next CurrentSuspension re-reads it. // invalidateSuspension drops the account's cached block so the next CurrentSuspension re-reads it.
// Called after Suspend and LiftSuspension. // Called after Suspend and LiftSuspension.
func (s *Store) invalidateSuspension(accountID uuid.UUID) { func (s *Store) invalidateSuspension(accountID uuid.UUID) {
@@ -0,0 +1,84 @@
package account
import (
"context"
"time"
"github.com/google/uuid"
"go.uber.org/zap"
)
// suspensionSweepInterval is how often the sweeper re-checks for temporary blocks
// that lapsed. A minute is well under the coarsest block grain (operators pick day
// presets) while keeping the query trivial.
const suspensionSweepInterval = time.Minute
// suspensionExpiryQuerier is the slice of the account store the sweeper depends on:
// the accounts whose temporary block lapsed in a window. *Store satisfies it; a fake
// drives the sweeper's unit tests.
type suspensionExpiryQuerier interface {
SuspensionsExpiredBetween(ctx context.Context, since, until time.Time) ([]uuid.UUID, error)
}
// SuspensionSweeper re-evaluates chat write access when a temporary block self-
// expires. No operator action fires on expiry — the suspension gate just re-reads
// the wall clock — so without this a temporarily blocked player would stay muted in
// the moderated discussion chat after their block lapsed. Each tick it finds blocks
// that expired since the previous tick and calls onExpire for the affected accounts;
// onExpire is wired to publish the chat-access-changed event, after which the gateway
// re-resolves the true eligibility. A liberal call (an account that still has another
// active block) is therefore harmless. The window is in-memory, so a block that
// expires while the process is down is not re-granted until the next operator action
// or the player rejoins — an accepted best-effort gap.
type SuspensionSweeper struct {
store suspensionExpiryQuerier
onExpire func(accountID uuid.UUID)
log *zap.Logger
// since is the upper bound of the previous swept window; the next sweep covers
// (since, now]. It advances only on a successful query, so a failed tick retries
// the same window rather than dropping expiries.
since time.Time
}
// NewSuspensionSweeper builds the sweeper over the account store, the per-account
// expiry callback (publishing the chat-access-changed event) and a logger. The first
// window opens at construction time, so blocks that lapsed earlier are not re-emitted.
func NewSuspensionSweeper(store *Store, onExpire func(accountID uuid.UUID), log *zap.Logger) *SuspensionSweeper {
if log == nil {
log = zap.NewNop()
}
return &SuspensionSweeper{store: store, onExpire: onExpire, log: log, since: time.Now().UTC()}
}
// Interval reports the sweep cadence, for the startup log line.
func (w *SuspensionSweeper) Interval() time.Duration { return suspensionSweepInterval }
// Run sweeps every Interval until ctx is cancelled.
func (w *SuspensionSweeper) Run(ctx context.Context) {
ticker := time.NewTicker(suspensionSweepInterval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
w.sweep(ctx)
}
}
}
// sweep emits a chat-access-changed signal for every account whose temporary block
// lapsed in (since, now], then advances the window. On a query error it keeps the
// window so the next tick retries it.
func (w *SuspensionSweeper) sweep(ctx context.Context) {
now := time.Now().UTC()
ids, err := w.store.SuspensionsExpiredBetween(ctx, w.since, now)
if err != nil {
w.log.Warn("suspension expiry sweep failed", zap.Error(err))
return
}
w.since = now
for _, id := range ids {
w.onExpire(id)
}
}
@@ -0,0 +1,80 @@
package account
import (
"context"
"errors"
"testing"
"time"
"github.com/google/uuid"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.uber.org/zap"
)
// fakeExpiryQuerier records the `since` bound of each call and replays a scripted
// result/error per call, so the sweeper's window and dispatch logic is testable
// without a database.
type fakeExpiryQuerier struct {
results [][]uuid.UUID
errs []error
sinces []time.Time
idx int
}
func (f *fakeExpiryQuerier) SuspensionsExpiredBetween(_ context.Context, since, _ time.Time) ([]uuid.UUID, error) {
f.sinces = append(f.sinces, since)
i := f.idx
f.idx++
if i < len(f.errs) && f.errs[i] != nil {
return nil, f.errs[i]
}
if i < len(f.results) {
return f.results[i], nil
}
return nil, nil
}
func newSweeper(store suspensionExpiryQuerier, onExpire func(uuid.UUID)) *SuspensionSweeper {
return &SuspensionSweeper{
store: store,
onExpire: onExpire,
log: zap.NewNop(),
since: time.Now().Add(-time.Minute).UTC(),
}
}
func TestSuspensionSweeperDispatchesAndAdvances(t *testing.T) {
id1, id2 := uuid.New(), uuid.New()
fake := &fakeExpiryQuerier{results: [][]uuid.UUID{{id1, id2}, nil}}
var got []uuid.UUID
w := newSweeper(fake, func(id uuid.UUID) { got = append(got, id) })
first := w.since
w.sweep(context.Background())
assert.Equal(t, []uuid.UUID{id1, id2}, got, "every expired account is dispatched")
assert.True(t, w.since.After(first), "the window advances on success")
// A second sweep opens the next window at the previous upper bound.
prev := w.since
w.sweep(context.Background())
require.Len(t, fake.sinces, 2)
assert.True(t, fake.sinces[1].After(fake.sinces[0]), "consecutive windows are contiguous and forward")
assert.True(t, fake.sinces[1].Equal(prev), "the next window starts at the previous upper bound")
}
func TestSuspensionSweeperKeepsWindowOnError(t *testing.T) {
fake := &fakeExpiryQuerier{errs: []error{errors.New("db down")}}
w := newSweeper(fake, func(uuid.UUID) { t.Fatal("onExpire must not run when the query fails") })
before := w.since
w.sweep(context.Background())
assert.True(t, w.since.Equal(before), "the window is retained on error so the next tick retries it")
}
func TestNewSuspensionSweeperDefaults(t *testing.T) {
w := NewSuspensionSweeper(nil, func(uuid.UUID) {}, nil)
assert.Equal(t, time.Minute, w.Interval())
assert.NotNil(t, w.log, "a nil logger is tolerated")
assert.WithinDuration(t, time.Now().UTC(), w.since, time.Second, "the first window opens at construction time")
}
@@ -5,6 +5,26 @@
list is in-memory and resets on a backend restart. An account sustaining list is in-memory and resets on a backend restart. An account sustaining
{{.FlagThreshold}}+ rejected calls within {{.FlagWindow}} is soft-flagged for review {{.FlagThreshold}}+ rejected calls within {{.FlagWindow}} is soft-flagged for review
below — never banned automatically; clear the flag on the user card.</p> below — never banned automatically; clear the flag on the user card.</p>
<section class="panel"><h2>Active IP bans</h2>
<p class="note">Temporary IP bans the gateway is currently enforcing (in-memory, prod-only;
reset on a gateway restart). Unban applies on the gateway's next sync.</p>
<table class="list">
<thead><tr><th>IP</th><th>Reason</th><th>Since</th><th>Expires</th><th></th></tr></thead>
<tbody>
{{range .Bans}}
<tr>
<td><code>{{.IP}}</code></td>
<td>{{.Reason}}</td>
<td>{{.Since}}</td>
<td>{{.Expires}}</td>
<td><form class="form" method="post" action="/_gm/bans/unban"><input type="hidden" name="ip" value="{{.IP}}"><button type="submit">Unban</button></form></td>
</tr>
{{else}}
<tr><td colspan="5"><span class="note">no active bans</span></td></tr>
{{end}}
</tbody>
</table>
</section>
<section class="panel"><h2>Recent episodes</h2> <section class="panel"><h2>Recent episodes</h2>
<table class="list"> <table class="list">
<thead><tr><th>Class</th><th>Key</th><th class="num">Rejected</th><th>First seen</th><th>Last seen</th></tr></thead> <thead><tr><th>Class</th><th>Key</th><th class="num">Rejected</th><th>First seen</th><th>Last seen</th></tr></thead>
+14 -4
View File
@@ -389,17 +389,27 @@ type BroadcastView struct {
ConnectorEnabled bool ConnectorEnabled bool
} }
// ThrottledView is the rate-limit observability page: the recent gateway-reported // ThrottledView is the rate-limit observability page: the temporary IP bans the
// throttle episodes (in-memory, reset on restart) and the accounts currently // gateway is currently enforcing, the recent gateway-reported throttle episodes
// carrying the high-rate flag. FlagThreshold and FlagWindow caption the active // (in-memory, reset on restart) and the accounts currently carrying the high-rate
// auto-flag tuning. // flag. FlagThreshold and FlagWindow caption the active auto-flag tuning.
type ThrottledView struct { type ThrottledView struct {
Bans []BanRow
Episodes []ThrottleEpisodeRow Episodes []ThrottleEpisodeRow
Flagged []FlaggedAccountRow Flagged []FlaggedAccountRow
FlagThreshold int FlagThreshold int
FlagWindow string FlagWindow string
} }
// BanRow is one temporary IP ban the gateway is enforcing, with its reason and its
// since/expiry timestamps; the row carries an unban action.
type BanRow struct {
IP string
Reason string
Since string
Expires string
}
// ThrottleEpisodeRow is one recently throttled limiter key. UserID links to the // ThrottleEpisodeRow is one recently throttled limiter key. UserID links to the
// user card and is set only for the user class (the other classes key by IP). // user card and is set only for the user class (the other classes key by IP).
type ThrottleEpisodeRow struct { type ThrottleEpisodeRow struct {
+92
View File
@@ -0,0 +1,92 @@
// Package banview mirrors the gateway's active IP bans for the admin console and
// collects operator unban requests for the gateway to apply. Like ratewatch it is
// in-memory, single-instance and resets on a backend restart by design — the
// gateway re-reports its active set on the next sync, and the durable effect (the
// ban itself) lives in the gateway, not here.
package banview
import (
"sort"
"sync"
"time"
)
// Ban is one active IP ban as reported by the gateway.
type Ban struct {
IP string
Reason string
Since time.Time
Expires time.Time
}
// View holds the last-reported active bans and the operator's pending unbans.
type View struct {
now func() time.Time
mu sync.Mutex
bans map[string]Ban // last reported active set, keyed by IP
unban map[string]struct{} // IPs an operator marked for unban
}
// New constructs an empty View.
func New() *View {
return &View{now: time.Now, bans: make(map[string]Ban), unban: make(map[string]struct{})}
}
// Ingest replaces the mirrored active set with the gateway's latest report,
// skipping entries with an empty IP or one that has already expired.
func (v *View) Ingest(active []Ban) {
now := v.now()
v.mu.Lock()
defer v.mu.Unlock()
v.bans = make(map[string]Ban, len(active))
for _, b := range active {
if b.IP == "" || !now.Before(b.Expires) {
continue
}
v.bans[b.IP] = b
}
}
// Recent returns the mirrored active bans, most recently banned first.
func (v *View) Recent() []Ban {
now := v.now()
v.mu.Lock()
defer v.mu.Unlock()
out := make([]Ban, 0, len(v.bans))
for _, b := range v.bans {
if now.Before(b.Expires) {
out = append(out, b)
}
}
sort.Slice(out, func(i, j int) bool { return out[i].Since.After(out[j].Since) })
return out
}
// RequestUnban records an operator request to lift the ban on ip; the gateway
// applies it on its next sync (so the console reflects it within the sync
// interval). An empty ip is ignored.
func (v *View) RequestUnban(ip string) {
if ip == "" {
return
}
v.mu.Lock()
defer v.mu.Unlock()
v.unban[ip] = struct{}{}
}
// DrainUnbans returns and clears the IPs operators have marked for unban since the
// previous drain. It returns nil when there are none.
func (v *View) DrainUnbans() []string {
v.mu.Lock()
defer v.mu.Unlock()
if len(v.unban) == 0 {
return nil
}
out := make([]string, 0, len(v.unban))
for ip := range v.unban {
out = append(out, ip)
}
clear(v.unban)
return out
}
+64
View File
@@ -0,0 +1,64 @@
package banview
import (
"testing"
"time"
)
func viewAt(clk *time.Time) *View {
v := New()
v.now = func() time.Time { return *clk }
return v
}
func TestIngestRecentDropsExpired(t *testing.T) {
clk := time.Date(2026, 6, 21, 12, 0, 0, 0, time.UTC)
v := viewAt(&clk)
v.Ingest([]Ban{
{IP: "1.1.1.1", Reason: "tripwire", Since: clk, Expires: clk.Add(time.Hour)},
{IP: "2.2.2.2", Reason: "rejections", Since: clk.Add(-2 * time.Hour), Expires: clk.Add(-time.Hour)}, // expired
{IP: "", Reason: "x", Since: clk, Expires: clk.Add(time.Hour)}, // empty IP
})
got := v.Recent()
if len(got) != 1 || got[0].IP != "1.1.1.1" || got[0].Reason != "tripwire" {
t.Fatalf("Recent = %+v, want one live ban for 1.1.1.1", got)
}
}
func TestIngestReplaces(t *testing.T) {
clk := time.Date(2026, 6, 21, 12, 0, 0, 0, time.UTC)
v := viewAt(&clk)
v.Ingest([]Ban{{IP: "1.1.1.1", Since: clk, Expires: clk.Add(time.Hour)}})
v.Ingest([]Ban{{IP: "2.2.2.2", Since: clk, Expires: clk.Add(time.Hour)}})
got := v.Recent()
if len(got) != 1 || got[0].IP != "2.2.2.2" {
t.Fatalf("Recent = %+v, want only the latest report (2.2.2.2)", got)
}
}
func TestRecentOrdersBySince(t *testing.T) {
clk := time.Date(2026, 6, 21, 12, 0, 0, 0, time.UTC)
v := viewAt(&clk)
v.Ingest([]Ban{
{IP: "old", Since: clk.Add(-10 * time.Minute), Expires: clk.Add(time.Hour)},
{IP: "new", Since: clk.Add(-1 * time.Minute), Expires: clk.Add(time.Hour)},
})
got := v.Recent()
if len(got) != 2 || got[0].IP != "new" || got[1].IP != "old" {
t.Fatalf("Recent order = %+v, want most recent first", got)
}
}
func TestUnbanRoundTrip(t *testing.T) {
clk := time.Date(2026, 6, 21, 12, 0, 0, 0, time.UTC)
v := viewAt(&clk)
v.RequestUnban("3.3.3.3")
v.RequestUnban("") // ignored
drained := v.DrainUnbans()
if len(drained) != 1 || drained[0] != "3.3.3.3" {
t.Fatalf("DrainUnbans = %v, want [3.3.3.3]", drained)
}
if again := v.DrainUnbans(); again != nil {
t.Fatalf("second DrainUnbans = %v, want nil (cleared)", again)
}
}
+7 -7
View File
@@ -1,10 +1,10 @@
// Package connector is the backend's gRPC client for the Telegram platform // Package connector is the backend's gRPC client for operator broadcasts: a direct
// connector side-service. The admin console uses it to send operator broadcasts: // message to one user, or a post to the game channel. It calls the gateway's
// a direct message to one user, or a post to the game channel, through the single // bot-link relay (which forwards the send to the remote bot over the reverse mTLS
// bot. The connector lives on the trusted internal network, so the connection uses // link and reports back whether it was delivered). The relay lives on the trusted
// insecure (plaintext) transport credentials (docs/ARCHITECTURE.md §12). It mirrors // internal network, so the connection uses insecure (plaintext) transport
// gateway/internal/connector, narrowed to the two broadcast methods the admin // credentials (docs/ARCHITECTURE.md §12). It speaks the Telegram service contract,
// surface needs. // narrowed to the two broadcast methods the admin surface needs.
package connector package connector
import ( import (
+1 -1
View File
@@ -21,7 +21,7 @@ const (
// ActionResign abandons the game. // ActionResign abandons the game.
ActionResign ActionResign
// ActionTimeout is the auto-resignation a missed turn becomes; recorded by // ActionTimeout is the auto-resignation a missed turn becomes; recorded by
// the game domain in a later stage, never produced by the engine itself. // the game domain, never produced by the engine itself.
ActionTimeout ActionTimeout
) )
+1 -1
View File
@@ -10,7 +10,7 @@
// characters (see decode.go and docs/ARCHITECTURE.md §9.1), so archived games // characters (see decode.go and docs/ARCHITECTURE.md §9.1), so archived games
// replay independently of any dictionary. Second, the engine owns rules and // replay independently of any dictionary. Second, the engine owns rules and
// scoring only: turn scheduling, the 24-hour timeout, persistence and transport // scoring only: turn scheduling, the 24-hour timeout, persistence and transport
// belong to the game domain in a later stage. // belong to the game domain.
package engine package engine
import ( import (
+18 -8
View File
@@ -31,7 +31,7 @@ type entry struct {
// Registry holds the dictionaries resident in memory, addressed by variant and // Registry holds the dictionaries resident in memory, addressed by variant and
// dictionary version, and the solvers built over them. Several versions of a // dictionary version, and the solvers built over them. Several versions of a
// variant may be resident at once; a game pins the version it started on. The // variant may be resident at once; a game pins the version it started on. The
// admin reload flow (a later stage) registers a new version through Load. // admin reload flow registers a new version through Load.
// Registry is safe for concurrent use. // Registry is safe for concurrent use.
type Registry struct { type Registry struct {
mu sync.RWMutex mu sync.RWMutex
@@ -70,11 +70,21 @@ func Open(dir, version string, variants ...Variant) (*Registry, error) {
// immediate subdirectory of dir: a subdirectory named V contributes, under // immediate subdirectory of dir: a subdirectory named V contributes, under
// version V, the variants whose committed DAWG it carries. This is the // version V, the variants whose committed DAWG it carries. This is the
// restart-side of the admin dictionary reload — a version reloaded into dir/<V>/ // restart-side of the admin dictionary reload — a version reloaded into dir/<V>/
// at runtime is resident again after a restart. A subdirectory named like the // at runtime is resident again after a restart. The flat dir's version is resolved
// boot version is skipped (the flat dir already is the boot version). A partially // from its .seed_version marker (see resolveSeedVersion): a fresh dir records
// loaded registry is closed before any error is returned. // bootVersion, an already-seeded dir keeps its recorded label and ignores bootVersion,
// so a bumped build seed never relabels live bytes. A subdirectory named like the
// resolved seed version is skipped (the flat dir already is it). A partially loaded
// registry is closed before any error is returned.
func OpenWithVersions(dir, bootVersion string) (*Registry, error) { func OpenWithVersions(dir, bootVersion string) (*Registry, error) {
r, err := Open(dir, bootVersion) // Resolve the flat dir's version from its seed marker first: on an already-seeded
// volume the marker wins and bootVersion is ignored, so a bumped build seed cannot
// relabel live bytes (see resolveSeedVersion).
seed, err := resolveSeedVersion(dir, bootVersion)
if err != nil {
return nil, err
}
r, err := Open(dir, seed)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -84,9 +94,9 @@ func OpenWithVersions(dir, bootVersion string) (*Registry, error) {
return nil, fmt.Errorf("engine: scan dictionary dir %s: %w", dir, err) return nil, fmt.Errorf("engine: scan dictionary dir %s: %w", dir, err)
} }
for _, e := range entries { for _, e := range entries {
// Skip non-directories, the boot version (already loaded as the flat dir) // Skip non-directories, the resolved seed version (already loaded as the flat
// and dot-prefixed directories (the upload staging area, dir/.staging/). // dir) and dot-prefixed directories (the upload staging area, dir/.staging/).
if !e.IsDir() || e.Name() == bootVersion || strings.HasPrefix(e.Name(), ".") { if !e.IsDir() || e.Name() == seed || strings.HasPrefix(e.Name(), ".") {
continue continue
} }
if _, err := r.LoadAvailable(filepath.Join(dir, e.Name()), e.Name()); err != nil { if _, err := r.LoadAvailable(filepath.Join(dir, e.Name()), e.Name()); err != nil {
+98
View File
@@ -5,6 +5,7 @@ import (
"io" "io"
"os" "os"
"path/filepath" "path/filepath"
"strings"
"testing" "testing"
) )
@@ -112,6 +113,103 @@ func TestOpenWithVersionsSkipsDotDirs(t *testing.T) {
} }
} }
// TestOpenWithVersionsRecordsSeedMarker verifies the first boot records the seed
// version in the flat dir's marker, the marker is not mistaken for a version, and a
// reboot at the same seed version succeeds.
func TestOpenWithVersionsRecordsSeedMarker(t *testing.T) {
dir := t.TempDir()
for _, v := range Variants() {
copyDawg(t, testDictDir(), dir, v)
}
reg, err := OpenWithVersions(dir, "v1")
if err != nil {
t.Fatalf("first open: %v", err)
}
if got := reg.Versions(VariantEnglish); len(got) != 1 || got[0] != "v1" {
t.Errorf("versions = %v, want only [v1] (marker not a version)", got)
}
_ = reg.Close()
data, err := os.ReadFile(filepath.Join(dir, seedMarkerFile))
if err != nil {
t.Fatalf("read seed marker: %v", err)
}
if got := strings.TrimSpace(string(data)); got != "v1" {
t.Fatalf("seed marker = %q, want v1", got)
}
reg2, err := OpenWithVersions(dir, "v1")
if err != nil {
t.Fatalf("reboot at same seed: %v", err)
}
_ = reg2.Close()
}
// TestOpenWithVersionsMarkerWinsOverBoot verifies the recorded .seed_version marker
// is authoritative: once a directory is seeded, a different bootVersion
// (BACKEND_DICT_VERSION) is ignored — the flat dir keeps its recorded label — so a
// bumped build seed on a live volume cannot relabel the already-seeded bytes.
func TestOpenWithVersionsMarkerWinsOverBoot(t *testing.T) {
dir := t.TempDir()
for _, v := range Variants() {
copyDawg(t, testDictDir(), dir, v)
}
reg, err := OpenWithVersions(dir, "v1") // seeds the marker = v1
if err != nil {
t.Fatalf("seed open: %v", err)
}
_ = reg.Close()
// Reboot with a bumped boot version: the marker (v1) wins, no error, v2 ignored.
reg2, err := OpenWithVersions(dir, "v2")
if err != nil {
t.Fatalf("reboot with bumped boot version: %v", err)
}
defer func() { _ = reg2.Close() }()
if got := reg2.Versions(VariantEnglish); len(got) != 1 || got[0] != "v1" {
t.Errorf("versions = %v, want [v1] (marker wins, v2 ignored)", got)
}
if _, err := reg2.Solver(VariantEnglish, "v2"); !errors.Is(err, ErrUnknownVersion) {
t.Errorf("v2 must not be resident: got %v", err)
}
data, _ := os.ReadFile(filepath.Join(dir, seedMarkerFile))
if got := strings.TrimSpace(string(data)); got != "v1" {
t.Errorf("marker = %q, want v1 (unchanged)", got)
}
}
// TestOpenWithVersionsBumpedBootKeepsSubdir mirrors the live-contour case: a volume
// seeded as v1 with a v2 subdirectory (uploaded via the console), booted with a bumped
// build seed bootVersion=v2. The marker (v1) wins for the flat dir, and the v2
// subdirectory is still loaded — not skipped as "the boot version" — so both versions
// stay resident. (Skipping it would silently leave only the flat v1 bytes under v2.)
func TestOpenWithVersionsBumpedBootKeepsSubdir(t *testing.T) {
dir := t.TempDir()
for _, v := range Variants() {
copyDawg(t, testDictDir(), dir, v)
}
reg0, err := OpenWithVersions(dir, "v1") // seed marker = v1
if err != nil {
t.Fatalf("seed: %v", err)
}
_ = reg0.Close()
copyDawg(t, testDictDir(), filepath.Join(dir, "v2"), VariantEnglish) // console upload
reg, err := OpenWithVersions(dir, "v2") // bumped build seed
if err != nil {
t.Fatalf("boot v2: %v", err)
}
defer func() { _ = reg.Close() }()
if _, err := reg.Solver(VariantEnglish, "v1"); err != nil {
t.Errorf("flat v1 must stay resident: %v", err)
}
if _, err := reg.Solver(VariantEnglish, "v2"); err != nil {
t.Errorf("v2 subdir must be resident (not skipped): %v", err)
}
}
// TestReloadRegistersNewVersion verifies Load adds a second version to a variant // TestReloadRegistersNewVersion verifies Load adds a second version to a variant
// already resident, moves the latest pointer and keeps the earlier version. // already resident, moves the latest pointer and keeps the earlier version.
func TestReloadRegistersNewVersion(t *testing.T) { func TestReloadRegistersNewVersion(t *testing.T) {
+53
View File
@@ -0,0 +1,53 @@
package engine
import (
"errors"
"fmt"
"os"
"path/filepath"
"strings"
)
// seedMarkerFile names the file, in the flat dictionary directory, that records the
// version the directory was first seeded as. It is dot-prefixed so OpenWithVersions'
// version scan skips it (like the .staging upload area).
const seedMarkerFile = ".seed_version"
// resolveSeedVersion returns the version label the flat dictionary directory is
// addressed by, recording it on first use.
//
// The contour's dictionary lives on a named volume seeded from the image once and
// never re-seeded (deploy/docker-compose.yml). The flat DAWGs carry no embedded
// version, so the version a volume was first seeded as is recorded in a
// .seed_version marker and is **authoritative** from then on:
//
// - fresh directory (no marker): record bootVersion (the build's
// BACKEND_DICT_VERSION) and return it — the seed of a fresh volume;
// - already-seeded directory: return the recorded marker and ignore bootVersion.
//
// So bumping the build seed on a live volume is a harmless no-op (it only takes
// effect on a future fresh volume) instead of relabelling the already-seeded bytes —
// which would void games pinned to the prior label and mis-serve new ones. New games
// still pin the active version (DB-persisted, set by the admin console), which is the
// real way a running contour moves to a new release.
//
// A directory that cannot be written makes the first record fail; that also breaks
// the admin console (which writes version subdirectories here), so the error is
// returned rather than swallowed, matching the package's fail-loud dictionary setup.
func resolveSeedVersion(dir, bootVersion string) (string, error) {
path := filepath.Join(dir, seedMarkerFile)
data, err := os.ReadFile(path)
if err != nil && !errors.Is(err, os.ErrNotExist) {
return "", fmt.Errorf("engine: read dictionary seed marker %s: %w", path, err)
}
if err == nil {
if recorded := strings.TrimSpace(string(data)); recorded != "" {
return recorded, nil
}
// An empty/corrupt marker falls through and is rewritten from bootVersion.
}
if werr := os.WriteFile(path, []byte(bootVersion+"\n"), 0o644); werr != nil {
return "", fmt.Errorf("engine: record dictionary seed marker %s: %w", path, werr)
}
return bootVersion, nil
}
+12 -8
View File
@@ -63,6 +63,7 @@ type gameCache struct {
type cachedGame struct { type cachedGame struct {
game *engine.Game game *engine.Game
seats []Seat
variant string variant string
lastAccess time.Time lastAccess time.Time
} }
@@ -71,24 +72,27 @@ func newGameCache(ttl time.Duration, now func() time.Time) *gameCache {
return &gameCache{entries: make(map[uuid.UUID]*cachedGame), ttl: ttl, now: now} return &gameCache{entries: make(map[uuid.UUID]*cachedGame), ttl: ttl, now: now}
} }
// get returns the live game for id and refreshes its idle timer, or (nil, false). // get returns the live game and its immutable seat list for id and refreshes its idle
func (c *gameCache) get(id uuid.UUID) (*engine.Game, bool) { // timer, or (nil, nil, false). The seats let a read check membership (and label seats)
// without re-loading the game from the store, since seats never change after a game starts.
func (c *gameCache) get(id uuid.UUID) (*engine.Game, []Seat, bool) {
c.mu.Lock() c.mu.Lock()
defer c.mu.Unlock() defer c.mu.Unlock()
e, ok := c.entries[id] e, ok := c.entries[id]
if !ok { if !ok {
return nil, false return nil, nil, false
} }
e.lastAccess = c.now() e.lastAccess = c.now()
return e.game, true return e.game, e.seats, true
} }
// put stores g as the live game for id. variant labels the entry so the active- // put stores g as the live game for id together with its seat list. variant labels the
// games gauge can report counts by variant without inspecting engine internals. // entry so the active-games gauge can report counts by variant without inspecting engine
func (c *gameCache) put(id uuid.UUID, g *engine.Game, variant string) { // internals; seats are the game's immutable seat standings for the membership fast path.
func (c *gameCache) put(id uuid.UUID, g *engine.Game, variant string, seats []Seat) {
c.mu.Lock() c.mu.Lock()
defer c.mu.Unlock() defer c.mu.Unlock()
c.entries[id] = &cachedGame{game: g, variant: variant, lastAccess: c.now()} c.entries[id] = &cachedGame{game: g, seats: seats, variant: variant, lastAccess: c.now()}
} }
// remove drops id from the cache (used on a finished game and after a failed // remove drops id from the cache (used on a finished game and after a failed
+1 -1
View File
@@ -16,5 +16,5 @@
// word-check tool with complaint capture, per-player game state, history and GCG // word-check tool with complaint capture, per-player game state, history and GCG
// export, and the per-game turn-timeout sweeper that auto-resigns an overdue // export, and the per-game turn-timeout sweeper that auto-resigns an overdue
// player (honouring their daily away window). The HTTP surface that fronts these // player (honouring their daily away window). The HTTP surface that fronts these
// operations is added with the gateway in a later stage. // operations is exposed to the gateway.
package game package game
+3 -3
View File
@@ -94,8 +94,8 @@ func TestGameCacheEviction(t *testing.T) {
cur := time.Unix(1_700_000_000, 0) cur := time.Unix(1_700_000_000, 0)
cache := newGameCache(time.Hour, func() time.Time { return cur }) cache := newGameCache(time.Hour, func() time.Time { return cur })
id := uuid.New() id := uuid.New()
cache.put(id, nil, "scrabble_en") cache.put(id, nil, "scrabble_en", nil)
if _, ok := cache.get(id); !ok { if _, _, ok := cache.get(id); !ok {
t.Fatal("game must be resident after put") t.Fatal("game must be resident after put")
} }
cur = cur.Add(30 * time.Minute) cur = cur.Add(30 * time.Minute)
@@ -104,7 +104,7 @@ func TestGameCacheEviction(t *testing.T) {
if n := cache.sweep(); n != 1 { if n := cache.sweep(); n != 1 {
t.Errorf("sweep evicted %d, want 1", n) t.Errorf("sweep evicted %d, want 1", n)
} }
if _, ok := cache.get(id); ok { if _, _, ok := cache.get(id); ok {
t.Error("game must be evicted after idle TTL") t.Error("game must be evicted after idle TTL")
} }
if cache.size() != 0 { if cache.size() != 0 {
+27 -18
View File
@@ -287,12 +287,12 @@ func (svc *Service) Create(ctx context.Context, params CreateParams) (Game, erro
if err := svc.store.CreateGame(ctx, ins, seats, seeding.draws); err != nil { if err := svc.store.CreateGame(ctx, ins, seats, seeding.draws); err != nil {
return Game{}, err return Game{}, err
} }
svc.cache.put(id, g, params.Variant.String())
svc.metrics.recordStarted(ctx, params.Variant, params.VsAI) svc.metrics.recordStarted(ctx, params.Variant, params.VsAI)
created, err := svc.store.GetGame(ctx, id) created, err := svc.store.GetGame(ctx, id)
if err != nil { if err != nil {
return Game{}, err return Game{}, err
} }
svc.cache.put(id, g, params.Variant.String(), created.Seats)
// Honest-AI game seated with a robot: if the robot moves first, reply at once // Honest-AI game seated with a robot: if the robot moves first, reply at once
// (the periodic driver is the fallback). No-op for every human-only game. // (the periodic driver is the fallback). No-op for every human-only game.
svc.triggerAI(created) svc.triggerAI(created)
@@ -890,26 +890,35 @@ func (svc *Service) timeoutGame(ctx context.Context, gameID uuid.UUID, now time.
// EvaluatePlay previews a tentative play for a seated player against the current // EvaluatePlay previews a tentative play for a seated player against the current
// board without committing it: whether it is legal and what it would score. // board without committing it: whether it is legal and what it would score.
func (svc *Service) EvaluatePlay(ctx context.Context, gameID, accountID uuid.UUID, tiles []engine.TileRecord) (EvalResult, error) { func (svc *Service) EvaluatePlay(ctx context.Context, gameID, accountID uuid.UUID, tiles []engine.TileRecord) (EvalResult, error) {
pre, err := svc.store.GetGame(ctx, gameID)
if err != nil {
return EvalResult{}, err
}
if _, ok := pre.seatOf(accountID); !ok {
return EvalResult{}, ErrNotAPlayer
}
if pre.Status == StatusFinished {
return EvalResult{}, ErrFinished
}
unlock := svc.locks.lock(gameID) unlock := svc.locks.lock(gameID)
defer unlock() defer unlock()
g, err := svc.liveGame(ctx, pre)
if err != nil { // Hot path: an active game stays cached — the engine game is mutated in place across
return EvalResult{}, err // moves and evicted only when it finishes — so on a hit the cached live game and its
// immutable seat list answer the membership check and the score with no DB read. This
// preview is fired on every tile placement, the hottest gameplay call at scale.
g, seats, ok := svc.cache.get(gameID)
if !ok {
// Cold path: load and validate from the store, then replay into the cache.
pre, err := svc.store.GetGame(ctx, gameID)
if err != nil {
return EvalResult{}, err
}
if pre.Status == StatusFinished {
return EvalResult{}, ErrFinished
}
if g, err = svc.liveGame(ctx, pre); err != nil {
return EvalResult{}, err
}
seats = pre.Seats
} }
if !seatedIn(seats, accountID) {
return EvalResult{}, ErrNotAPlayer
}
validateStart := time.Now() validateStart := time.Now()
rec, err := g.EvaluatePlay(tiles) rec, err := g.EvaluatePlay(tiles)
svc.metrics.recordValidate(ctx, pre.Variant, validateStart) svc.metrics.recordValidate(ctx, g.Variant(), validateStart)
if err != nil { if err != nil {
if errors.Is(err, engine.ErrIllegalPlay) { if errors.Is(err, engine.ErrIllegalPlay) {
return EvalResult{Valid: false}, nil return EvalResult{Valid: false}, nil
@@ -1359,7 +1368,7 @@ func (svc *Service) ExportGCG(ctx context.Context, gameID uuid.UUID) (string, er
// liveGame returns the live engine.Game for pre, rebuilding it from the journal // liveGame returns the live engine.Game for pre, rebuilding it from the journal
// on a cache miss. Callers must hold the per-game lock. // on a cache miss. Callers must hold the per-game lock.
func (svc *Service) liveGame(ctx context.Context, pre Game) (*engine.Game, error) { func (svc *Service) liveGame(ctx context.Context, pre Game) (*engine.Game, error) {
if g, ok := svc.cache.get(pre.ID); ok { if g, _, ok := svc.cache.get(pre.ID); ok {
return g, nil return g, nil
} }
g, err := svc.replay(ctx, pre) g, err := svc.replay(ctx, pre)
@@ -1374,7 +1383,7 @@ func (svc *Service) liveGame(ctx context.Context, pre Game) (*engine.Game, error
} }
} }
if !g.Over() { if !g.Over() {
svc.cache.put(pre.ID, g, pre.Variant.String()) svc.cache.put(pre.ID, g, pre.Variant.String(), pre.Seats)
} }
return g, nil return g, nil
} }
+23 -17
View File
@@ -355,27 +355,33 @@ func (s *Store) ExpiredOpen(ctx context.Context, now time.Time) ([]OpenGame, err
// GetGame loads the games row joined with its seats (ordered by seat), or // GetGame loads the games row joined with its seats (ordered by seat), or
// ErrNotFound. // ErrNotFound.
func (s *Store) GetGame(ctx context.Context, id uuid.UUID) (Game, error) { func (s *Store) GetGame(ctx context.Context, id uuid.UUID) (Game, error) {
gstmt := postgres.SELECT(table.Games.AllColumns). // One round-trip: the game joined with its seats. A LEFT JOIN keeps a (would-be)
FROM(table.Games). // seatless game returning the game with no seats, exactly as the prior two-query
// version did; ORDER BY seat preserves seat order. The games columns repeat per seat
// row — cheap at 2-4 seats, and one round-trip instead of two, which matters because
// GetGame is the universal "load the game" step on every game operation.
stmt := postgres.SELECT(table.Games.AllColumns, table.GamePlayers.AllColumns).
FROM(table.Games.LEFT_JOIN(table.GamePlayers, table.GamePlayers.GameID.EQ(table.Games.GameID))).
WHERE(table.Games.GameID.EQ(postgres.UUID(id))). WHERE(table.Games.GameID.EQ(postgres.UUID(id))).
LIMIT(1) ORDER_BY(table.GamePlayers.Seat.ASC())
var grow model.Games var rows []struct {
if err := gstmt.QueryContext(ctx, s.db, &grow); err != nil { model.Games
if errors.Is(err, qrm.ErrNoRows) { model.GamePlayers
return Game{}, ErrNotFound }
} if err := stmt.QueryContext(ctx, s.db, &rows); err != nil {
return Game{}, fmt.Errorf("game: get %s: %w", id, err) return Game{}, fmt.Errorf("game: get %s: %w", id, err)
} }
if len(rows) == 0 {
sstmt := postgres.SELECT(table.GamePlayers.AllColumns). return Game{}, ErrNotFound
FROM(table.GamePlayers).
WHERE(table.GamePlayers.GameID.EQ(postgres.UUID(id))).
ORDER_BY(table.GamePlayers.Seat.ASC())
var srows []model.GamePlayers
if err := sstmt.QueryContext(ctx, s.db, &srows); err != nil {
return Game{}, fmt.Errorf("game: get seats %s: %w", id, err)
} }
return projectGame(grow, srows) seats := make([]model.GamePlayers, 0, len(rows))
for i := range rows {
// Skip the phantom all-NULL seat row a LEFT JOIN yields for a seatless game.
if rows[i].GamePlayers.GameID == id {
seats = append(seats, rows[i].GamePlayers)
}
}
return projectGame(rows[0].Games, seats)
} }
// GetGameVariant reads just a game's variant — a cheap single-column lookup the edge uses // GetGameVariant reads just a game's variant — a cheap single-column lookup the edge uses
+13 -1
View File
@@ -105,7 +105,7 @@ const MaxActiveQuickGames = 10
const aiPlayerName = "AI" const aiPlayerName = "AI"
// CreateParams describes a new game. Seats lists the seated accounts in turn // CreateParams describes a new game. Seats lists the seated accounts in turn
// order (seat 0 moves first); lobby/matchmaking assembles it in a later stage. // order (seat 0 moves first); lobby/matchmaking assembles it.
type CreateParams struct { type CreateParams struct {
Variant engine.Variant Variant engine.Variant
Seats []uuid.UUID Seats []uuid.UUID
@@ -184,6 +184,18 @@ func (g Game) seatOf(accountID uuid.UUID) (int, bool) {
return 0, false return 0, false
} }
// seatedIn reports whether accountID holds a seat in seats. It backs the read-side
// membership check against the cached, immutable seat list, so a hot read can skip
// loading the game from the store.
func seatedIn(seats []Seat, accountID uuid.UUID) bool {
for _, s := range seats {
if s.AccountID == accountID {
return true
}
}
return false
}
// MoveResult is the outcome of a committed transition: the decoded move and the // MoveResult is the outcome of a committed transition: the decoded move and the
// post-move game, plus the actor's own refilled rack and the bag size after the draw // post-move game, plus the actor's own refilled rack and the bag size after the draw
// (Rack/BagLen), so the mover renders the next state from the response without a // (Rack/BagLen), so the mover renders the next state from the response without a
+12 -6
View File
@@ -118,10 +118,13 @@ func TestProvisionTelegramSeedsNewAccountOnly(t *testing.T) {
store := account.NewStore(testDB) store := account.NewStore(testDB)
ext := "tg-" + uuid.NewString() ext := "tg-" + uuid.NewString()
acc, err := store.ProvisionTelegram(ctx, ext, "ru-RU", "thehandle", "Иван") acc, created, err := store.ProvisionTelegram(ctx, ext, "ru-RU", "thehandle", "Иван")
if err != nil { if err != nil {
t.Fatalf("provision telegram: %v", err) t.Fatalf("provision telegram: %v", err)
} }
if !created {
t.Error("created = false on first contact, want true")
}
if acc.PreferredLanguage != "ru" { if acc.PreferredLanguage != "ru" {
t.Errorf("PreferredLanguage = %q, want ru", acc.PreferredLanguage) t.Errorf("PreferredLanguage = %q, want ru", acc.PreferredLanguage)
} }
@@ -133,10 +136,13 @@ func TestProvisionTelegramSeedsNewAccountOnly(t *testing.T) {
} }
// A later login with different fields returns the same account, unchanged. // A later login with different fields returns the same account, unchanged.
again, err := store.ProvisionTelegram(ctx, ext, "en", "other", "Other") again, created, err := store.ProvisionTelegram(ctx, ext, "en", "other", "Other")
if err != nil { if err != nil {
t.Fatalf("re-provision telegram: %v", err) t.Fatalf("re-provision telegram: %v", err)
} }
if created {
t.Error("created = true on a repeat login, want false")
}
if again.ID != acc.ID { if again.ID != acc.ID {
t.Errorf("re-provision id = %s, want %s", again.ID, acc.ID) t.Errorf("re-provision id = %s, want %s", again.ID, acc.ID)
} }
@@ -150,7 +156,7 @@ func TestProvisionTelegramSeedsNewAccountOnly(t *testing.T) {
// language CHECK. // language CHECK.
func TestProvisionTelegramUnknownLanguageDefaults(t *testing.T) { func TestProvisionTelegramUnknownLanguageDefaults(t *testing.T) {
ctx := context.Background() ctx := context.Background()
acc, err := account.NewStore(testDB).ProvisionTelegram(ctx, "tg-"+uuid.NewString(), "fr", "", "") acc, _, err := account.NewStore(testDB).ProvisionTelegram(ctx, "tg-"+uuid.NewString(), "fr", "", "")
if err != nil { if err != nil {
t.Fatalf("provision telegram: %v", err) t.Fatalf("provision telegram: %v", err)
} }
@@ -166,7 +172,7 @@ func TestProvisionTelegramUnknownLanguageDefaults(t *testing.T) {
func TestHighRateFlagRoundTrip(t *testing.T) { func TestHighRateFlagRoundTrip(t *testing.T) {
ctx := context.Background() ctx := context.Background()
store := account.NewStore(testDB) store := account.NewStore(testDB)
acc, err := store.ProvisionTelegram(ctx, "tg-"+uuid.NewString(), "en", "", "Player") acc, _, err := store.ProvisionTelegram(ctx, "tg-"+uuid.NewString(), "en", "", "Player")
if err != nil { if err != nil {
t.Fatalf("provision telegram: %v", err) t.Fatalf("provision telegram: %v", err)
} }
@@ -222,7 +228,7 @@ func TestIdentityExternalID(t *testing.T) {
ctx := context.Background() ctx := context.Background()
store := account.NewStore(testDB) store := account.NewStore(testDB)
ext := "tg-" + uuid.NewString() ext := "tg-" + uuid.NewString()
acc, err := store.ProvisionTelegram(ctx, ext, "en", "", "Tg User") acc, _, err := store.ProvisionTelegram(ctx, ext, "en", "", "Tg User")
if err != nil { if err != nil {
t.Fatalf("provision telegram: %v", err) t.Fatalf("provision telegram: %v", err)
} }
@@ -247,7 +253,7 @@ func TestIdentityExternalID(t *testing.T) {
func TestNotificationsInAppOnlyRoundTrip(t *testing.T) { func TestNotificationsInAppOnlyRoundTrip(t *testing.T) {
ctx := context.Background() ctx := context.Background()
store := account.NewStore(testDB) store := account.NewStore(testDB)
acc, err := store.ProvisionTelegram(ctx, "tg-"+uuid.NewString(), "en", "", "Player") acc, _, err := store.ProvisionTelegram(ctx, "tg-"+uuid.NewString(), "en", "", "Player")
if err != nil { if err != nil {
t.Fatalf("provision telegram: %v", err) t.Fatalf("provision telegram: %v", err)
} }
+1 -1
View File
@@ -222,7 +222,7 @@ func TestConsoleGameDetailRobotSchedule(t *testing.T) {
func TestConsoleThrottledViewAndFlagClear(t *testing.T) { func TestConsoleThrottledViewAndFlagClear(t *testing.T) {
ctx := context.Background() ctx := context.Background()
accounts := account.NewStore(testDB) accounts := account.NewStore(testDB)
acc, err := accounts.ProvisionTelegram(ctx, "tg-"+uuid.NewString(), "en", "", "Throttled Player") acc, _, err := accounts.ProvisionTelegram(ctx, "tg-"+uuid.NewString(), "en", "", "Throttled Player")
if err != nil { if err != nil {
t.Fatalf("provision: %v", err) t.Fatalf("provision: %v", err)
} }
@@ -0,0 +1,315 @@
//go:build integration
package inttest
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
"time"
"github.com/google/uuid"
"go.uber.org/zap/zaptest"
"scrabble/backend/internal/account"
"scrabble/backend/internal/notify"
"scrabble/backend/internal/server"
"scrabble/backend/internal/session"
)
// chatAccessBody mirrors the backend's /internal/chat-access JSON for the test.
type chatAccessBody struct {
ExternalID string `json:"external_id"`
Registered bool `json:"registered"`
Eligible bool `json:"eligible"`
}
// chatAccess issues the gateway-internal chat-access query and asserts a 200.
func chatAccess(t *testing.T, srv *server.Server, body string) chatAccessBody {
t.Helper()
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/api/v1/internal/chat-access", strings.NewReader(body))
req.Header.Set("Content-Type", "application/json")
srv.Handler().ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("chat-access %s = %d: %s", body, rec.Code, rec.Body.String())
}
var b chatAccessBody
if err := json.Unmarshal(rec.Body.Bytes(), &b); err != nil {
t.Fatalf("decode chat-access: %v", err)
}
return b
}
// TestChatAccessResolver drives the gateway-internal eligibility resolver over HTTP:
// the registered/suspended/chat_muted truth table by Telegram identity and by account
// id, the suspension dominating the chat_muted role, an unknown identity reported
// unregistered, and an account with no Telegram identity carrying an empty external_id.
func TestChatAccessResolver(t *testing.T) {
ctx := context.Background()
accounts := account.NewStore(testDB)
srv := server.New(":0", server.Deps{Logger: zaptest.NewLogger(t), DB: testDB, Accounts: accounts})
ext := "tg-" + uuid.NewString()
acc, _, err := accounts.ProvisionTelegram(ctx, ext, "en", "", "Chatter")
if err != nil {
t.Fatalf("provision: %v", err)
}
id := acc.ID
byExt := func() chatAccessBody { return chatAccess(t, srv, `{"external_id":"`+ext+`"}`) }
byUser := func() chatAccessBody { return chatAccess(t, srv, `{"user_id":"`+id.String()+`"}`) }
// A registered, unsuspended, unmuted account is eligible by either address, and the
// account-id query resolves back to its Telegram identity.
if b := byExt(); !b.Registered || !b.Eligible || b.ExternalID != ext {
t.Fatalf("fresh by external_id = %+v, want registered+eligible+ext", b)
}
if b := byUser(); !b.Registered || !b.Eligible || b.ExternalID != ext {
t.Fatalf("fresh by user_id = %+v, want registered+eligible+ext", b)
}
// A suspension mutes; a lift restores.
if _, err := accounts.Suspend(ctx, id, nil, "", "", nil); err != nil {
t.Fatalf("suspend: %v", err)
}
if b := byExt(); !b.Registered || b.Eligible {
t.Fatalf("suspended = %+v, want registered but not eligible", b)
}
if err := accounts.LiftSuspension(ctx, id); err != nil {
t.Fatalf("lift: %v", err)
}
if b := byExt(); !b.Eligible {
t.Fatalf("after lift = %+v, want eligible", b)
}
// The chat_muted role mutes independently; a revoke restores.
if err := accounts.GrantRole(ctx, id, account.RoleChatMuted); err != nil {
t.Fatalf("grant chat_muted: %v", err)
}
if b := byExt(); !b.Registered || b.Eligible {
t.Fatalf("chat_muted = %+v, want registered but not eligible", b)
}
// Suspension dominates: while chat_muted is set, lifting a concurrent suspension
// must not re-grant chat (the role still mutes).
if _, err := accounts.Suspend(ctx, id, nil, "", "", nil); err != nil {
t.Fatalf("suspend over mute: %v", err)
}
if b := byExt(); b.Eligible {
t.Fatalf("suspended+muted = %+v, want not eligible", b)
}
if err := accounts.LiftSuspension(ctx, id); err != nil {
t.Fatalf("lift over mute: %v", err)
}
if b := byExt(); b.Eligible {
t.Fatalf("lifted but still muted = %+v, want not eligible", b)
}
if err := accounts.RevokeRole(ctx, id, account.RoleChatMuted); err != nil {
t.Fatalf("revoke chat_muted: %v", err)
}
if b := byExt(); !b.Eligible {
t.Fatalf("after revoke = %+v, want eligible", b)
}
// An unknown Telegram identity is unregistered (and thus left muted).
if b := chatAccess(t, srv, `{"external_id":"tg-missing-`+uuid.NewString()+`"}`); b.Registered || b.Eligible {
t.Fatalf("unknown identity = %+v, want neither registered nor eligible", b)
}
// An account with no Telegram identity (a guest) carries an empty external_id, so
// the gateway has nothing to gate.
guest := provisionGuest(t)
if b := chatAccess(t, srv, `{"user_id":"`+guest.String()+`"}`); b.ExternalID != "" || b.Registered {
t.Fatalf("guest by user_id = %+v, want empty external_id and not registered", b)
}
// A request naming neither address is a bad request.
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/api/v1/internal/chat-access", strings.NewReader(`{}`))
req.Header.Set("Content-Type", "application/json")
srv.Handler().ServeHTTP(rec, req)
if rec.Code != http.StatusBadRequest {
t.Fatalf("empty query = %d, want 400", rec.Code)
}
}
// captureNotifier records every published intent so a test can assert which live
// events a console action emitted.
type captureNotifier struct {
mu sync.Mutex
intents []notify.Intent
}
func (c *captureNotifier) Publish(in ...notify.Intent) {
c.mu.Lock()
defer c.mu.Unlock()
c.intents = append(c.intents, in...)
}
// count returns how many intents of kind addressed to user were captured.
func (c *captureNotifier) count(user uuid.UUID, kind string) int {
c.mu.Lock()
defer c.mu.Unlock()
n := 0
for _, in := range c.intents {
if in.UserID == user && in.Kind == kind {
n++
}
}
return n
}
// TestChatAccessPublishedOnModeration drives the admin console and asserts each
// moderation action that can change chat eligibility — block, unblock, and the
// chat_muted role grant/revoke — emits the chat_access_changed signal the gateway
// turns into a chat-gate command.
func TestChatAccessPublishedOnModeration(t *testing.T) {
notifier := &captureNotifier{}
srv := server.New(":0", server.Deps{
Logger: zaptest.NewLogger(t),
DB: testDB,
Accounts: account.NewStore(testDB),
Games: newGameService(),
Registry: testRegistry,
DictDir: dictDir(),
Notifier: notifier,
})
h := srv.Handler()
id := provisionAccount(t)
base := "http://admin.test/_gm/users/" + id.String()
const origin = "http://admin.test"
steps := []struct {
name, path, body string
want string
}{
{"block", "/block", "duration=permanent", "Blocked"},
{"unblock", "/unblock", "", "Unblocked"},
{"grant chat_muted", "/grant-role", "role=chat_muted", "Role granted"},
{"revoke chat_muted", "/revoke-role", "role=chat_muted", "Role revoked"},
}
for i, s := range steps {
code, body := consoleDo(h, http.MethodPost, base+s.path, s.body, origin)
if code != http.StatusOK || !strings.Contains(body, s.want) {
t.Fatalf("%s = %d, has %q = %v", s.name, code, s.want, strings.Contains(body, s.want))
}
if got := notifier.count(id, notify.KindChatAccessChanged); got != i+1 {
t.Fatalf("after %s: chat_access_changed count = %d, want %d", s.name, got, i+1)
}
}
}
// TestChatAccessPublishedOnFirstRegistration checks that a Telegram first contact
// (the sessions/telegram endpoint creating the account) emits chat_access_changed —
// the re-grant for a user who joined the moderated chat before registering — and that
// a repeat login does not re-emit.
func TestChatAccessPublishedOnFirstRegistration(t *testing.T) {
notifier := &captureNotifier{}
srv := server.New(":0", server.Deps{
Logger: zaptest.NewLogger(t),
DB: testDB,
Accounts: account.NewStore(testDB),
Sessions: session.NewService(session.NewStore(testDB), session.NewCache()),
Notifier: notifier,
})
h := srv.Handler()
ext := "tg-" + uuid.NewString()
post := func() {
rec := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodPost, "/api/v1/internal/sessions/telegram",
strings.NewReader(`{"external_id":"`+ext+`","language_code":"en","first_name":"Reg"}`))
req.Header.Set("Content-Type", "application/json")
h.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("telegram auth = %d: %s", rec.Code, rec.Body.String())
}
}
post()
acc, err := account.NewStore(testDB).AccountByIdentity(context.Background(), account.KindTelegram, ext)
if err != nil {
t.Fatalf("lookup: %v", err)
}
if got := notifier.count(acc.ID, notify.KindChatAccessChanged); got != 1 {
t.Fatalf("first registration: chat_access_changed count = %d, want 1", got)
}
// A repeat login (the account already exists) must not re-emit.
post()
if got := notifier.count(acc.ID, notify.KindChatAccessChanged); got != 1 {
t.Fatalf("repeat login: chat_access_changed count = %d, want still 1", got)
}
}
// TestSuspensionsExpiredBetween checks the sweeper's window query: a non-lifted
// temporary block whose expiry falls in the window is returned, while one outside the
// window, a permanent block, and a lifted block are not.
func TestSuspensionsExpiredBetween(t *testing.T) {
ctx := context.Background()
accounts := account.NewStore(testDB)
// A temporary block whose expiry already lapsed at a known instant.
tempID := provisionAccount(t)
expiry := time.Now().Add(-time.Hour).Truncate(time.Second)
if _, err := accounts.Suspend(ctx, tempID, &expiry, "", "", nil); err != nil {
t.Fatalf("suspend temp: %v", err)
}
contains := func(ids []uuid.UUID, want uuid.UUID) bool {
for _, id := range ids {
if id == want {
return true
}
}
return false
}
// A window straddling the expiry returns the account.
got, err := accounts.SuspensionsExpiredBetween(ctx, expiry.Add(-time.Minute), expiry.Add(time.Minute))
if err != nil {
t.Fatalf("expired between: %v", err)
}
if !contains(got, tempID) {
t.Fatalf("window over expiry missing the lapsed block %s", tempID)
}
// A window entirely after the expiry does not.
got, err = accounts.SuspensionsExpiredBetween(ctx, expiry.Add(time.Minute), expiry.Add(2*time.Minute))
if err != nil {
t.Fatalf("expired between (after): %v", err)
}
if contains(got, tempID) {
t.Fatalf("window after expiry should not return %s", tempID)
}
// A permanent block never appears, even in a wide window.
permID := provisionAccount(t)
if _, err := accounts.Suspend(ctx, permID, nil, "", "", nil); err != nil {
t.Fatalf("suspend perm: %v", err)
}
// A lifted block does not appear either. The block must still be in force when lifted
// (LiftSuspension only lifts in-force blocks), so its expiry is in the future and the
// wide window below still covers it — yet lifted_at excludes it.
liftID := provisionAccount(t)
liftExpiry := time.Now().Add(30 * time.Minute).Truncate(time.Second)
if _, err := accounts.Suspend(ctx, liftID, &liftExpiry, "", "", nil); err != nil {
t.Fatalf("suspend lift: %v", err)
}
if err := accounts.LiftSuspension(ctx, liftID); err != nil {
t.Fatalf("lift: %v", err)
}
wide, err := accounts.SuspensionsExpiredBetween(ctx, time.Now().Add(-2*time.Hour), time.Now().Add(time.Hour))
if err != nil {
t.Fatalf("expired between (wide): %v", err)
}
if contains(wide, permID) {
t.Fatalf("permanent block %s must not be reported as expired", permID)
}
if contains(wide, liftID) {
t.Fatalf("lifted block %s must not be reported as expired", liftID)
}
}
+1 -1
View File
@@ -62,7 +62,7 @@ func TestEmailConfirmFlow(t *testing.T) {
} }
// TestEmailAlreadyTakenByAnotherAccount refuses to bind an email confirmed by a // TestEmailAlreadyTakenByAnotherAccount refuses to bind an email confirmed by a
// different account (merge is a later stage). // different account (combining two accounts is the separate link/merge flow).
func TestEmailAlreadyTakenByAnotherAccount(t *testing.T) { func TestEmailAlreadyTakenByAnotherAccount(t *testing.T) {
ctx := context.Background() ctx := context.Background()
store := account.NewStore(testDB) store := account.NewStore(testDB)
+6
View File
@@ -543,6 +543,12 @@ func TestEvaluatePlayPreview(t *testing.T) {
if bad.Valid { if bad.Valid {
t.Error("disconnected play must be invalid") t.Error("disconnected play must be invalid")
} }
// A non-seated account cannot preview: with the game warm in the live cache, the
// membership check runs against the cached seat list (the hot path that skips GetGame).
if _, err := svc.EvaluatePlay(ctx, g.ID, provisionAccount(t), hint.Tiles); !errors.Is(err, game.ErrNotAPlayer) {
t.Errorf("evaluate by a non-player = %v, want ErrNotAPlayer", err)
}
} }
// TestConcurrentSubmitSerialized confirms the per-game lock lets only one of two // TestConcurrentSubmitSerialized confirms the per-game lock lets only one of two
@@ -38,7 +38,7 @@ func TestSuspensionGate(t *testing.T) {
Accounts: accounts, Accounts: accounts,
}) })
acc, err := accounts.ProvisionTelegram(ctx, "tg-"+uuid.NewString(), "ru", "", "Blocked") acc, _, err := accounts.ProvisionTelegram(ctx, "tg-"+uuid.NewString(), "ru", "", "Blocked")
if err != nil { if err != nil {
t.Fatalf("provision: %v", err) t.Fatalf("provision: %v", err)
} }
+1 -1
View File
@@ -18,7 +18,7 @@ func TestUserListFilter(t *testing.T) {
st := account.NewStore(testDB) st := account.NewStore(testDB)
uniq := uuid.NewString() uniq := uuid.NewString()
human, err := st.ProvisionTelegram(ctx, "tg-"+uniq, "en", "", "Zzqxhuman") human, _, err := st.ProvisionTelegram(ctx, "tg-"+uniq, "en", "", "Zzqxhuman")
if err != nil { if err != nil {
t.Fatalf("provision human: %v", err) t.Fatalf("provision human: %v", err)
} }
+10
View File
@@ -216,6 +216,16 @@ func BannerChanged(userID uuid.UUID) Intent {
return Notification(userID, NotifyBanner) return Notification(userID, NotifyBanner)
} }
// ChatAccessChanged signals that userID's eligibility to write in the moderated
// Telegram discussion chat may have changed (an admin block/unblock, a chat_muted
// grant/revoke, or a temporary block lapsing). It carries no payload: the gateway
// resolves the user's Telegram identity and current eligibility and pushes the
// resulting chat-gate command to the bot. Unlike the lobby notifications it is an
// infra signal — a distinct top-level kind, never an out-of-app rendered message.
func ChatAccessChanged(userID uuid.UUID) Intent {
return Intent{UserID: userID, Kind: KindChatAccessChanged, EventID: eventID()}
}
// eventID returns a best-effort correlation id for one emitted event. // eventID returns a best-effort correlation id for one emitted event.
func eventID() string { func eventID() string {
if id, err := uuid.NewV7(); err == nil { if id, err := uuid.NewV7(); err == nil {
+7
View File
@@ -35,6 +35,13 @@ const (
// KindGameOver announces a finished game to each seated player, driving the // KindGameOver announces a finished game to each seated player, driving the
// out-of-app "game over" push. // out-of-app "game over" push.
KindGameOver = "game_over" KindGameOver = "game_over"
// KindChatAccessChanged signals that a player's eligibility to write in the
// moderated Telegram discussion chat may have changed (an admin block or unblock,
// a chat_muted grant or revoke, or a temporary block lapsing). It carries no
// payload and is never fanned out to in-app clients: the gateway consumes it to
// resolve the player's Telegram identity and current eligibility and push the
// resulting chat-gate command to the bot.
KindChatAccessChanged = "chat_access_changed"
) )
// Notification sub-kinds carried in a KindNotification event payload; the client // Notification sub-kinds carried in a KindNotification event payload; the client
+131
View File
@@ -0,0 +1,131 @@
package server
import (
"context"
"errors"
"net/http"
"github.com/gin-gonic/gin"
"github.com/google/uuid"
"scrabble/backend/internal/account"
"scrabble/backend/internal/notify"
)
// chatAccessRequest is the gateway's chat write-eligibility query, addressed either
// by Telegram identity (ExternalID — the join path, when the bot sees a user enter
// the chat) or by account id (UserID — the change path, resolving an emitted
// chat-access-changed event). Exactly one field is set.
type chatAccessRequest struct {
ExternalID string `json:"external_id"`
UserID string `json:"user_id"`
}
// chatAccessResponse is the resolved eligibility. ExternalID echoes the account's
// Telegram identity (empty when it has none — the gateway then has nothing to gate);
// Registered reports whether the lookup found an account at all; Eligible is the
// final gate the bot applies (registered and neither admin-suspended nor chat-muted).
type chatAccessResponse struct {
ExternalID string `json:"external_id"`
Registered bool `json:"registered"`
Eligible bool `json:"eligible"`
}
// handleChatAccess resolves whether a Telegram user may write in the moderated
// discussion chat. It is gateway-internal: the gateway's bot-link serves the bot's
// join-time query (by external_id) and resolves an emitted chat-access-changed event
// (by user_id) through it.
func (s *Server) handleChatAccess(c *gin.Context) {
var req chatAccessRequest
if err := c.ShouldBindJSON(&req); err != nil {
abortBadRequest(c, "invalid body")
return
}
switch {
case req.ExternalID != "":
s.respondChatAccessByExternalID(c, req.ExternalID)
case req.UserID != "":
s.respondChatAccessByUserID(c, req.UserID)
default:
abortBadRequest(c, "external_id or user_id required")
}
}
// respondChatAccessByExternalID answers the join-path query: an unknown identity is
// reported unregistered (and left muted); a known one carries its current eligibility.
func (s *Server) respondChatAccessByExternalID(c *gin.Context, externalID string) {
ctx := c.Request.Context()
resp := chatAccessResponse{ExternalID: externalID}
acc, err := s.accounts.AccountByIdentity(ctx, account.KindTelegram, externalID)
if errors.Is(err, account.ErrNotFound) {
c.JSON(http.StatusOK, resp)
return
}
if err != nil {
s.abortErr(c, err)
return
}
resp.Registered = true
eligible, err := s.chatEligible(ctx, acc.ID)
if err != nil {
s.abortErr(c, err)
return
}
resp.Eligible = eligible
c.JSON(http.StatusOK, resp)
}
// respondChatAccessByUserID answers the change-path query: an account with no
// Telegram identity carries an empty external_id (nothing for the gateway to gate);
// otherwise it carries the identity and the current eligibility.
func (s *Server) respondChatAccessByUserID(c *gin.Context, raw string) {
ctx := c.Request.Context()
uid, err := uuid.Parse(raw)
if err != nil {
abortBadRequest(c, "invalid user_id")
return
}
var resp chatAccessResponse
ext, err := s.accounts.IdentityExternalID(ctx, uid, account.KindTelegram)
if errors.Is(err, account.ErrNotFound) {
c.JSON(http.StatusOK, resp)
return
}
if err != nil {
s.abortErr(c, err)
return
}
resp.ExternalID = ext
resp.Registered = true
eligible, err := s.chatEligible(ctx, uid)
if err != nil {
s.abortErr(c, err)
return
}
resp.Eligible = eligible
c.JSON(http.StatusOK, resp)
}
// chatEligible reports whether the account may write in the moderated discussion
// chat: not currently admin-suspended and not holding the chat_muted role. A
// suspension dominates — it mutes regardless of the role. Registration is established
// by the caller's identity lookup.
func (s *Server) chatEligible(ctx context.Context, accountID uuid.UUID) (bool, error) {
if _, blocked, err := s.accounts.CurrentSuspension(ctx, accountID); err != nil {
return false, err
} else if blocked {
return false, nil
}
muted, err := s.accounts.HasRole(ctx, accountID, account.RoleChatMuted)
if err != nil {
return false, err
}
return !muted, nil
}
// publishChatAccessChange emits the chat-access-changed signal for the account, so
// the gateway re-resolves the player's chat eligibility and pushes the chat-gate
// command to the bot. Best-effort (notify.Nop when no notifier is wired).
func (s *Server) publishChatAccessChange(id uuid.UUID) {
s.notifier.Publish(notify.ChatAccessChanged(id))
}
+12
View File
@@ -37,11 +37,23 @@ func (s *Server) registerRoutes() {
// before delivering an out-of-app notification. // before delivering an out-of-app notification.
in.POST("/push-target", s.handlePushTarget) in.POST("/push-target", s.handlePushTarget)
} }
if s.accounts != nil {
// Moderated-chat write eligibility for the Telegram bot: resolve a Telegram
// identity (the bot's join-time query) or an account id (a chat-access-changed
// event) to whether the user may write in the discussion chat. It needs only the
// account store, not the session service, so it registers independently.
s.internal.POST("/chat-access", s.handleChatAccess)
}
if s.ratewatch != nil { if s.ratewatch != nil {
// The gateway's periodic rate-limiter rejection summary: feeds the // The gateway's periodic rate-limiter rejection summary: feeds the
// admin console's throttled view and the high-rate auto-flag. // admin console's throttled view and the high-rate auto-flag.
s.internal.POST("/ratelimit/report", s.handleRateLimitReport) s.internal.POST("/ratelimit/report", s.handleRateLimitReport)
} }
if s.banview != nil {
// The gateway's periodic active-ban sync: feeds the admin console's
// active-bans panel and returns the operator's pending unbans.
s.internal.POST("/bans/sync", s.handleBanSync)
}
u := s.user u := s.user
if s.accounts != nil { if s.accounts != nil {
u.GET("/profile", s.handleProfile) u.GET("/profile", s.handleProfile)
@@ -66,6 +66,7 @@ func (s *Server) registerConsole(router *gin.Engine) {
gm.POST("/reasons/:id/update", s.consoleUpdateReason) gm.POST("/reasons/:id/update", s.consoleUpdateReason)
gm.POST("/reasons/:id/delete", s.consoleDeleteReason) gm.POST("/reasons/:id/delete", s.consoleDeleteReason)
gm.GET("/throttled", s.consoleThrottled) gm.GET("/throttled", s.consoleThrottled)
gm.POST("/bans/unban", s.consoleUnban)
gm.GET("/games", s.consoleGames) gm.GET("/games", s.consoleGames)
gm.GET("/games/:id", s.consoleGameDetail) gm.GET("/games/:id", s.consoleGameDetail)
gm.GET("/complaints", s.consoleComplaints) gm.GET("/complaints", s.consoleComplaints)
@@ -874,6 +875,13 @@ func (s *Server) consoleThrottled(c *gin.Context) {
view.Episodes = append(view.Episodes, row) view.Episodes = append(view.Episodes, row)
} }
} }
if s.banview != nil {
for _, b := range s.banview.Recent() {
view.Bans = append(view.Bans, adminconsole.BanRow{
IP: b.IP, Reason: b.Reason, Since: fmtTime(b.Since), Expires: fmtTime(b.Expires),
})
}
}
flagged, err := s.accounts.ListFlaggedHighRate(ctx) flagged, err := s.accounts.ListFlaggedHighRate(ctx)
if err != nil { if err != nil {
s.consoleError(c, err) s.consoleError(c, err)
@@ -887,6 +895,21 @@ func (s *Server) consoleThrottled(c *gin.Context) {
s.renderConsole(c, "throttled", "throttled", "Throttled", view) s.renderConsole(c, "throttled", "throttled", "Throttled", view)
} }
// consoleUnban lifts a temporary IP ban — the operator's manual override. The
// gateway applies it on its next active-ban sync, so the ban clears within the
// sync interval rather than immediately.
func (s *Server) consoleUnban(c *gin.Context) {
ip := trimForm(c, "ip")
if ip == "" {
s.renderConsoleMessage(c, "Invalid", "an IP address is required", "/_gm/throttled")
return
}
if s.banview != nil {
s.banview.RequestUnban(ip)
}
s.renderConsoleMessage(c, "Unban requested", fmt.Sprintf("%s will be unbanned on the next gateway sync", ip), "/_gm/throttled")
}
// consoleClearHighRateFlag clears the soft high-rate marker — the operator's // consoleClearHighRateFlag clears the soft high-rate marker — the operator's
// reversible review action. // reversible review action.
func (s *Server) consoleClearHighRateFlag(c *gin.Context) { func (s *Server) consoleClearHighRateFlag(c *gin.Context) {
@@ -964,6 +987,9 @@ func (s *Server) consoleBlockUser(c *gin.Context) {
s.consoleError(c, err) s.consoleError(c, err)
return return
} }
// Re-evaluate the player's moderated-chat write access: a block mutes them in
// the discussion chat if they are currently in it.
s.publishChatAccessChange(id)
s.renderConsoleMessage(c, "Blocked", fmt.Sprintf("account blocked; %d game(s) forfeited", forfeited), back) s.renderConsoleMessage(c, "Blocked", fmt.Sprintf("account blocked; %d game(s) forfeited", forfeited), back)
} }
@@ -978,6 +1004,9 @@ func (s *Server) consoleUnblockUser(c *gin.Context) {
s.consoleError(c, err) s.consoleError(c, err)
return return
} }
// Re-evaluate the player's moderated-chat write access: an unblock restores it
// (unless they are still chat-muted) for a member currently in the chat.
s.publishChatAccessChange(id)
s.renderConsoleMessage(c, "Unblocked", "the block was lifted; lost games are not restored", "/_gm/users/"+id.String()) s.renderConsoleMessage(c, "Unblocked", "the block was lifted; lost games are not restored", "/_gm/users/"+id.String())
} }
@@ -248,6 +248,9 @@ func (s *Server) consoleGrantRole(c *gin.Context) {
if role == account.RoleNoBanner { if role == account.RoleNoBanner {
s.publishBannerChange(id) s.publishBannerChange(id)
} }
if role == account.RoleChatMuted {
s.publishChatAccessChange(id)
}
s.renderConsoleMessage(c, "Role granted", "granted "+role, back) s.renderConsoleMessage(c, "Role granted", "granted "+role, back)
} }
@@ -270,6 +273,9 @@ func (s *Server) consoleRevokeRole(c *gin.Context) {
if role == account.RoleNoBanner { if role == account.RoleNoBanner {
s.publishBannerChange(id) s.publishBannerChange(id)
} }
if role == account.RoleChatMuted {
s.publishChatAccessChange(id)
}
s.renderConsoleMessage(c, "Role revoked", "revoked "+role, back) s.renderConsoleMessage(c, "Role revoked", "revoked "+role, back)
} }
+7 -1
View File
@@ -35,11 +35,17 @@ func (s *Server) handleTelegramAuth(c *gin.Context) {
abortBadRequest(c, "external_id is required") abortBadRequest(c, "external_id is required")
return return
} }
acc, err := s.accounts.ProvisionTelegram(c.Request.Context(), req.ExternalID, req.LanguageCode, req.Username, req.FirstName) acc, created, err := s.accounts.ProvisionTelegram(c.Request.Context(), req.ExternalID, req.LanguageCode, req.Username, req.FirstName)
if err != nil { if err != nil {
s.abortErr(c, err) s.abortErr(c, err)
return return
} }
if created {
// First registration: re-evaluate moderated-chat write access, so a user who
// joined the chat before registering is granted on the spot (no chat_member
// event fires on registration).
s.publishChatAccessChange(acc.ID)
}
s.mintSession(c, acc) s.mintSession(c, acc)
} }
+48
View File
@@ -0,0 +1,48 @@
package server
import (
"net/http"
"time"
"github.com/gin-gonic/gin"
"scrabble/backend/internal/banview"
)
// banSyncRequest mirrors the gateway's active-ban report: every entry is one
// currently-enforced IP ban.
type banSyncRequest struct {
Active []banSyncEntry `json:"active"`
}
// banSyncEntry is one active ban in the sync request.
type banSyncEntry struct {
IP string `json:"ip"`
Reason string `json:"reason"`
Since time.Time `json:"since"`
Expires time.Time `json:"expires"`
}
// banSyncResponse returns the IPs an operator has marked for unban for the gateway
// to apply on its next sync.
type banSyncResponse struct {
Unban []string `json:"unban"`
}
// handleBanSync ingests the gateway's active-ban report into the ban view (the
// admin console's active-bans panel) and returns the operator's pending unbans.
// Internal, gateway-only: like the rate-limit report it trusts the network
// segment and carries no user identity.
func (s *Server) handleBanSync(c *gin.Context) {
var req banSyncRequest
if err := c.ShouldBindJSON(&req); err != nil {
abortBadRequest(c, "invalid ban sync")
return
}
bans := make([]banview.Ban, 0, len(req.Active))
for _, e := range req.Active {
bans = append(bans, banview.Ban{IP: e.IP, Reason: e.Reason, Since: e.Since, Expires: e.Expires})
}
s.banview.Ingest(bans)
c.JSON(http.StatusOK, banSyncResponse{Unban: s.banview.DrainUnbans()})
}
+12 -5
View File
@@ -1,10 +1,10 @@
// Package server wires the backend's HTTP listener: the gin engine, its route // Package server wires the backend's HTTP listener: the gin engine, its route
// groups, the per-request telemetry middleware and the start/stop lifecycle. // groups, the per-request telemetry middleware and the start/stop lifecycle.
// //
// The /api/v1 route groups (public, user, internal, admin) are created here so // The /api/v1 route groups (public, user, internal, admin) attach their endpoints
// later stages attach their endpoints to a stable structure; the /user group // to a stable structure; the /user group requires the X-User-ID identity header.
// requires the X-User-ID identity header. The probes /healthz (liveness) and // The probes /healthz (liveness) and /readyz (database + session-cache readiness)
// /readyz (database + session-cache readiness) are unauthenticated. // are unauthenticated.
package server package server
import ( import (
@@ -20,6 +20,7 @@ import (
"scrabble/backend/internal/account" "scrabble/backend/internal/account"
"scrabble/backend/internal/adminconsole" "scrabble/backend/internal/adminconsole"
"scrabble/backend/internal/ads" "scrabble/backend/internal/ads"
"scrabble/backend/internal/banview"
"scrabble/backend/internal/connector" "scrabble/backend/internal/connector"
"scrabble/backend/internal/engine" "scrabble/backend/internal/engine"
"scrabble/backend/internal/feedback" "scrabble/backend/internal/feedback"
@@ -83,6 +84,10 @@ type Deps struct {
// admin console's throttled view + the high-rate auto-flag. A nil RateWatch // admin console's throttled view + the high-rate auto-flag. A nil RateWatch
// disables the internal report endpoint and the console view. // disables the internal report endpoint and the console view.
RateWatch *ratewatch.Watch RateWatch *ratewatch.Watch
// BanView mirrors the gateway's active IP bans for the admin console and
// collects operator unban requests. A nil BanView disables the internal
// ban-sync endpoint and the console's active-bans panel.
BanView *banview.View
// Ads is the advertising-banner domain service: campaign rotation feeding the // Ads is the advertising-banner domain service: campaign rotation feeding the
// profile.get banner block, plus the banner admin console section. A nil Ads // profile.get banner block, plus the banner admin console section. A nil Ads
// omits the banner block and disables the banner console. // omits the banner block and disables the banner console.
@@ -115,6 +120,7 @@ type Server struct {
dictDir string dictDir string
connector *connector.Client connector *connector.Client
ratewatch *ratewatch.Watch ratewatch *ratewatch.Watch
banview *banview.View
ads *ads.Service ads *ads.Service
notifier notify.Publisher notifier notify.Publisher
console *adminconsole.Renderer console *adminconsole.Renderer
@@ -164,6 +170,7 @@ func New(addr string, deps Deps) *Server {
dictDir: deps.DictDir, dictDir: deps.DictDir,
connector: deps.Connector, connector: deps.Connector,
ratewatch: deps.RateWatch, ratewatch: deps.RateWatch,
banview: deps.BanView,
ads: deps.Ads, ads: deps.Ads,
notifier: notifier, notifier: notifier,
http: &http.Server{Addr: addr, Handler: engine}, http: &http.Server{Addr: addr, Handler: engine},
@@ -238,7 +245,7 @@ func (s *Server) Invitations() *lobby.InvitationService { return s.invitations }
func (s *Server) Emails() *account.EmailService { return s.emails } func (s *Server) Emails() *account.EmailService { return s.emails }
// Handler returns the underlying HTTP handler. It lets tests drive the server // Handler returns the underlying HTTP handler. It lets tests drive the server
// without binding a socket and lets later stages compose the backend behind // without binding a socket and lets callers compose the backend behind
// another listener. // another listener.
func (s *Server) Handler() http.Handler { return s.http.Handler } func (s *Server) Handler() http.Handler { return s.http.Handler }
+1 -2
View File
@@ -8,8 +8,7 @@ import (
) )
// Service mints, resolves, and revokes sessions over the store and the // Service mints, resolves, and revokes sessions over the store and the
// write-through cache. The gateway is its only caller (from a later stage); the // write-through cache. The gateway is its only caller.
// HTTP surface is wired then.
type Service struct { type Service struct {
store *Store store *Store
cache *Cache cache *Cache
+2 -2
View File
@@ -3,8 +3,8 @@
// in as a message kind. It owns the friendships, blocks and chat_messages tables, // in as a message kind. It owns the friendships, blocks and chat_messages tables,
// reads the account-level block toggles through account.Store, and gates chat and // reads the account-level block toggles through account.Store, and gates chat and
// nudge on game state through a GameReader so it never imports the engine. The // nudge on game state through a GameReader so it never imports the engine. The
// live delivery of chat and nudges (push / in-app stream) belongs to the gateway // live delivery of chat and nudges (push / in-app stream) belongs to the gateway;
// in a later stage; this package only persists and reads them. // this package only persists and reads them.
package social package social
import ( import (
+17 -8
View File
@@ -10,12 +10,13 @@ POSTGRES_USER=scrabble
POSTGRES_PASSWORD=change-me # required POSTGRES_PASSWORD=change-me # required
# --- Dictionary ------------------------------------------------------------- # --- Dictionary -------------------------------------------------------------
# scrabble-dictionary release tag baked into the image as the SEED dictionary # scrabble-dictionary release tag baked into the image as the SEED dictionary for a
# (image build-arg; also labels the resident seed version). After first boot the # FRESH volume (image build-arg; also labels the resident seed version). After first
# dawg-data volume preserves versions uploaded through the admin console, and the # boot the dawg-data volume preserves versions uploaded through the admin console and
# active version lives in the DB — so bumping this on a later deploy does NOT change # the active version lives in the DB. On a live volume a changed value is ignored (the
# a running contour; update the dictionary through /_gm/dictionary instead. # recorded .seed_version marker wins — the seed-drift guard); change a running
DICT_VERSION=v1.0.0 # contour's dictionary through /_gm/dictionary (ARCHITECTURE.md §5).
DICT_VERSION=v1.2.1
# --- Logging ---------------------------------------------------------------- # --- Logging ----------------------------------------------------------------
LOG_LEVEL=info LOG_LEVEL=info
@@ -37,10 +38,18 @@ VITE_GATEWAY_URL=
GRAFANA_ROOT_URL=/_gm/grafana/ # set the full https URL behind a real domain GRAFANA_ROOT_URL=/_gm/grafana/ # set the full https URL behind a real domain
GRAFANA_ADMIN_PASSWORD=admin GRAFANA_ADMIN_PASSWORD=admin
# --- Telegram connector ----------------------------------------------------- # --- Telegram validator + bot -----------------------------------------------
AWG_CONF= # required; AmneziaWG sidecar config # The token is shared: the validator uses it as the HMAC secret, the bot for the
# Bot API. The bot-link wiring (validator/relay/mTLS addresses) is hard-wired in
# docker-compose.yml; the mTLS material is NOT here — run `deploy/gen-certs.sh`
# (writes deploy/certs/, gitignored) before `docker compose up`.
AWG_CONF= # required; AmneziaWG sidecar config (the bot's Telegram egress)
TELEGRAM_BOT_TOKEN= # required TELEGRAM_BOT_TOKEN= # required
TELEGRAM_GAME_CHANNEL_ID= TELEGRAM_GAME_CHANNEL_ID=
TELEGRAM_CHAT_ID= # moderated discussion chat (channel's linked group); empty disables gating
TELEGRAM_PROMO_BOT_TOKEN= # optional standalone promo bot token; empty disables it
TELEGRAM_BOT_USERNAME= # main bot @username without the @ (promo message); required when the promo token is set
TELEGRAM_BOT_LINK= # main bot Mini App link for the promo button (reuse VITE_TELEGRAM_LINK); required when the promo token is set
TELEGRAM_MINIAPP_URL= # required TELEGRAM_MINIAPP_URL= # required
TELEGRAM_TEST_ENV=false TELEGRAM_TEST_ENV=false
TELEGRAM_API_BASE_URL= TELEGRAM_API_BASE_URL=
+90 -18
View File
@@ -1,8 +1,8 @@
# deploy # deploy
The full Scrabble contour: `backend` + `gateway` + the static `landing` + Postgres + The full Scrabble contour: `backend` + `gateway` + the static `landing` + Postgres +
the Telegram connector (with a VPN sidecar) + the observability stack (OTel the Telegram `validator` + `bot` (the bot with a VPN sidecar) + the observability stack
Collector → Prometheus + Tempo → Grafana), fronted by a **caddy** that owns a single (OTel Collector → Prometheus + Tempo → Grafana), fronted by a **caddy** that owns a single
`/_gm` Basic-Auth (the admin console + Grafana). Topology and the decision record are in `/_gm` Basic-Auth (the admin console + Grafana). Topology and the decision record are in
[`../docs/ARCHITECTURE.md`](../docs/ARCHITECTURE.md) §13; this file is the [`../docs/ARCHITECTURE.md`](../docs/ARCHITECTURE.md) §13; this file is the
operational reference for **every environment variable**. operational reference for **every environment variable**.
@@ -16,11 +16,13 @@ operational reference for **every environment variable**.
| `landing` | built (`gateway/Dockerfile`, target `landing`) | Static landing page at `/` (caddy:2-alpine + the shared Vite build, `deploy/landing/Caddyfile`); absorbs stray public paths. | | `landing` | built (`gateway/Dockerfile`, target `landing`) | Static landing page at `/` (caddy:2-alpine + the shared Vite build, `deploy/landing/Caddyfile`); absorbs stray public paths. |
| `backend` | built (`backend/Dockerfile`) | Domain service; bakes in the DAWG dictionaries; runs migrations at boot. | | `backend` | built (`backend/Dockerfile`) | Domain service; bakes in the DAWG dictionaries; runs migrations at boot. |
| `postgres` | `postgres:17-alpine` | Database (named volume, `pg_isready` healthcheck). | | `postgres` | `postgres:17-alpine` | Database (named volume, `pg_isready` healthcheck). |
| `vpn` + `telegram` | sidecar + built (`platform/telegram/Dockerfile`) | Telegram connector; egresses through the AmneziaWG sidecar; internal gRPC at `telegram:9091`. | | `validator` | built (`platform/telegram/Dockerfile`, target `validator`) | Telegram HMAC validator (no VPN, no Bot API); internal gRPC at `validator:9091`. Game login depends only on this. |
| `vpn` + `bot` | sidecar + built (`platform/telegram/Dockerfile`, target `bot`) | Telegram bot, gated to the **`telegram-local`** profile; egresses through the AmneziaWG sidecar and dials the gateway bot-link (mTLS) at `gateway:9443`. The test contour activates the profile; the prod **main** host omits it and runs the bot standalone on its **own host** (`docker-compose.bot.yml`, no VPN — native Bot API egress). |
| `otelcol` | `otel/opentelemetry-collector-contrib` | OTLP/gRPC `:4317` → Prometheus scrape (`:9464`) + Tempo. | | `otelcol` | `otel/opentelemetry-collector-contrib` | OTLP/gRPC `:4317` → Prometheus scrape (`:9464`) + Tempo. |
| `prometheus` | `prom/prometheus` | Metrics, 15d retention. | | `prometheus` | `prom/prometheus` | Metrics, 15d retention (7d in prod). |
| `tempo` | `grafana/tempo` | Traces, 72h retention. | | `tempo` | `grafana/tempo` | Traces, 72h retention. |
| `grafana` | `grafana/grafana` | Dashboards (provisioned), anonymous-admin behind caddy's `/_gm/grafana`. | | `grafana` | `grafana/grafana` | Dashboards (provisioned), anonymous-admin behind caddy's `/_gm/grafana`. |
| `node_exporter` | `quay.io/prometheus/node-exporter` | Host CPU/memory/disk metrics (Prometheus job `node`); the OOM signal on the tight prod main host (2 vCPU / 1.9 GiB). |
Networking: inter-service traffic is on the private `internal` network Networking: inter-service traffic is on the private `internal` network
(project-scoped DNS); only `caddy` joins the shared external `edge` network so the (project-scoped DNS); only `caddy` joins the shared external `edge` network so the
@@ -58,12 +60,19 @@ compose binds from this directory.
| Variable | Gitea kind | Purpose | | Variable | Gitea kind | Purpose |
| --- | --- | --- | | --- | --- | --- |
| `POSTGRES_PASSWORD` | secret | Postgres password (also embedded in `BACKEND_POSTGRES_DSN`). | | `POSTGRES_PASSWORD` | secret | Postgres password (also embedded in `BACKEND_POSTGRES_DSN`). |
| `AWG_CONF` | secret | AmneziaWG config for the VPN sidecar (the connector's only egress). **Must not contain a `DNS=` line** — it hijacks the shared netns's resolv.conf and breaks the connector resolving `otelcol` (telemetry export). Without it, Docker's resolver handles both `otelcol` and `api.telegram.org`. |
| `GM_BASICAUTH_HASH` | secret | bcrypt hash gating `/_gm` (admin console + Grafana). Generate with `docker run --rm caddy:2-alpine caddy hash-password --plaintext '<pw>'`. | | `GM_BASICAUTH_HASH` | secret | bcrypt hash gating `/_gm` (admin console + Grafana). Generate with `docker run --rm caddy:2-alpine caddy hash-password --plaintext '<pw>'`. |
| `TELEGRAM_MINIAPP_URL` | variable | The Mini App URL the connector hands out in deep links / buttons. | | `TELEGRAM_MINIAPP_URL` | variable | The Mini App URL the bot hands out in deep links / buttons. |
**Plus the bot token**`TELEGRAM_BOT_TOKEN` (secret). It defaults to empty in **Plus the bot token**`TELEGRAM_BOT_TOKEN` (secret), shared by the validator (HMAC
compose, but the connector **fails at boot** when it is empty. secret) and the bot (Bot API). It defaults to empty in compose, but both **fail at
boot** when it is empty.
**Conditionally — `AWG_CONF`** (secret): the AmneziaWG config for the VPN sidecar, needed
only when the `telegram-local` profile runs (the test contour and local runs with the
bot). It is **not** `:?`-guarded — compose interpolates profiled-out services too, so the
prod main host (no VPN) must not require it. It **must not contain a `DNS=` line** — that
hijacks the shared netns's resolv.conf and breaks the bot resolving `otelcol` / `gateway`;
without it Docker's resolver handles `otelcol`, `gateway` and `api.telegram.org`.
## Optional variables (with defaults) ## Optional variables (with defaults)
@@ -71,8 +80,8 @@ compose, but the connector **fails at boot** when it is empty.
| --- | --- | --- | --- | | --- | --- | --- | --- |
| `POSTGRES_DB` | variable | `scrabble` | Database name. | | `POSTGRES_DB` | variable | `scrabble` | Database name. |
| `POSTGRES_USER` | variable | `scrabble` | Database user. | | `POSTGRES_USER` | variable | `scrabble` | Database user. |
| `DICT_VERSION` | variable | `v1.0.0` | `scrabble-dictionary` release tag baked into the backend image (build-arg). | | `DICT_VERSION` | variable | `v1.2.1` | `scrabble-dictionary` release tag baked into the backend image as the **seed for a fresh volume** (build-arg). A live contour changes dictionary through the admin console, not this; on a seeded volume a changed value is ignored (the recorded `.seed_version` marker wins — the seed-drift guard, ARCHITECTURE.md §5). Set per contour as `TEST_`/`PROD_DICT_VERSION`. |
| `LOG_LEVEL` | variable | `info` | Shared log level for backend / gateway / connector (`debug\|info\|warn\|error`). | | `LOG_LEVEL` | variable | `info` | Shared log level for backend / gateway / validator / bot (`debug\|info\|warn\|error`). |
| `CADDY_SITE_ADDRESS` | variable | `:80` | Caddy site address. Test: `:80` (host caddy terminates TLS). Prod: a domain, so caddy does its own ACME. | | `CADDY_SITE_ADDRESS` | variable | `:80` | Caddy site address. Test: `:80` (host caddy terminates TLS). Prod: a domain, so caddy does its own ACME. |
| `GM_BASICAUTH_USER` | variable | `gm` | Username for the `/_gm` Basic-Auth. | | `GM_BASICAUTH_USER` | variable | `gm` | Username for the `/_gm` Basic-Auth. |
| `GRAFANA_ROOT_URL` | variable | `/_gm/grafana/` | Grafana root URL (sub-path serving). Set the full `https://<domain>/_gm/grafana/` behind a real domain. | | `GRAFANA_ROOT_URL` | variable | `/_gm/grafana/` | Grafana root URL (sub-path serving). Set the full `https://<domain>/_gm/grafana/` behind a real domain. |
@@ -95,14 +104,77 @@ These are hard-wired in `docker-compose.yml` (no `${...}`), pointing the service
at each other on the `internal` network — listed here so they are not mistaken for at each other on the `internal` network — listed here so they are not mistaken for
missing config: `BACKEND_POSTGRES_DSN` (→ `postgres`, `search_path=backend`), missing config: `BACKEND_POSTGRES_DSN` (→ `postgres`, `search_path=backend`),
`GATEWAY_BACKEND_HTTP_URL`/`_GRPC_ADDR` (→ `backend`), `GATEWAY_BACKEND_HTTP_URL`/`_GRPC_ADDR` (→ `backend`),
`GATEWAY_CONNECTOR_ADDR`/`BACKEND_CONNECTOR_ADDR` (→ `telegram:9091`), and all three `GATEWAY_VALIDATOR_ADDR` (→ `validator:9091`), `BACKEND_CONNECTOR_ADDR` (→ the gateway
services' `*_OTEL_*_EXPORTER=otlp` `OTEL_EXPORTER_OTLP_ENDPOINT=http://otelcol:4317` bot-link relay `gateway:9092`), the bot's `TELEGRAM_GATEWAY_ADDR` (→ `gateway:9443`,
(`_INSECURE=true`). The connector shares the VPN sidecar's netns: routing to the mTLS) with the `GATEWAY_BOTLINK_*` / `TELEGRAM_BOTLINK_*` cert paths under `/certs` (the
collector's internal IP is fine (connected route), but its `AWG_CONF` must **not** mTLS material is generated by `deploy/gen-certs.sh`, gitignored, regenerated each
set a `DNS=` directive — that hijacks resolv.conf and breaks resolving `otelcol` deploy), and all services' `*_OTEL_*_EXPORTER=otlp`
("produced zero addresses"); without it the netns uses Docker's resolver, which `OTEL_EXPORTER_OTLP_ENDPOINT=http://otelcol:4317`
resolves both `otelcol` and `api.telegram.org`. `GATEWAY_ADMIN_*` is intentionally (`_INSECURE=true`). The bot shares the VPN sidecar's netns: routing to the
**unset** — caddy owns `/_gm` in the contour. collector's / gateway's internal IP is fine (connected route), but its `AWG_CONF` must
**not** set a `DNS=` directive — that hijacks resolv.conf and breaks resolving `otelcol`
/ `gateway` ("produced zero addresses"); without it the netns uses Docker's resolver,
which resolves `otelcol`, `gateway` and `api.telegram.org`. `GATEWAY_ADMIN_*` is
intentionally **unset** — caddy owns `/_gm` in the contour.
## Production rollout
Prod runs on **two hosts** (main = full stack + ACME on the domain; tg = the bot only,
native Bot API, no VPN), one-time provisioned by **[`ansible/`](ansible/)** (docker, a
non-sudo `deploy` user holding the CI key, key-only sshd, default-deny ufw, fail2ban).
Re-run `ansible/` after a host resize — it is idempotent.
**To roll out:** merge `development → master` (CI green), then run the **`prod-deploy`**
workflow manually (Gitea → Actions → prod-deploy → run from `master`, input
`confirm=deploy`). It builds + pushes the images to the registry, ships the
compose/config/certs/env over SSH, deploys the main host with `prod-deploy.sh` (rolling,
health-gated, **auto-rollback to the previous tag**), then the bot host, then probes the
public site. After `master` is green this workflow is the **only** thing that touches
prod — nothing auto-deploys there. It runs four visible jobs: **build → deploy-main →
deploy-bot → verify** (the per-service rolling shows in the deploy-main log).
**Versioning.** Each release is a git tag `vX.Y.Z` on `master`; the deploy stamps
`git describe --tags` into every image tag, every binary (`-ldflags``pkg/version`
the `service.version` telemetry attribute) and the SPA About screen. Tag the release
before running the deploy:
```sh
git tag -a v1.0.0 -m v1.0.0 && git push origin v1.0.0
```
**Manual rollback** (any time after a successful deploy). Run the **`prod-rollback`**
workflow (Gitea → Actions → prod-rollback, `confirm=rollback`). Leave `target_version`
blank to roll back to the previously deployed version (read from the host's
`PREVIOUS_TAG`), or set it to a release tag from the **Releases** page. It re-deploys
that already-published image rolling + health-gated — no rebuild, no DB migration
(image rollback is DB-safe under the expand-contract rule). The registry keeps every
release tag, so any prior release is reachable.
**Migrations** must be **expand-contract** (backward-compatible; goose is forward-only):
the automatic rollback is image-only and never restores the DB. A deploy that changes
`backend/internal/postgres/migrations/` opens a maintenance window — the backend (sole
writer) is stopped for a consistent `pg_dump` into `/opt/scrabble/dumps` before the new
backend migrates. **Manual DB restore** (only if a migration was destructive):
`docker exec -i scrabble-postgres psql -U scrabble -d scrabble -c 'DROP SCHEMA backend CASCADE'`,
then pipe the dump into the same `psql`, and redeploy the matching old tag.
**bot-link cert rotation:** regenerate (`deploy/gen-certs.sh /tmp/c --force`), reset the
five `PROD_BOTLINK_*` secrets from `/tmp/c`, and re-run the workflow — both hosts redeploy
together with the fresh CA.
**Sizing / monitoring:** the main host launches undersized (2 vCPU / 1.9 GiB); the prod
overlay trims limits + `GOMAXPROCS=2` + 7d Prometheus retention, and `node_exporter` feeds
host memory to Grafana (`/_gm/grafana/`). Watch host memory and resize at Selectel when
players arrive.
**`PROD_` Gitea set** (mirrors `TEST_`, mapped onto the unprefixed names above) — secrets:
`PROD_{POSTGRES_PASSWORD, GM_BASICAUTH_HASH, GRAFANA_ADMIN_PASSWORD, TELEGRAM_BOT_TOKEN,
TELEGRAM_PROMO_BOT_TOKEN, REGISTRY_PASSWORD, SSH_KEY, SSH_KNOWN_HOSTS, BOTLINK_CA,
BOTLINK_GATEWAY_CERT, BOTLINK_GATEWAY_KEY, BOTLINK_BOT_CERT, BOTLINK_BOT_KEY}`; variables:
`PROD_{REGISTRY_USER, MAIN_HOST, TG_HOST, CADDY_SITE_ADDRESS, GM_BASICAUTH_USER,
GRAFANA_ROOT_URL, LOG_LEVEL, DICT_VERSION, TELEGRAM_MINIAPP_URL, TELEGRAM_GAME_CHANNEL_ID,
TELEGRAM_CHAT_ID, TELEGRAM_BOT_USERNAME, VITE_TELEGRAM_BOT_ID, VITE_TELEGRAM_LINK,
VITE_TELEGRAM_GAME_CHANNEL_NAME}`.
## Host-side setup (outside this repo) ## Host-side setup (outside this repo)
+49
View File
@@ -0,0 +1,49 @@
# Prod host provisioning
Idempotent Ansible that prepares the two production hosts. It installs Docker, a
non-sudo `deploy` service account, SSH hardening, a default-deny firewall,
fail2ban, unattended security upgrades and time sync. It does **not** deploy the
application — that is `.gitea/workflows/prod-deploy.yaml`'s job, running as the
`deploy` account this playbook creates.
Hosts are referenced by `~/.ssh/config` aliases (`scrabble-main-ops`,
`scrabble-tg-ops`), so no IPs or key paths live in the repo.
## Prerequisites (controller)
- `ansible` with the bundled collections (`community.general`, `community.docker`,
`ansible.posix`).
- The two hosts reachable as root via the ssh-config aliases, host keys already
accepted into `known_hosts` (`host_key_checking = True`).
## One-time: the CI deploy key
The CI prod-deploy workflow logs into the hosts as `deploy` using a dedicated
key. Generate it once on the controller, authorize its public half via the
playbook, and store its private half **only** in the Gitea `PROD_SSH_KEY` secret:
```sh
ssh-keygen -t ed25519 -N '' -C scrabble-ci-deploy \
-f ~/.ssh/scrabble_ci_deploy_ed25519
# private half -> Gitea secret PROD_SSH_KEY (set via API); never commit it
```
## Run
```sh
cd deploy/ansible
ansible-playbook site.yml
```
The playbook reads the public key from `~/.ssh/scrabble_ci_deploy_ed25519.pub` by
default; override with `-e deploy_ci_pubkey_path=/path/to/key.pub`. Re-running is
safe (idempotent) and survives a host resize.
## What each host gets
- **both** (`common`): docker-ce + compose plugin, `daemon.json` (live-restore,
10m×3 log rotation), `deploy` user (docker group, no sudo), key-only sshd,
`ufw` default-deny incoming + allow SSH, fail2ban sshd jail, unattended
upgrades, chrony, `/opt/scrabble/{config,certs,dumps,images}`.
- **main**: `ufw` opens 80/443/9443; the external `edge` docker network.
- **tg**: verifies direct `api.telegram.org` egress (the no-VPN assumption).
+11
View File
@@ -0,0 +1,11 @@
[defaults]
inventory = inventory.ini
roles_path = roles
interpreter_python = /usr/bin/python3
host_key_checking = True
stdout_callback = yaml
deprecation_warnings = False
retry_files_enabled = False
[ssh_connection]
pipelining = True
+21
View File
@@ -0,0 +1,21 @@
---
# Service account the CI prod-deploy workflow uses to drive docker on the hosts.
# Membership in the docker group is root-equivalent (docker socket access), which
# is all the deploy workflow needs; the account is deliberately not given sudo.
deploy_user: deploy
# Public half of the dedicated CI deploy SSH key, read from the controller at run
# time. The private half is generated on the controller during provisioning and
# stored ONLY in the Gitea PROD_SSH_KEY secret; it is never committed. Override the
# path with -e deploy_ci_pubkey_path=/path/to/key.pub if the key lives elsewhere.
deploy_ci_pubkey_path: "{{ lookup('env', 'HOME') }}/.ssh/scrabble_ci_deploy_ed25519.pub"
deploy_ci_pubkey: "{{ lookup('file', deploy_ci_pubkey_path) }}"
# Base directory the deploy workflow rsyncs compose files, config, certs and dumps
# into. Owned by deploy_user so the workflow needs no elevation.
scrabble_base_dir: /opt/scrabble
# Docker daemon json-file log rotation, mirroring the compose x-logging anchor so
# the host's own containers (and any ad-hoc runs) rotate identically.
docker_log_max_size: "10m"
docker_log_max_file: "3"
+19
View File
@@ -0,0 +1,19 @@
# Production inventory for Stage 18.
#
# Hosts resolve through the operator's ~/.ssh/config aliases, so HostName (public
# IP), User and IdentityFile live there — no IPs or key paths are committed here.
# scrabble-main-ops -> main stack host (public IP, domain erudit-game.ru)
# scrabble-tg-ops -> Telegram bot host (direct Bot API egress, no VPN)
[main]
scrabble-main-ops
[tg]
scrabble-tg-ops
[prod:children]
main
tg
[prod:vars]
ansible_user=root
@@ -0,0 +1,15 @@
---
- name: restart docker
ansible.builtin.service:
name: docker
state: restarted
- name: reload sshd
ansible.builtin.service:
name: ssh
state: reloaded
- name: restart fail2ban
ansible.builtin.service:
name: fail2ban
state: restarted
+167
View File
@@ -0,0 +1,167 @@
---
# Common baseline applied to both prod hosts: Docker engine, a non-sudo deploy
# service account, SSH hardening, a default-deny firewall, fail2ban, unattended
# security upgrades and time sync. Every task is idempotent.
- name: Install base packages
ansible.builtin.apt:
name:
- ca-certificates
- curl
- gnupg
- ufw
- fail2ban
- unattended-upgrades
- chrony
state: present
update_cache: true
cache_valid_time: 3600
# --- Docker engine (official repo; trixie is published upstream) ---------------
- name: Create apt keyring directory
ansible.builtin.file:
path: /etc/apt/keyrings
state: directory
mode: "0755"
- name: Install Docker apt GPG key
ansible.builtin.get_url:
url: https://download.docker.com/linux/debian/gpg
dest: /etc/apt/keyrings/docker.asc
mode: "0644"
- name: Add Docker apt repository
ansible.builtin.apt_repository:
repo: >-
deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc]
https://download.docker.com/linux/debian {{ ansible_distribution_release }} stable
filename: docker
state: present
- name: Install Docker engine and the compose plugin
ansible.builtin.apt:
name:
- docker-ce
- docker-ce-cli
- containerd.io
- docker-buildx-plugin
- docker-compose-plugin
state: present
update_cache: true
- name: Configure the Docker daemon (live-restore + log rotation)
ansible.builtin.template:
src: daemon.json.j2
dest: /etc/docker/daemon.json
mode: "0644"
notify: restart docker
- name: Enable and start Docker
ansible.builtin.service:
name: docker
enabled: true
state: started
# --- Deploy service account ----------------------------------------------------
- name: Create the deploy service account
ansible.builtin.user:
name: "{{ deploy_user }}"
groups: docker
append: true
shell: /bin/bash
create_home: true
- name: Ensure the deploy .ssh directory
ansible.builtin.file:
path: "/home/{{ deploy_user }}/.ssh"
state: directory
owner: "{{ deploy_user }}"
group: "{{ deploy_user }}"
mode: "0700"
- name: Authorize the CI deploy SSH key (exclusive)
ansible.builtin.copy:
dest: "/home/{{ deploy_user }}/.ssh/authorized_keys"
content: "{{ deploy_ci_pubkey }}\n"
owner: "{{ deploy_user }}"
group: "{{ deploy_user }}"
mode: "0600"
# --- SSH hardening -------------------------------------------------------------
- name: Harden sshd (key-only auth)
ansible.builtin.template:
src: sshd-hardening.conf.j2
dest: /etc/ssh/sshd_config.d/10-scrabble-hardening.conf
mode: "0644"
validate: sshd -t -f %s
notify: reload sshd
# --- Firewall (default deny incoming) ------------------------------------------
# SSH is allowed before the policy flips so enabling ufw never locks us out.
- name: Allow SSH through the firewall
community.general.ufw:
rule: allow
name: OpenSSH
- name: Default-deny incoming, allow outgoing
community.general.ufw:
direction: "{{ item.direction }}"
policy: "{{ item.policy }}"
loop:
- { direction: incoming, policy: deny }
- { direction: outgoing, policy: allow }
- name: Enable the firewall
community.general.ufw:
state: enabled
# --- fail2ban ------------------------------------------------------------------
- name: Configure the fail2ban sshd jail
ansible.builtin.template:
src: jail.local.j2
dest: /etc/fail2ban/jail.local
mode: "0644"
notify: restart fail2ban
- name: Enable and start fail2ban
ansible.builtin.service:
name: fail2ban
enabled: true
state: started
# --- Unattended security upgrades + time sync ----------------------------------
- name: Enable unattended upgrades
ansible.builtin.copy:
dest: /etc/apt/apt.conf.d/20auto-upgrades
mode: "0644"
content: |
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";
- name: Enable and start chrony
ansible.builtin.service:
name: chrony
enabled: true
state: started
# --- Deploy directories --------------------------------------------------------
- name: Create the scrabble base directories
ansible.builtin.file:
path: "{{ scrabble_base_dir }}/{{ item }}"
state: directory
owner: "{{ deploy_user }}"
group: "{{ deploy_user }}"
mode: "0750"
loop:
- ""
- config
- certs
- dumps
- images
@@ -0,0 +1,8 @@
{
"live-restore": true,
"log-driver": "json-file",
"log-opts": {
"max-size": "{{ docker_log_max_size }}",
"max-file": "{{ docker_log_max_file }}"
}
}
@@ -0,0 +1,9 @@
# Managed by Ansible (deploy/ansible).
[DEFAULT]
bantime = 1h
findtime = 10m
maxretry = 5
backend = systemd
[sshd]
enabled = true
@@ -0,0 +1,6 @@
# Managed by Ansible (deploy/ansible). Key-only authentication.
# root stays reachable by key (prohibit-password) for provisioning re-runs.
PasswordAuthentication no
PermitRootLogin prohibit-password
PubkeyAuthentication yes
KbdInteractiveAuthentication no
+18
View File
@@ -0,0 +1,18 @@
---
# Main stack host: public web + bot-link ports and the external 'edge' network
# the compose stack attaches caddy to.
- name: Open public web and bot-link ports
community.general.ufw:
rule: allow
port: "{{ item }}"
proto: tcp
loop:
- "80" # HTTP (ACME challenge + redirect to HTTPS)
- "443" # HTTPS (caddy edge)
- "9443" # bot-link mTLS (remote bot dials in; mutual TLS gates access)
- name: Ensure the external 'edge' docker network exists
community.docker.docker_network:
name: edge
state: present
+19
View File
@@ -0,0 +1,19 @@
---
# Telegram bot host: holds no inbound port beyond SSH (the bot dials out to the
# Bot API and into the main host's bot-link). We only verify direct Bot API
# egress here, since the "no VPN" decision depends on it.
- name: Verify direct Telegram Bot API egress (no VPN on this host)
ansible.builtin.uri:
url: https://api.telegram.org/
method: GET
status_code: [200, 301, 302, 401, 404] # any HTTP reply proves reachability
timeout: 10
register: tg_egress
failed_when: false
- name: Report Telegram reachability
ansible.builtin.debug:
msg: >-
api.telegram.org reachable:
{{ (tg_egress.status | default(0) | int) > 0 }} (status {{ tg_egress.status | default('none') }})
+31
View File
@@ -0,0 +1,31 @@
---
# Production host provisioning. Idempotent: safe to re-run after a host resize.
# Prepares hosts only (docker, hardening, service account, firewall); the
# application is deployed separately by .gitea/workflows/prod-deploy.yaml.
- name: Common baseline (both hosts)
hosts: prod
become: true
pre_tasks:
- name: Require a well-formed CI deploy public key
ansible.builtin.assert:
that:
- deploy_ci_pubkey | length > 0
- deploy_ci_pubkey is search('^(ssh|ecdsa)-')
fail_msg: >-
deploy_ci_pubkey is empty or malformed. Generate the key first
(see deploy/ansible/README.md) or override deploy_ci_pubkey_path.
roles:
- common
- name: Main stack host
hosts: main
become: true
roles:
- main
- name: Telegram bot host
hosts: tg
become: true
roles:
- tg
+20 -2
View File
@@ -38,10 +38,28 @@
} }
} }
# The game SPA and the Connect edge are served by the gateway. # The game SPA and the Connect edge are served by the gateway. Strip any
# client-supplied X-Scrabble-Honeypot here so the gateway only ever honours the
# tag the honeypot block sets below (a client cannot self-tag a real request).
@gateway path /app /app/* /telegram /telegram/* /scrabble.edge.v1.Gateway/* @gateway path /app /app/* /telegram /telegram/* /scrabble.edge.v1.Gateway/*
handle @gateway { handle @gateway {
reverse_proxy gateway:8081 reverse_proxy gateway:8081 {
header_up -X-Scrabble-Honeypot
}
}
# Honeypot decoy paths: classic vulnerability-scanner bait no real client ever
# requests. Route them to the gateway tagged with X-Scrabble-Honeypot — the set
# replaces any client-supplied value — so it logs the scanner hit and (in prod)
# bans the source IP. (A delete + set in one block would not work: Caddy applies
# header_up deletions after sets, which would strip the tag we just set; the real
# endpoints instead strip the header in the @gateway block above.) Keep this list
# disjoint from every legitimate landing/app path.
@honeypot path /.env /.git /.git/* /.aws/* /wp-login.php /wp-admin /wp-admin/* /phpmyadmin /phpmyadmin/*
handle @honeypot {
reverse_proxy gateway:8081 {
header_up X-Scrabble-Honeypot 1
}
} }
# Everything else — the public landing at / and any stray path — is static. # Everything else — the public landing at / and any stray path — is static.
+53
View File
@@ -0,0 +1,53 @@
# Production Telegram bot host descriptor (standalone — NOT an overlay). Run only on
# the bot host:
# docker compose -f docker-compose.bot.yml up -d
#
# The bot egresses to the Bot API directly (no VPN sidecar) and dials the main host's
# published bot-link :9443 over mTLS. It exports no telemetry — otelcol lives on the
# main host and is unreachable from here — so observe it via `docker logs` on this host.
# Values come from the prod-deploy workflow (PROD_ secrets/variables); BOT_IMAGE is the
# pushed registry tag and BOTLINK_GATEWAY_ADDR is the main host's <ip>:9443.
name: scrabble-bot
services:
bot:
container_name: scrabble-telegram-bot
image: ${BOT_IMAGE:?set BOT_IMAGE to the registry tag}
restart: unless-stopped
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
environment:
TELEGRAM_BOT_TOKEN: ${TELEGRAM_BOT_TOKEN:?set TELEGRAM_BOT_TOKEN}
TELEGRAM_GAME_CHANNEL_ID: ${TELEGRAM_GAME_CHANNEL_ID:-}
TELEGRAM_CHAT_ID: ${TELEGRAM_CHAT_ID:-}
TELEGRAM_PROMO_BOT_TOKEN: ${TELEGRAM_PROMO_BOT_TOKEN:-}
TELEGRAM_BOT_USERNAME: ${TELEGRAM_BOT_USERNAME:-}
TELEGRAM_BOT_LINK: ${TELEGRAM_BOT_LINK:-}
TELEGRAM_MINIAPP_URL: ${TELEGRAM_MINIAPP_URL:?set TELEGRAM_MINIAPP_URL}
# Real Bot API in prod (the test contour pins TELEGRAM_TEST_ENV=true instead).
TELEGRAM_TEST_ENV: "false"
TELEGRAM_API_BASE_URL: ${TELEGRAM_API_BASE_URL:-}
TELEGRAM_OWNS_UPDATES: "true"
# Dials the main host's published bot-link. ServerName stays `gateway` (the cert
# SAN), so TLS validation is independent of the dial address.
TELEGRAM_GATEWAY_ADDR: ${BOTLINK_GATEWAY_ADDR:?set BOTLINK_GATEWAY_ADDR (main:9443)}
TELEGRAM_BOTLINK_SERVER_NAME: gateway
TELEGRAM_BOTLINK_TLS_CERT: /certs/bot.crt
TELEGRAM_BOTLINK_TLS_KEY: /certs/bot.key
TELEGRAM_BOTLINK_TLS_CA: /certs/ca.crt
TELEGRAM_LOG_LEVEL: ${LOG_LEVEL:-info}
TELEGRAM_SERVICE_NAME: scrabble-telegram-bot
# No telemetry export: otelcol is on the main host, unreachable from here.
TELEGRAM_OTEL_TRACES_EXPORTER: none
TELEGRAM_OTEL_METRICS_EXPORTER: none
GOMAXPROCS: "1"
volumes:
- ${SCRABBLE_CONFIG_DIR:-.}/certs:/certs:ro
deploy:
resources:
limits:
cpus: "1.0"
memory: 256M
+98
View File
@@ -0,0 +1,98 @@
# Production main-host overlay, applied on top of docker-compose.yml on the main host:
# docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
#
# It (1) publishes caddy 80/443 — there is no host caddy in prod, so the contour caddy
# owns the edge and does its own ACME on CADDY_SITE_ADDRESS — and the gateway bot-link
# :9443 the remote bot dials in over mTLS; and (2) retunes the baseline limits down for the
# 2 vCPU / 1.9 GiB host (GOMAXPROCS=2, smaller memory caps, shorter Prometheus
# retention). The contour launches deliberately undersized at zero players; the added
# node_exporter + Grafana watch host memory so it can be resized at Selectel when
# traffic arrives.
#
# The bot + its VPN sidecar are absent here (the telegram-local profile is not
# activated); the prod bot runs on its own host from docker-compose.bot.yml.
services:
caddy:
ports:
- "80:80"
- "443:443"
deploy:
resources:
limits:
memory: 96M
gateway:
# Prod pulls the pushed image by tag instead of building locally; the base
# build: section stays dormant because the deploy always pulls first.
image: ${REGISTRY:?set REGISTRY}/scrabble-gateway:${TAG:?set TAG}
ports:
- "9443:9443"
environment:
# 2 vCPU host: align the Go scheduler with the cgroup quota (the baseline's 3-core gateway needs 3 cores).
GOMAXPROCS: "2"
deploy:
resources:
limits:
cpus: "2.0"
memory: 384M
backend:
image: ${REGISTRY:?set REGISTRY}/scrabble-backend:${TAG:?set TAG}
deploy:
resources:
limits:
memory: 384M
postgres:
deploy:
resources:
limits:
memory: 384M
validator:
image: ${REGISTRY:?set REGISTRY}/scrabble-telegram-validator:${TAG:?set TAG}
deploy:
resources:
limits:
memory: 96M
landing:
image: ${REGISTRY:?set REGISTRY}/scrabble-landing:${TAG:?set TAG}
deploy:
resources:
limits:
memory: 64M
otelcol:
deploy:
resources:
limits:
memory: 256M
prometheus:
command:
- --config.file=/etc/prometheus/prometheus.yml
- --storage.tsdb.retention.time=7d
deploy:
resources:
limits:
memory: 256M
tempo:
deploy:
resources:
limits:
memory: 384M
grafana:
deploy:
resources:
limits:
memory: 256M
postgres_exporter:
deploy:
resources:
limits:
memory: 64M
+157 -39
View File
@@ -1,6 +1,6 @@
# Full deploy descriptor for the Scrabble test contour: backend + gateway + # Full deploy descriptor for the Scrabble test contour: backend + gateway +
# Postgres + the Telegram connector (with its VPN sidecar) + the observability # Postgres + the Telegram validator + bot (the bot with its VPN sidecar) + the
# stack (OTel Collector -> Prometheus + Tempo -> Grafana). Driven by # observability stack (OTel Collector -> Prometheus + Tempo -> Grafana). Driven by
# .gitea/workflows/ci.yaml (`docker compose up -d --build`); env values are # .gitea/workflows/ci.yaml (`docker compose up -d --build`); env values are
# interpolated from Gitea Actions TEST_ secrets/variables exported by the deploy # interpolated from Gitea Actions TEST_ secrets/variables exported by the deploy
# job (see deploy/.env.example for the unprefixed names). # job (see deploy/.env.example for the unprefixed names).
@@ -19,13 +19,15 @@
# the test contour; the host caddy terminates TLS and forwards. For prod # the test contour; the host caddy terminates TLS and forwards. For prod
# (no host caddy) set CADDY_SITE_ADDRESS to the domain so the caddy # (no host caddy) set CADDY_SITE_ADDRESS to the domain so the caddy
# does its own ACME — the contour is then self-contained. # does its own ACME — the contour is then self-contained.
# - The connector egresses to api.telegram.org through the `vpn` sidecar # - The validator answers internal gRPC at `validator:9091` (no VPN, HMAC only).
# (network_mode: service:vpn); it answers internal gRPC at `telegram:9091`. # The bot egresses to api.telegram.org through the `vpn` sidecar (network_mode:
# service:vpn) and dials the gateway bot-link (mTLS) at `gateway:9443`. The
# backend admin relay reaches the gateway at `gateway:9092` (plaintext).
name: scrabble name: scrabble
# Bound every container's json-file logs. R7 measured the backend emitting a # Bound every container's json-file logs. The backend emits a per-request latency
# per-request latency line at info (~14 MiB / 30 min under the 500-player stress # line at info (~14 MiB / 30 min under the 500-player peak); without rotation the
# peak); without rotation the volume grows unbounded. 10 MiB x 3 files caps each # volume grows unbounded. 10 MiB x 3 files caps each
# container at 30 MiB. Applied to every service via the *default-logging alias. # container at 30 MiB. Applied to every service via the *default-logging alias.
x-logging: &default-logging x-logging: &default-logging
driver: json-file driver: json-file
@@ -50,8 +52,8 @@ services:
retries: 30 retries: 30
volumes: volumes:
- postgres-data:/var/lib/postgresql/data - postgres-data:/var/lib/postgresql/data
# R7 starting limits: 512M leaves headroom over the default 128 MB shared_buffers + # 512M leaves headroom over the default 128 MB shared_buffers + per-connection
# per-connection memory (R2 peaked at 28 backends / 69 MiB RSS); tighten after the run. # memory (the load harness peaked at 28 backends / 69 MiB RSS).
deploy: deploy:
resources: resources:
limits: limits:
@@ -66,7 +68,11 @@ services:
context: .. context: ..
dockerfile: backend/Dockerfile dockerfile: backend/Dockerfile
args: args:
DICT_VERSION: ${DICT_VERSION:-v1.0.0} # Seed dictionary for a FRESH volume; the per-contour value comes from the
# deploy env (Gitea TEST_/PROD_DICT_VERSION). See the volume note below.
DICT_VERSION: ${DICT_VERSION:-v1.2.1}
# Build version stamped into the binary (git tag; see pkg/version).
VERSION: ${APP_VERSION:-dev}
restart: unless-stopped restart: unless-stopped
logging: *default-logging logging: *default-logging
depends_on: depends_on:
@@ -75,12 +81,14 @@ services:
environment: environment:
# search_path=backend matches the migrations (00001 creates the schema). # search_path=backend matches the migrations (00001 creates the schema).
BACKEND_POSTGRES_DSN: postgres://${POSTGRES_USER:-scrabble}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB:-scrabble}?sslmode=disable&search_path=backend BACKEND_POSTGRES_DSN: postgres://${POSTGRES_USER:-scrabble}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB:-scrabble}?sslmode=disable&search_path=backend
# R7 tuned: the pool sat at its 25-conn cap (28 backends total) at 500 players; # The pool caps at 25 conns (~28 backends) around 500 players; 40 gives headroom
# 40 gives headroom for bursts. Postgres (2 cores / 512 MiB) handles it. # for bursts. Postgres (2 cores / 512 MiB) handles it.
BACKEND_POSTGRES_MAX_OPEN_CONNS: "40" BACKEND_POSTGRES_MAX_OPEN_CONNS: "40"
BACKEND_HTTP_ADDR: ":8080" BACKEND_HTTP_ADDR: ":8080"
BACKEND_GRPC_ADDR: ":9090" BACKEND_GRPC_ADDR: ":9090"
BACKEND_CONNECTOR_ADDR: telegram:9091 # Admin broadcasts go to the gateway's bot-link relay, which forwards them to
# the remote bot and reports back whether they were delivered.
BACKEND_CONNECTOR_ADDR: gateway:9092
BACKEND_LOG_LEVEL: ${LOG_LEVEL:-info} BACKEND_LOG_LEVEL: ${LOG_LEVEL:-info}
BACKEND_SERVICE_NAME: scrabble-backend BACKEND_SERVICE_NAME: scrabble-backend
BACKEND_OTEL_TRACES_EXPORTER: otlp BACKEND_OTEL_TRACES_EXPORTER: otlp
@@ -94,15 +102,17 @@ services:
# (the image's /opt/dawg is owned by the nonroot UID, which the fresh volume # (the image's /opt/dawg is owned by the nonroot UID, which the fresh volume
# inherits). The admin console writes new version subdirectories here, and the # inherits). The admin console writes new version subdirectories here, and the
# volume preserves them — and the versions in-progress games pin — across # volume preserves them — and the versions in-progress games pin — across
# redeploys. Once seeded the volume is not re-seeded, so bumping DICT_VERSION on a # redeploys. Once seeded the volume is not re-seeded: DICT_VERSION is the seed for
# later deploy has no effect on an existing contour; update via the console # a FRESH volume only. On a live volume a changed DICT_VERSION is ignored (the
# instead (docs/ARCHITECTURE.md §5). # recorded .seed_version marker wins — the seed-drift guard), so a running contour's
# dictionary is changed through the admin console, not by bumping the seed
# (docs/ARCHITECTURE.md §5).
volumes: volumes:
- dawg-data:/opt/dawg - dawg-data:/opt/dawg
# No container healthcheck: the distroless image has no shell/wget. Readiness # No container healthcheck: the distroless image has no shell/wget. Readiness
# is covered by the CI post-deploy probe (GET / through caddy). # is covered by the CI post-deploy probe (GET / through caddy).
# R7 starting limits (generous over the R2 ~1-core / <=100 MiB peak); tightened to # Generous over the ~1-core / <=100 MiB measured peak; the prod overlay trims these
# the agreed prod values after the final stress run. deploy.resources.limits is # to the launch-host values. deploy.resources.limits is
# honoured by `docker compose up` (Compose v2), not only by swarm. # honoured by `docker compose up` (Compose v2), not only by swarm.
deploy: deploy:
resources: resources:
@@ -124,6 +134,8 @@ services:
VITE_TELEGRAM_GAME_CHANNEL_NAME: ${VITE_TELEGRAM_GAME_CHANNEL_NAME:-} VITE_TELEGRAM_GAME_CHANNEL_NAME: ${VITE_TELEGRAM_GAME_CHANNEL_NAME:-}
VITE_GATEWAY_URL: ${VITE_GATEWAY_URL:-} VITE_GATEWAY_URL: ${VITE_GATEWAY_URL:-}
VITE_APP_VERSION: ${APP_VERSION:-dev} VITE_APP_VERSION: ${APP_VERSION:-dev}
# Go binary version (the SPA's VITE_APP_VERSION is the same git tag).
VERSION: ${APP_VERSION:-dev}
restart: unless-stopped restart: unless-stopped
logging: *default-logging logging: *default-logging
depends_on: [backend] depends_on: [backend]
@@ -131,7 +143,24 @@ services:
GATEWAY_HTTP_ADDR: ":8081" GATEWAY_HTTP_ADDR: ":8081"
GATEWAY_BACKEND_HTTP_URL: http://backend:8080 GATEWAY_BACKEND_HTTP_URL: http://backend:8080
GATEWAY_BACKEND_GRPC_ADDR: backend:9090 GATEWAY_BACKEND_GRPC_ADDR: backend:9090
GATEWAY_CONNECTOR_ADDR: telegram:9091 # Telegram auth validates against the home validator (plaintext, internal).
GATEWAY_VALIDATOR_ADDR: validator:9091
# The reverse bot-link: the bot dials :9443 over mTLS; the backend admin relay
# reaches the gateway at :9092 (plaintext, internal). In the test contour both
# listeners stay on the internal network (the bot shares the VPN netns); in prod
# the bot is a separate host and :9443 is published with public certificates.
GATEWAY_BOTLINK_ADDR: ":9443"
GATEWAY_BOTLINK_RELAY_ADDR: ":9092"
GATEWAY_BOTLINK_TLS_CERT: /certs/gateway.crt
GATEWAY_BOTLINK_TLS_KEY: /certs/gateway.key
GATEWAY_BOTLINK_TLS_CA: /certs/ca.crt
# Anti-abuse IP ban (fail2ban-style), fed by rate-limit rejections and the
# honeypot/honeytoken. Off by default: it bans by client IP, which is only
# real in prod — the test contour arrives as one shared NAT address, so a ban
# there would be self-inflicted (the honeypot/honeytoken still log). Prod sets
# these from PROD_ inputs; GATEWAY_HONEYTOKEN is the planted bearer trap.
GATEWAY_ABUSE_BAN_ENABLED: ${GATEWAY_ABUSE_BAN_ENABLED:-false}
GATEWAY_HONEYTOKEN: ${GATEWAY_HONEYTOKEN:-}
GATEWAY_LOG_LEVEL: ${LOG_LEVEL:-info} GATEWAY_LOG_LEVEL: ${LOG_LEVEL:-info}
GATEWAY_SERVICE_NAME: scrabble-gateway GATEWAY_SERVICE_NAME: scrabble-gateway
GATEWAY_OTEL_TRACES_EXPORTER: otlp GATEWAY_OTEL_TRACES_EXPORTER: otlp
@@ -142,7 +171,11 @@ services:
GOMAXPROCS: "3" GOMAXPROCS: "3"
# GATEWAY_ADMIN_* intentionally unset: in the deployed contour the front # GATEWAY_ADMIN_* intentionally unset: in the deployed contour the front
# caddy owns the /_gm Basic-Auth and routes /_gm to the backend directly. # caddy owns the /_gm Basic-Auth and routes /_gm to the backend directly.
# R7 tuned: the gateway holds one h2c connection per player, so at 500 players it # The bot-link mTLS material (CA + gateway server leaf). Generated by
# deploy/gen-certs.sh for the test contour; supplied from PROD_ secrets in prod.
volumes:
- ${SCRABBLE_CONFIG_DIR:-.}/certs:/certs:ro
# The gateway holds one h2c connection per player, so at 500 players it
# bursts into a 2-core cap (~2.49% transport_error on game.state); 3 cores absorbs # bursts into a 2-core cap (~2.49% transport_error on game.state); 3 cores absorbs
# the bursts. Per-connection overhead is the realistic prod cost — size for it. # the bursts. Per-connection overhead is the realistic prod cost — size for it.
deploy: deploy:
@@ -178,53 +211,118 @@ services:
memory: 128M memory: 128M
networks: [internal] networks: [internal]
# --- Telegram connector (egress via the VPN sidecar) ----------------------- # --- Telegram validator (home; HMAC only, no VPN, no Telegram egress) -------
# The validator holds the bot token solely as the HMAC secret and never reaches
# the Bot API, so it runs on the main network with no VPN. Game login validates
# against it and stays up even when the remote bot or the bot-link is down.
validator:
container_name: scrabble-telegram-validator
image: scrabble-telegram-validator:latest
build:
context: ..
dockerfile: platform/telegram/Dockerfile
target: validator
args:
VERSION: ${APP_VERSION:-dev}
restart: unless-stopped
logging: *default-logging
environment:
# The token is the HMAC secret only; the validator never calls the Bot API. An
# empty value crash-loops only the validator; the rest of the contour comes up.
TELEGRAM_BOT_TOKEN: ${TELEGRAM_BOT_TOKEN:-}
TELEGRAM_VALIDATOR_GRPC_ADDR: ":9091"
TELEGRAM_LOG_LEVEL: ${LOG_LEVEL:-info}
TELEGRAM_SERVICE_NAME: scrabble-telegram-validator
TELEGRAM_OTEL_TRACES_EXPORTER: otlp
TELEGRAM_OTEL_METRICS_EXPORTER: otlp
OTEL_EXPORTER_OTLP_ENDPOINT: http://otelcol:4317
OTEL_EXPORTER_OTLP_INSECURE: "true"
GOMAXPROCS: "1"
deploy:
resources:
limits:
cpus: "1.0"
memory: 128M
networks: [internal]
# --- Telegram bot (egress via the VPN sidecar in test; dials the gateway) ---
# vpn + bot are gated to the `telegram-local` profile: the test contour runs them
# locally (CI passes --profile telegram-local), the prod main host omits them, and
# the prod bot runs on its own host from deploy/docker-compose.bot.yml.
vpn: vpn:
container_name: scrabble-telegram-vpn container_name: scrabble-telegram-vpn
image: docker.iliadenisov.ru/developer/amneziawg-sidecar:latest image: docker.iliadenisov.ru/developer/amneziawg-sidecar:latest
profiles: ["telegram-local"]
restart: unless-stopped restart: unless-stopped
logging: *default-logging logging: *default-logging
privileged: true privileged: true
environment: environment:
AWG_CONF: ${AWG_CONF:?set AWG_CONF} # Required by the vpn sidecar, which is gated to the telegram-local profile.
# Compose can't scope a `:?` guard to a profile (interpolation runs for
# profiled-out services too) and the prod main host has no VPN, so this is a soft
# default; the test contour always supplies TEST_AWG_CONF and the sidecar validates it.
AWG_CONF: ${AWG_CONF:-}
networks: networks:
internal: internal:
aliases: [telegram] aliases: [telegram]
telegram: bot:
container_name: scrabble-telegram container_name: scrabble-telegram-bot
image: scrabble-telegram:latest image: scrabble-telegram-bot:latest
profiles: ["telegram-local"]
build: build:
context: .. context: ..
dockerfile: platform/telegram/Dockerfile dockerfile: platform/telegram/Dockerfile
target: bot
args:
VERSION: ${APP_VERSION:-dev}
restart: unless-stopped restart: unless-stopped
logging: *default-logging logging: *default-logging
depends_on: [vpn] depends_on: [vpn]
network_mode: "service:vpn" network_mode: "service:vpn"
environment: environment:
# The bot token lives ONLY in this container (ARCHITECTURE.md §12). The connector # The bot token lives on the bot host (ARCHITECTURE.md §12). The bot requires it
# requires it at boot; an empty value leaves the Telegram side down while the rest # at boot; an empty value leaves the bot down while the rest of the contour comes up.
# of the contour still comes up.
TELEGRAM_BOT_TOKEN: ${TELEGRAM_BOT_TOKEN:-} TELEGRAM_BOT_TOKEN: ${TELEGRAM_BOT_TOKEN:-}
TELEGRAM_GAME_CHANNEL_ID: ${TELEGRAM_GAME_CHANNEL_ID:-} TELEGRAM_GAME_CHANNEL_ID: ${TELEGRAM_GAME_CHANNEL_ID:-}
# The moderated discussion chat (a channel's linked group) the bot gates write
# access in. Empty disables gating. The group must ALLOW sending by default — the bot
# only restricts (mutes the ineligible) — and the bot must be an admin there with the
# "Ban users" right; chat_member updates are delivered only to a chat admin.
TELEGRAM_CHAT_ID: ${TELEGRAM_CHAT_ID:-}
# The optional standalone promo bot (its own token) answering /start with a button
# into the main bot's app. Empty disables it; when set it needs the main bot's
# @username and the Mini App link (reused from the UI's VITE_TELEGRAM_LINK).
TELEGRAM_PROMO_BOT_TOKEN: ${TELEGRAM_PROMO_BOT_TOKEN:-}
TELEGRAM_BOT_USERNAME: ${TELEGRAM_BOT_USERNAME:-}
TELEGRAM_BOT_LINK: ${TELEGRAM_BOT_LINK:-}
TELEGRAM_MINIAPP_URL: ${TELEGRAM_MINIAPP_URL:?set TELEGRAM_MINIAPP_URL} TELEGRAM_MINIAPP_URL: ${TELEGRAM_MINIAPP_URL:?set TELEGRAM_MINIAPP_URL}
TELEGRAM_GRPC_ADDR: ":9091"
TELEGRAM_TEST_ENV: ${TELEGRAM_TEST_ENV:-false} TELEGRAM_TEST_ENV: ${TELEGRAM_TEST_ENV:-false}
TELEGRAM_API_BASE_URL: ${TELEGRAM_API_BASE_URL:-} TELEGRAM_API_BASE_URL: ${TELEGRAM_API_BASE_URL:-}
TELEGRAM_OWNS_UPDATES: "true"
# The bot dials the gateway bot-link over mTLS. In test it reaches the gateway by
# its internal service name through the VPN netns (Docker resolver, off-tunnel,
# like otelcol); in prod it is a separate host dialing the gateway's public port.
TELEGRAM_GATEWAY_ADDR: gateway:9443
TELEGRAM_BOTLINK_SERVER_NAME: gateway
TELEGRAM_BOTLINK_TLS_CERT: /certs/bot.crt
TELEGRAM_BOTLINK_TLS_KEY: /certs/bot.key
TELEGRAM_BOTLINK_TLS_CA: /certs/ca.crt
TELEGRAM_LOG_LEVEL: ${LOG_LEVEL:-info} TELEGRAM_LOG_LEVEL: ${LOG_LEVEL:-info}
TELEGRAM_SERVICE_NAME: scrabble-telegram TELEGRAM_SERVICE_NAME: scrabble-telegram-bot
# The connector shares the VPN sidecar's netns. Routing to the collector's # The bot shares the VPN sidecar's netns. Routing to internal IPs stays off the
# internal IP stays off the tunnel (connected route), but the sidecar's DNS # tunnel (connected route), but the sidecar's DNS hijacks name resolution:
# hijacks name resolution: AWG_CONF must NOT carry a `DNS=` directive, else # AWG_CONF must NOT carry a `DNS=` directive, else `otelcol` and `gateway` won't
# `otelcol` won't resolve ("produced zero addresses"). Without DNS= the netns # resolve. Without DNS= the netns uses Docker's resolver, which resolves the
# uses Docker's resolver, which resolves both otelcol and api.telegram.org # internal services and api.telegram.org (see deploy/README.md).
# (see deploy/README.md).
TELEGRAM_OTEL_TRACES_EXPORTER: otlp TELEGRAM_OTEL_TRACES_EXPORTER: otlp
TELEGRAM_OTEL_METRICS_EXPORTER: otlp TELEGRAM_OTEL_METRICS_EXPORTER: otlp
OTEL_EXPORTER_OTLP_ENDPOINT: http://otelcol:4317 OTEL_EXPORTER_OTLP_ENDPOINT: http://otelcol:4317
OTEL_EXPORTER_OTLP_INSECURE: "true" OTEL_EXPORTER_OTLP_INSECURE: "true"
# The connector is light (the stress run does not drive Telegram); one P suffices. # The bot is light (the stress run does not drive Telegram); one P suffices.
GOMAXPROCS: "1" GOMAXPROCS: "1"
volumes:
- ${SCRABBLE_CONFIG_DIR:-.}/certs:/certs:ro
deploy: deploy:
resources: resources:
limits: limits:
@@ -304,8 +402,8 @@ services:
volumes: volumes:
- ${SCRABBLE_CONFIG_DIR:-.}/tempo/tempo.yaml:/etc/tempo/tempo.yaml:ro - ${SCRABBLE_CONFIG_DIR:-.}/tempo/tempo.yaml:/etc/tempo/tempo.yaml:ro
- tempo-data:/var/tempo - tempo-data:/var/tempo
# R7 tuned: tempo reached the 1 GiB cap during the final run (446 MiB in R2); # Tempo reached the 1 GiB cap under sustained load (446 MiB in earlier runs);
# raised to 2 GiB for headroom against OOM under sustained tracing load. # raised to 2 GiB for headroom against OOM.
deploy: deploy:
resources: resources:
limits: limits:
@@ -363,6 +461,26 @@ services:
memory: 128M memory: 128M
networks: [internal] networks: [internal]
# node_exporter exports host CPU/memory/disk metrics. The prod main host runs a tight
# 1.9 GiB budget, so host memory pressure — not just per-container docker_stats — is
# what warns before an OOM. Prometheus scrapes it at :9100 (see prometheus.yml).
node_exporter:
container_name: scrabble-node-exporter
image: quay.io/prometheus/node-exporter:v1.8.2
restart: unless-stopped
logging: *default-logging
command:
- --path.rootfs=/host
- --collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host)($|/)
pid: host
volumes:
- /:/host:ro,rslave
deploy:
resources:
limits:
memory: 64M
networks: [internal]
networks: networks:
internal: internal:
name: scrabble-internal name: scrabble-internal
+68
View File
@@ -0,0 +1,68 @@
#!/usr/bin/env bash
# Generate the bot-link mTLS material for the TEST contour: a private CA, a gateway
# server leaf and a bot client leaf. The gateway requires the leaf + the bot the
# client leaf to bring up the reverse bot-link; the CA signs both so each peer trusts
# only the other.
#
# Production certificates come from PROD_ secrets, NOT this script. Private keys never
# leave the host/secrets; deploy/certs/ is gitignored. The script is idempotent: it
# reuses an existing CA + leaves unless --force is passed (rotation re-mints the
# leaves from the same long-lived CA).
#
# Usage:
# deploy/gen-certs.sh [--force] [dir]
# Env:
# BOTLINK_GATEWAY_NAME gateway certificate SAN/CN (default: gateway, the compose
# service name the bot dials in the test contour)
set -euo pipefail
force=0
dir=""
for arg in "$@"; do
case "$arg" in
--force) force=1 ;;
*) dir="$arg" ;;
esac
done
dir="${dir:-$(cd "$(dirname "$0")" && pwd)/certs}"
gw_name="${BOTLINK_GATEWAY_NAME:-gateway}"
mkdir -p "$dir"
cd "$dir"
if [[ -f gateway.crt && -f bot.crt && "$force" -ne 1 ]]; then
echo "gen-certs: certificates already present in $dir (use --force to rotate the leaves)"
exit 0
fi
# CA: long-lived (10y), reused across leaf rotations.
if [[ ! -f ca.crt || ! -f ca.key ]]; then
openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:prime256v1 -nodes \
-keyout ca.key -out ca.crt -days 3650 -subj "/CN=scrabble-botlink-ca"
echo "gen-certs: created CA"
fi
# Gateway server leaf (SAN matches the name the bot dials).
openssl req -newkey ec -pkeyopt ec_paramgen_curve:prime256v1 -nodes \
-keyout gateway.key -out gateway.csr -subj "/CN=${gw_name}"
openssl x509 -req -in gateway.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
-out gateway.crt -days 825 \
-extfile <(printf "subjectAltName=DNS:%s,DNS:localhost\nextendedKeyUsage=serverAuth\nkeyUsage=critical,digitalSignature\n" "$gw_name")
# Bot client leaf.
openssl req -newkey ec -pkeyopt ec_paramgen_curve:prime256v1 -nodes \
-keyout bot.key -out bot.csr -subj "/CN=scrabble-bot"
openssl x509 -req -in bot.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
-out bot.crt -days 825 \
-extfile <(printf "extendedKeyUsage=clientAuth\nkeyUsage=critical,digitalSignature\n")
rm -f gateway.csr bot.csr ca.srl
# The gateway and bot run on distroless **nonroot** (UID 65532) and bind-mount this
# dir read-only; a key owned by the deploy user must still be readable by that UID, so
# the leaves are world-readable (0644, like the .crt files). These are ephemeral
# TEST certificates regenerated every deploy on the trusted runner host; production
# keys come from PROD_ secrets, not this script. The CA key never enters a container —
# keep it owner-only.
chmod 644 ./ca.crt ./gateway.crt ./gateway.key ./bot.crt ./bot.key
chmod 600 ./ca.key
echo "gen-certs: wrote ca.crt, gateway.crt/key (CN=${gw_name}), bot.crt/key to $dir"
+143
View File
@@ -0,0 +1,143 @@
#!/usr/bin/env bash
# Production main-host deploy driver. Runs ON the main host, invoked over SSH by
# .gitea/workflows/prod-deploy.yaml as the deploy user (which must already be
# `docker login`ed to the registry). It pulls the images at the new tag and rolls
# the stack ONE service at a time in dependency order (least -> most dependent),
# health-checking after each; any failure rolls the whole stack back to the
# previously deployed tag.
#
# A schema migration adds a maintenance window: the backend (the only writer) is
# stopped so a consistent pg_dump is taken before the new backend migrates forward.
# Image rollback alone is safe under the expand-contract migration rule, so the
# automatic rollback never touches the database; the dump is kept for a MANUAL
# restore if a migration turned out to be destructive (see deploy/prod/README.md).
#
# Required env (exported by the workflow over SSH):
# REGISTRY registry namespace, e.g. docker.iliadenisov.ru/developer
# TAG new image tag (the deployed git SHA)
# PREV_TAG previously deployed tag, or "none" on the first deploy
# MIGRATION "1" when the deploy carries a schema migration, else "0"
# Optional: COMPOSE_DIR ENV_FILE DUMP_DIR STATE_FILE POSTGRES_USER POSTGRES_DB
set -uo pipefail
# Runtime compose vars (POSTGRES_*, GM_*, GRAFANA_*, CADDY_*, TELEGRAM_*, REGISTRY,
# SCRABBLE_CONFIG_DIR, ...) come from a shell-sourceable env file the workflow writes
# with single-quoted values. Exporting them into the process environment lets compose
# interpolate ${...} without re-parsing the value — a plain --env-file would mangle the
# literal '$' in the bcrypt GM_BASICAUTH_HASH.
ENV_FILE="${ENV_FILE:-/opt/scrabble/env.sh}"
# shellcheck disable=SC1090
[ -f "$ENV_FILE" ] && . "$ENV_FILE"
REGISTRY="${REGISTRY:?REGISTRY required (env.sh)}"
TAG="${TAG:?TAG required}"
PREV_TAG="${PREV_TAG:-none}"
MIGRATION="${MIGRATION:-0}"
COMPOSE_DIR="${COMPOSE_DIR:-/opt/scrabble/compose}"
DUMP_DIR="${DUMP_DIR:-/opt/scrabble/dumps}"
STATE_FILE="${STATE_FILE:-/opt/scrabble/DEPLOYED_TAG}"
# The prior deployed tag, preserved on every successful deploy so prod-rollback can
# target "the previous version" with no operator input.
PREV_STATE_FILE="${PREV_STATE_FILE:-/opt/scrabble/PREVIOUS_TAG}"
PG_USER="${POSTGRES_USER:-scrabble}"
PG_DB="${POSTGRES_DB:-scrabble}"
cd "$COMPOSE_DIR" || { echo "compose dir $COMPOSE_DIR missing"; exit 1; }
export REGISTRY
# otelcol joins the host docker group to read the socket; the GID varies per host.
DOCKER_GID="$(getent group docker | cut -d: -f3)"
export DOCKER_GID
dc() { docker compose -f docker-compose.yml -f docker-compose.prod.yml "$@"; }
use_tag() { export TAG="$1"; }
# --- health probes (one-off containers on the contour networks, like CI) --------
_probe() { docker run --rm --network "$1" alpine:3.20 wget -q -T 5 -O /dev/null "$2"; }
health_backend() { for _ in $(seq 1 20); do _probe scrabble-internal http://backend:8080/readyz && return 0; sleep 3; done; return 1; }
health_landing() { for _ in $(seq 1 20); do _probe scrabble-internal http://landing:80/ && return 0; sleep 3; done; return 1; }
health_postgres() { for _ in $(seq 1 30); do [ "$(docker inspect -f '{{.State.Health.Status}}' scrabble-postgres 2>/dev/null)" = healthy ] && return 0; sleep 2; done; return 1; }
health_running() { # health_running <container>: running, not restarting, stable restart count
local n="$1" s r c1 c2
for _ in $(seq 1 20); do
s="$(docker inspect -f '{{.State.Status}}' "$n" 2>/dev/null || echo missing)"
r="$(docker inspect -f '{{.State.Restarting}}' "$n" 2>/dev/null || echo true)"
if [ "$s" = running ] && [ "$r" = false ]; then
c1="$(docker inspect -f '{{.RestartCount}}' "$n")"; sleep 5
c2="$(docker inspect -f '{{.RestartCount}}' "$n")"
[ "$c1" = "$c2" ] && return 0
fi
sleep 3
done
return 1
}
roll() { # roll <service> <health-cmd...>
local svc="$1"; shift
echo ">>> rolling $svc -> $TAG"
dc up -d --no-build --no-deps "$svc" || return 1
"$@" || { echo "!!! $svc failed health check"; return 1; }
echo "<<< $svc healthy"
}
rollback() {
echo "########## ROLLBACK -> $PREV_TAG ##########"
if [ "$PREV_TAG" = none ]; then
echo "no previous tag (first deploy): cannot roll back; leaving the stack up for inspection."
return
fi
use_tag "$PREV_TAG"
dc up -d --no-build --remove-orphans
echo "rolled back to $PREV_TAG."
[ "$MIGRATION" = 1 ] && echo "NOTE: the DB is forward-migrated; a pre-deploy dump is in $DUMP_DIR — restore manually ONLY if the migration was destructive (see deploy/README.md, prod runbook)."
}
commit_tag() {
# Record the just-deployed tag as current, preserving the prior one as previous.
[ -f "$STATE_FILE" ] && cp "$STATE_FILE" "$PREV_STATE_FILE"
echo "$TAG" > "$STATE_FILE"
}
mkdir -p "$DUMP_DIR"
echo "=== prod deploy: tag=$TAG prev=$PREV_TAG migration=$MIGRATION ==="
use_tag "$TAG"
dc pull
# First deploy: nothing to roll from; bring the whole stack up and gate on health.
if [ -z "$(docker ps -aq -f name=scrabble-backend)" ]; then
echo "first deploy: bringing the whole stack up"
dc up -d --no-build --remove-orphans || { echo "compose up failed"; exit 1; }
health_backend || { echo "backend not ready"; exit 1; }
health_landing || { echo "landing not ready"; exit 1; }
commit_tag
echo "first deploy healthy ($TAG)."
exit 0
fi
# Migration deploy: freeze writes and snapshot a consistent dump before migrating.
if [ "$MIGRATION" = 1 ]; then
echo "migration deploy: opening maintenance window (stopping the backend = the only writer)"
dc stop backend
dump="$DUMP_DIR/pre-$TAG-$(date +%Y%m%d-%H%M%S).sql"
if ! docker exec scrabble-postgres pg_dump -U "$PG_USER" -d "$PG_DB" -n backend > "$dump"; then
echo "pg_dump failed; restarting the old backend and aborting"
dc start backend
exit 1
fi
echo "consistent dump: $dump"
fi
# Roll one service at a time, least -> most dependent; any failure rolls everything back.
roll postgres health_postgres || { rollback; exit 1; }
roll backend health_backend || { rollback; exit 1; }
roll gateway health_running scrabble-gateway || { rollback; exit 1; }
roll landing health_landing || { rollback; exit 1; }
roll validator health_running scrabble-telegram-validator || { rollback; exit 1; }
roll caddy health_running scrabble-caddy || { rollback; exit 1; }
# Observability + node_exporter: bring up the remainder and pick up any config changes.
dc up -d --no-build --remove-orphans || { rollback; exit 1; }
# Final internal sanity before committing the new tag.
health_backend || { rollback; exit 1; }
commit_tag
echo "=== deploy healthy ($TAG) ==="
+5
View File
@@ -18,3 +18,8 @@ scrape_configs:
- job_name: postgres_exporter - job_name: postgres_exporter
static_configs: static_configs:
- targets: ["postgres_exporter:9187"] - targets: ["postgres_exporter:9187"]
# Host-level metrics (memory/CPU/disk). Matters most on the prod main host's tight
# 1.9 GiB budget, where total host memory is the OOM-proximity signal.
- job_name: node
static_configs:
- targets: ["node_exporter:9100"]
+178 -61
View File
@@ -2,10 +2,8 @@
Source of truth for the platform architecture, transport, security model and Source of truth for the platform architecture, transport, security model and
cross-service contracts. User-visible behaviour per domain lives in cross-service contracts. User-visible behaviour per domain lives in
[`FUNCTIONAL.md`](FUNCTIONAL.md); the staged build order lives in [`FUNCTIONAL.md`](FUNCTIONAL.md). This document always describes the **current**
[`../PLAN.md`](../PLAN.md). This document always describes the **current** design, not the history of how it was reached.
design, not the history of how it was reached. Sections describing
not-yet-implemented components are marked *(planned)*.
## 1. Overview ## 1. Overview
@@ -45,16 +43,23 @@ Three executables plus per-platform side-services:
and a client **board-style** setting (bonus-label and a client **board-style** setting (bonus-label
mode). The visual/interaction design system is documented in mode). The visual/interaction design system is documented in
[`UI_DESIGN.md`](UI_DESIGN.md). [`UI_DESIGN.md`](UI_DESIGN.md).
- **`platform/telegram`** — the Telegram side-service (the "connector", module - **`platform/telegram`** — the Telegram side-service (module
`scrabble/platform/telegram`). It is the only component holding the bot token — **one `scrabble/platform/telegram`), split into two binaries that share the bot token
unified bot** (one token + one optional game channel, §3). It (**one bot**, one optional game channel, §3):
runs a Bot API long-poll loop (Mini App launch + `/start` deep-links) and serves - the **validator** (`cmd/validator`) verifies Mini App initData and Login Widget
a gRPC API (`pkg/proto/telegram/v1`) that `gateway` (Mini App initData validation data by HMAC (the bot token is the secret) and **never reaches the Bot API**, so
and out-of-app push) and `backend` (operator broadcasts) call over the it runs on the main host with no VPN. The gateway calls its gRPC API
trusted internal network. Its generic delivery methods are **platform-agnostic** (`pkg/proto/telegram/v1`) over the trusted internal network during Telegram auth,
(keyed by the identity `external_id`), so a future VK/MAX connector reuses them; only so **game login is independent of Telegram reachability** (§10).
initData validation is Telegram-specific. It runs in its own container, egressing to - the **bot** (`cmd/bot`) runs the Bot API long-poll (Mini App launch + `/start`
Telegram through a VPN sidecar. deep-links) and `sendMessage`, the only component reaching the Telegram Bot API.
It holds **no inbound port**: it dials the gateway over a reverse **mTLS bot-link**
(`pkg/proto/botlink/v1`) and executes the send commands the gateway pushes
(out-of-app push, operator broadcasts), so its egress lives on a host with native
Telegram access off the main host — a VPN sidecar in the test contour, a separate
host in prod (§12). Its delivery commands are **platform-agnostic** (keyed by the
identity `external_id`), so a future VK/MAX bot reuses them; only initData
validation is Telegram-specific.
```mermaid ```mermaid
flowchart LR flowchart LR
@@ -64,9 +69,11 @@ flowchart LR
Gateway -- in-app stream --> Client Gateway -- in-app stream --> Client
Backend -- pgx --> Postgres[(Postgres)] Backend -- pgx --> Postgres[(Postgres)]
Backend -. embeds .- Solver[[scrabble-solver library]] Backend -. embeds .- Solver[[scrabble-solver library]]
Gateway -- gRPC (validate initData, out-of-app push) --> Telegram[Telegram connector] Gateway -- gRPC (validate initData) --> Validator[Telegram validator]
Backend -. operator broadcasts (gRPC) .-> Telegram Bot[Telegram bot] -. dials, reverse mTLS bot-link .-> Gateway
Telegram -- Bot API (via VPN sidecar) --> TgCloud((Telegram)) Gateway -- send commands (out-of-app push, broadcasts) --> Bot
Backend -. operator broadcasts (gRPC relay) .-> Gateway
Bot -- Bot API --> TgCloud((Telegram))
``` ```
The MVP runs `gateway` and `backend` as single-instance processes inside a The MVP runs `gateway` and `backend` as single-instance processes inside a
@@ -119,7 +126,11 @@ dropped). Horizontal scaling is explicit future work.
and GCG are unaffected** (they stay decoded concrete characters, §9.1). and GCG are unaffected** (they stay decoded concrete characters, §9.1).
- **gateway ↔ backend (sync)**: plain HTTP REST/JSON. The gateway injects - **gateway ↔ backend (sync)**: plain HTTP REST/JSON. The gateway injects
`X-User-ID` for authenticated requests; `backend` never re-derives identity `X-User-ID` for authenticated requests; `backend` never re-derives identity
from the body. from the body. Because every sync call targets the one backend host, the
gateway's REST client widens its keep-alive pool well past the stdlib default
of 2 idle connections per host; otherwise the per-request connection churn
exhausts ephemeral ports and burns gateway CPU under load (see
[`../loadtest/REPORT.md`](../loadtest/REPORT.md)).
- **backend → gateway (live)**: a single gRPC server-stream carries live events - **backend → gateway (live)**: a single gRPC server-stream carries live events
(your-turn, opponent-moved, chat, nudge). The gateway bridges them to the (your-turn, opponent-moved, chat, nudge). The gateway bridges them to the
client's in-app stream while the app is open. Out-of-app delivery uses client's in-app stream while the app is open. Out-of-app delivery uses
@@ -132,15 +143,18 @@ signing, no anti-replay crypto** (these were considered and dropped — players
arrive from a platform rather than completing a mandatory registration). arrive from a platform rather than completing a mandatory registration).
- The gateway validates the originating credential **once** — Telegram `initData` - The gateway validates the originating credential **once** — Telegram `initData`
(delegated to the connector's `ValidateInitData` RPC, which holds the bot token — (delegated to the **validator's** `ValidateInitData` RPC, which holds the bot token —
the HMAC secret — so it never reaches the gateway), an email-code login, or a guest the HMAC secret — so it never reaches the gateway), an email-code login, or a guest
bootstrap — then mints a **thin opaque server session token** (`session_id`). First bootstrap — then mints a **thin opaque server session token** (`session_id`). First
Telegram contact seeds the new account's language (from the launch `language_code`) Telegram contact seeds the new account's language (from the launch `language_code`)
and display name (§4). and display name (§4). The validator runs on the main host and never reaches the Bot
- **Single bot.** The connector hosts **one unified bot** (one token + one optional API, so login does not depend on Telegram or the remote bot being up (§10, §12).
game channel). `ValidateInitData` validates `initData` against that single token and - **Single bot.** The platform side-service runs **one bot** (one token + one optional
returns only the Telegram user identity — there is no per-bot "service language" and no game channel), split into a home **validator** and a remote **bot** that share the
supported-languages set on the wire. The bot's chat messages and out-of-app push are token. `ValidateInitData` (the validator) validates `initData` against that single
token and returns only the Telegram user identity — there is no per-bot "service
language" and no supported-languages set on the wire. The bot's chat messages and
out-of-app push are
rendered in the recipient's **interface language** (`preferred_language`, en/ru), not in rendered in the recipient's **interface language** (`preferred_language`, en/ru), not in
any bot-scoped language, and the friend-invite **share link** (and its caption) point at any bot-scoped language, and the friend-invite **share link** (and its caption) point at
that one bot. First Telegram contact seeds the new account's `preferred_language` from the that one bot. First Telegram contact seeds the new account's `preferred_language` from the
@@ -207,7 +221,7 @@ arrive from a platform rather than completing a mandatory registration).
payment; no purchase flow yet) is carried on the account and ORed on a merge. payment; no purchase flow yet) is carried on the account and ORed on a merge.
- **Linking** is initiated from an authenticated profile and proves - **Linking** is initiated from an authenticated profile and proves
control of the identity before attaching it: **email** through the confirm-code control of the identity before attaching it: **email** through the confirm-code
flow, **Telegram** through the web **Login Widget** (validated by the connector, flow, **Telegram** through the web **Login Widget** (validated by the validator,
HMAC under `SHA-256(bot_token)` — distinct from Mini App initData; the gateway HMAC under `SHA-256(bot_token)` — distinct from Mini App initData; the gateway
passes the trusted `external_id` to the backend, as for `auth.telegram`). The passes the trusted `external_id` to the backend, as for `auth.telegram`). The
request step **always** sends/accepts the proof (no pre-send "already taken" request step **always** sends/accepts the proof (no pre-send "already taken"
@@ -273,9 +287,20 @@ Key points:
skipping the `.staging/` upload area) and restores the active pointer from skipping the `.staging/` upload area) and restores the active pointer from
`dictionary_state`. The volume preserves uploaded versions across redeploys; `dictionary_state`. The volume preserves uploaded versions across redeploys;
once seeded it is not re-seeded, so after bootstrap dictionary changes go through once seeded it is not re-seeded, so after bootstrap dictionary changes go through
the console rather than a rebuild. (The dictionaries ship as a versioned the console rather than a rebuild. Because the flat DAWGs carry no embedded
**release artifact** from the `scrabble-dictionary` repo; the build's version, `OpenWithVersions` records the version the flat directory was first seeded
`DICT_VERSION` selects only the seed.) at in a `.seed_version` marker on the volume and treats that marker as
**authoritative** (the **seed-drift guard**): on an already-seeded volume a later
`BACKEND_DICT_VERSION` is ignored, so a bumped build seed cannot relabel the
already-seeded bytes — which would otherwise silently serve the wrong dictionary
and void games pinned to the prior label. A running contour therefore moves to a
new release **through the console** (the prior version stays resident, so its games
keep replaying), and `DICT_VERSION` is the seed for a **fresh** volume only:
bumping it on a live contour is a harmless no-op that takes effect on the next
fresh volume. Set it per contour from the deploy's `TEST_`/`PROD_DICT_VERSION`.
(The dictionaries ship as a versioned **release artifact** from the
`scrabble-dictionary` repo; the build's `DICT_VERSION` selects
only the seed.)
- Move generation/validation/scoring use `Solver.GenerateMoves` (ranked), - Move generation/validation/scoring use `Solver.GenerateMoves` (ranked),
`Solver.ValidatePlay` and `Solver.ScorePlay`; board mutation uses `Solver.ValidatePlay` and `Solver.ScorePlay`; board mutation uses
`scrabble.Apply`. The engine adds its own deterministic, seeded tile **bag** `scrabble.Apply`. The engine adds its own deterministic, seeded tile **bag**
@@ -789,18 +814,26 @@ missed while the app was hidden. **Out-of-app platform push** is a fallback
the **gateway** routes from the same firehose: for an event whose recipient has **no the **gateway** routes from the same firehose: for an event whose recipient has **no
live in-app stream** it resolves the backend `/internal/push-target` (their Telegram live in-app stream** it resolves the backend `/internal/push-target` (their Telegram
`external_id`, the recipient's **interface language** (`preferred_language`) as the render `external_id`, the recipient's **interface language** (`preferred_language`) as the render
language, and the `notifications_in_app_only` flag). It then asks the **Telegram connector** language, and the `notifications_in_app_only` flag). It then pushes a deliver command over
to deliver — through the **single bot** a the **bot-link** to the remote **bot****fire-and-forget, best-effort** (dropped, with a
localized message with a Mini App deep-link button, only when the recipient has a Telegram metric, when no bot is connected) — only when the recipient has a Telegram identity and has
identity and has not confined notifications to the app, so the two channels never duplicate. The not confined notifications to the app, so the two channels never duplicate. The bot renders a
connector renders the message in that language; there is no per-bot routing. The out-of-app set is localized message with a Mini App deep-link button in that language; there is no per-bot
routing. The out-of-app set is
your-turn, game-over, nudge and the **invitation** (a new invitation) / friend-request notify sub-kinds; your-turn, game-over, nudge and the **invitation** (a new invitation) / friend-request notify sub-kinds;
the connector renders the message and skips the rest — so in-app-only sub-kinds like the bot renders the message and skips the rest — so in-app-only sub-kinds like
**invitation-update** (a response/withdrawal lobby sync) and **user-blocked/-unblocked** (a **invitation-update** (a response/withdrawal lobby sync) and **user-blocked/-unblocked** (a
block-state sync to the blocker) never become a platform push. Operator broadcasts block-state sync to the blocker) never become a platform push. Operator broadcasts
(`SendToUser` / `SendToGameChannel`, §10 admin) render in an **operator-chosen** language in (`SendToUser` / `SendToGameChannel`, §10 admin) render in an **operator-chosen** language in
the console, sent through the same single bot. Session-revocation events and the console; the backend calls them on the **gateway's bot-link relay**, which forwards them
cursor-based stream resume stay deferred (single-instance MVP). to the bot and **awaits its delivery ack** (so the console still reports delivered/not). Beyond
messages the same bot-link carries a **chat-gate control path** — a `ChatGate` command sets a user's
write access in the moderated discussion chat and the bot's unary `ResolveChatEligibility` resolves a
joiner's eligibility (neither renders a message; see *Moderated discussion chat* below). An optional
**standalone promo bot** runs in the bot container (`TELEGRAM_PROMO_BOT_TOKEN`): a second bot
answering `/start` with a URL button into the **main** bot's Mini App (`?startapp`, since a `web_app`
button would sign initData with the promo token); it is self-contained — no bot-link, no gateway.
Session-revocation events and cursor-based stream resume stay deferred (single-instance MVP).
A separate **advertising-banner** channel feeds the client's one-line strip (UI_DESIGN.md), A separate **advertising-banner** channel feeds the client's one-line strip (UI_DESIGN.md),
server-driven by `internal/ads`. An operator manages **campaigns** (each one placement order) in server-driven by `internal/ads`. An operator manages **campaigns** (each one placement order) in
@@ -834,14 +867,15 @@ edits take effect on the next `profile.get` (open/reconnect/foreground), not mid
## 11. Observability ## 11. Observability
- Structured logging with `go.uber.org/zap` (JSON). OpenTelemetry tracer and - Structured logging with `go.uber.org/zap` (JSON). OpenTelemetry tracer and
meter providers are wired in **all three services** (backend, gateway, the meter providers are wired in **all services** (backend, gateway, the Telegram
Telegram connector) through a shared `pkg/telemetry` bootstrap, env-gated per validator and bot) through a shared `pkg/telemetry` bootstrap, env-gated per
service by `{BACKEND,GATEWAY,TELEGRAM}_OTEL_{TRACES,METRICS}_EXPORTER` with a service by `{BACKEND,GATEWAY,TELEGRAM}_OTEL_{TRACES,METRICS}_EXPORTER` with a
default of `none` (so no collector is required locally or in CI). `stdout` is default of `none` (so no collector is required locally or in CI). `stdout` is
available for debugging; **`otlp`** (gRPC, endpoint from the standard available for debugging; **`otlp`** (gRPC, endpoint from the standard
`OTEL_EXPORTER_OTLP_*` environment) exports to a collector. The Postgres pool is `OTEL_EXPORTER_OTLP_*` environment) exports to a collector. The Postgres pool is
instrumented with otelsql and `otelgrpc` traces the backend↔gateway push stream instrumented with otelsql and `otelgrpc` traces the backend↔gateway push stream
and the gateway↔connector calls. The OTLP **Collector** (OTLP/gRPC → Prometheus and the gateway↔validator and bot-link calls; the gateway also exports
`botlink_connected_bots` and `botlink_commands_total` (by result) for the bot-link. The OTLP **Collector** (OTLP/gRPC → Prometheus
metrics + Tempo traces), **Prometheus** (15d), **Tempo** (72h) and **Grafana** metrics + Tempo traces), **Prometheus** (15d), **Tempo** (72h) and **Grafana**
(provisioned datasources + dashboards, behind the caddy `/_gm/grafana` Basic-Auth) (provisioned datasources + dashboards, behind the caddy `/_gm/grafana` Basic-Auth)
are stood up with the deploy (`deploy/`); the default exporter stays are stood up with the deploy (`deploy/`); the default exporter stays
@@ -896,6 +930,28 @@ edits take effect on the next `profile.get` (open/reconnect/foreground), not mid
list/detail, cleared by the operator, **never an automatic ban** and never a request list/detail, cleared by the operator, **never an automatic ban** and never a request
gate. The Edge/UX dashboard graphs the aggregate request rate against the rejection gate. The Edge/UX dashboard graphs the aggregate request rate against the rejection
rate by class. rate by class.
- **Temporary IP ban (prod-only):** with `GATEWAY_ABUSE_BAN_ENABLED` set, the gateway
enforces a fail2ban-style block keyed by client IP, fed by three signals: an IP that
sustains `GATEWAY_ABUSE_BAN_THRESHOLD` rate-limiter rejections within
`GATEWAY_ABUSE_BAN_WINDOW` (the IP-keyed public/email/admin classes — the user class
stays the soft-flag's concern, never the ban's), a **honeypot** decoy-path hit, and a
**honeytoken** (a planted bearer no real client holds, `GATEWAY_HONEYTOKEN`). A banned
IP is refused with **429** by an edge middleware (`abuseGuard`) before any work —
covering the Connect edge, the live stream and the static SPA/landing the per-op limiter
never gated. A rejection ban lasts `GATEWAY_ABUSE_BAN_DURATION`; a tripwire/honeytoken
hit is near-zero-false-positive and earns a longer fixed ban (1 h / 24 h). The ban is
**in-memory, single-instance and resets on restart**, like `ratewatch`; each ban
increments `gateway_abuse_banned_total` (`reason` = rejections/tripwire/honeytoken). The
decoy paths live only in the contour **caddy**, which tags them with `X-Scrabble-Honeypot`
(stripping any client-supplied value) and routes them to the gateway. It is **off by
default and only enabled in prod**: the ban keys by real client IP, which the shared-NAT
test contour does not expose (every client arrives as one address), so a ban there would
be self-inflicted — the honeypot/honeytoken still **log** in the contour, only the ban
*action* is gated. Operators see the active bans and lift them on the admin console's
**Throttled** page; the gateway syncs its active set to the backend every 30 s
(`POST /api/v1/internal/bans/sync`, network-trusted like the rejection report) and applies
the operator unbans the response returns, so a manual unban takes effect within the sync
interval.
- Unauthenticated `GET /healthz` (liveness) and `GET /readyz` (readiness — the - Unauthenticated `GET /healthz` (liveness) and `GET /readyz` (readiness — the
database answers a bounded ping and the session cache is warmed). database answers a bounded ping and the session cache is warmed).
- The backend serves a **second listener** — a gRPC server - The backend serves a **second listener** — a gRPC server
@@ -906,19 +962,22 @@ edits take effect on the next `profile.get` (open/reconnect/foreground), not mid
| Concern | Enforced by | | Concern | Enforced by |
| --- | --- | | --- | --- |
| Public rate limiting / anti-abuse | gateway (per-IP public/email/admin classes, per-user authenticated class; a request body cap of `GATEWAY_MAX_BODY_BYTES`; rejections are metered, summarised to the backend and surfaced in the admin console with a conservative reversible auto-flag — §11) | | Public rate limiting / anti-abuse | gateway (per-IP public/email/admin classes, per-user authenticated class; a request body cap of `GATEWAY_MAX_BODY_BYTES`; rejections are metered, summarised to the backend and surfaced in the admin console with a conservative reversible auto-flag — §11). In prod a **temporary IP ban** (`GATEWAY_ABUSE_BAN_ENABLED`) blocks an IP that sustains rejections or trips a **honeypot** decoy path / **honeytoken**, refused with 429 before any work; operators lift bans from the console. Off in the shared-NAT test contour, where the client IP is not real (§11) |
| Telegram initData validation (bot-token HMAC) | the Telegram connector; the gateway delegates it over gRPC, so the bot token lives only in the connector | | Telegram initData validation (bot-token HMAC) | the Telegram **validator**; the gateway delegates it over gRPC, so the bot token (the HMAC secret) lives only in the validator and the bot, never in the gateway |
| Session minting; email-code / guest validation | gateway (with backend) | | Session minting; email-code / guest validation | gateway (with backend) |
| Session → `user_id` resolution, `X-User-ID` injection | gateway | | Session → `user_id` resolution, `X-User-ID` injection | gateway |
| Authorisation, ownership, state transitions | backend (`X-User-ID` is the sole identity input) | | Authorisation, ownership, state transitions | backend (`X-User-ID` is the sole identity input) |
| Manual account block (suspension) | backend: a per-request gate refuses a blocked account on every `/api/v1/user/*` route except the block-status probe with **403 `account_blocked`**; the operator blocks/unblocks from the admin console (§11) | | Manual account block (suspension) | backend: a per-request gate refuses a blocked account on every `/api/v1/user/*` route except the block-status probe with **403 `account_blocked`**; the operator blocks/unblocks from the admin console (§11) |
| User feedback gate | backend rejects a guest or a `feedback_banned` account from submitting; the **gateway** also rejects a guest's `feedback.submit` (the `Op.NonGuest` flag + `is_guest` from session resolve) with **`guest_forbidden`** before any backend call; attachments are served `nosniff` with a download disposition for non-images (§15) | | User feedback gate | backend rejects a guest or a `feedback_banned` account from submitting; the **gateway** also rejects a guest's `feedback.submit` (the `Op.NonGuest` flag + `is_guest` from session resolve) with **`guest_forbidden`** before any backend call; attachments are served `nosniff` with a download disposition for non-images (§15) |
| Admin authentication | a single Basic-Auth gate on `/_gm/*`, forwarded **verbatim** to the backend's server-rendered admin console (and, in the deployed contour, routing `/_gm/grafana/*` to Grafana). In the deploy the **caddy** owns this gate (§13); a local non-caddy run uses the gateway's own `GATEWAY_ADMIN_*` proxy, which the per-IP admin limiter class guards ahead of its Basic-Auth — the caddy-fronted path has no limiter (stock caddy), an accepted gap. The backend trusts the proxy (no admin principal) and guards its state-changing POSTs with a **same-origin** check — the console's CSRF defence. No operator identity is tracked | | Admin authentication | a single Basic-Auth gate on `/_gm/*`, forwarded **verbatim** to the backend's server-rendered admin console (and, in the deployed contour, routing `/_gm/grafana/*` to Grafana). In the deploy the **caddy** owns this gate (§13); a local non-caddy run uses the gateway's own `GATEWAY_ADMIN_*` proxy, which the per-IP admin limiter class guards ahead of its Basic-Auth — the caddy-fronted path has no limiter (stock caddy), an accepted gap. The backend trusts the proxy (no admin principal) and guards its state-changing POSTs with a **same-origin** check — the console's CSRF defence. No operator identity is tracked |
| backend ↔ gateway ↔ connector trust | the network (only gateway may reach backend; the connector serves unauthenticated gRPC on the internal segment) | | backend ↔ gateway ↔ validator trust | the network (only gateway may reach backend; the validator and the gateway's admin bot-link relay serve unauthenticated gRPC on the trusted internal segment) |
| remote bot ↔ gateway (bot-link) | **mutual TLS**: a private CA signs the gateway server cert and the bot client cert, and each verifies the other. The bot dials out (no inbound port, no static IP), so the channel is guarded solely by mTLS — the bot client key is as sensitive as the token (§13) |
This is an explicit, accepted MVP risk: compromise of the gateway↔backend This is an explicit, accepted MVP risk: compromise of the gateway↔backend
network segment defeats backend authentication. Mitigated by network isolation; network segment defeats backend authentication. Mitigated by network isolation;
mutual auth is a future hardening step. mutual auth is a future hardening step. The **bot-link** is the exception — it
already uses mutual TLS, because it is the one inter-service link that leaves the
trusted segment (the remote bot lives off the main host).
**Manual account block (suspension).** Beyond the soft, reversible high-rate flag (§11, never a **Manual account block (suspension).** Beyond the soft, reversible high-rate flag (§11, never a
gate), an operator can hard-block an account from the admin console — permanently or until a gate), an operator can hard-block an account from the admin console — permanently or until a
@@ -936,9 +995,28 @@ revoked token would fail session resolution at the gateway *before* the gate, se
login instead of the blocked screen). A block instantly **forfeits** every active game the player login instead of the blocked screen). A block instantly **forfeits** every active game the player
is in (the opponent wins, exactly as a resignation — the engine resigns off-turn) and cancels is in (the opponent wins, exactly as a resignation — the engine resigns off-turn) and cancels
their open matchmaking games; a temporary block lapses automatically once its expiry passes (no their open matchmaking games; a temporary block lapses automatically once its expiry passes (no
sweeper the gate recomputes against `now`). No operator identity is recorded (shared sweeper for the gate — it recomputes against `now`). No operator identity is recorded (shared
Basic-Auth). Basic-Auth).
**Moderated discussion chat.** A channel's linked discussion group is gated by the Telegram bot
(`TELEGRAM_CHAT_ID`). The group **allows sending by default** and the bot only **restricts**: Telegram
intersects the chat default with each user's permission, so a per-user grant can never exceed a
deny-by-default group — the gate must mute the ineligible, not grant the eligible. A user may write
while they are **registered and neither admin-suspended nor holding the chat-only `chat_muted` role**
(`eligible = registered AND NOT suspended AND NOT chat_muted` — the game suspension dominates); the bot
**mutes** an ineligible member and **un-mutes** an eligible one it had muted, leaving an already-allowed
eligible member untouched (it acts only when the current state differs, so it is idempotent and never
loops on its own change). A single backend resolver behind `POST /api/v1/internal/chat-access` answers
both directions: the bot's `ResolveChatEligibility` on a `chat_member` event (over the mTLS bot-link),
and a `chat_access_changed` event — emitted on a block/unblock, a `chat_muted` grant/revoke, a first
Telegram registration, or a temporary block lapsing (a dedicated `account.SuspensionSweeper`, since no
request fires then) — drives a `ChatGate` command the gateway pushes to the bot. The bot applies it
only to a member currently in the chat (a per-user `getChatMember` probe, since bots cannot list
members); the signal is idempotent and is never an in-app or out-of-app message. `chat_muted` is an
`account_roles` entry (an operator toggle in the console), so it needs no schema change. The bot
must be an administrator in the group with the **restrict-members** right and `chat_member` in its
allowed updates.
**Short numeric codes** (email confirm-codes and friend codes) are stored **Short numeric codes** (email confirm-codes and friend codes) are stored
only as SHA-256 hashes and are short-lived and single-use. The unauthenticated only as SHA-256 hashes and are short-lived and single-use. The unauthenticated
email path carries a tight per-IP sub-limit (5 / 10 min); the **friend-code redeem** email path carries a tight per-IP sub-limit (5 / 10 min); the **friend-code redeem**
@@ -966,17 +1044,25 @@ routes `/_gm/grafana/*` to **Grafana** (anonymous-admin, so the one shared login
it with no per-user Grafana accounts) and the rest of `/_gm/*` to the backend-rendered it with no per-user Grafana accounts) and the rest of `/_gm/*` to the backend-rendered
**admin console**; `/app/`, `/telegram/` and the Connect path go to the gateway; the **admin console**; `/app/`, `/telegram/` and the Connect path go to the gateway; the
catch-all — notably the landing at `/` — goes to the landing container. The catch-all — notably the landing at `/` — goes to the landing container. The
**Telegram connector** runs as a separate container with **no public ingress** — it **Telegram validator** runs as a separate container with **no public ingress**,
long-polls Telegram and egresses through a VPN sidecar, answering only internal gRPC. answering only internal gRPC (HMAC, no Telegram egress). The **Telegram bot** holds
no inbound port either: it dials the gateway's **bot-link** (mTLS) and egresses to
Telegram — through a VPN sidecar in the test contour, from a separate host in prod.
The gateway exposes the bot-link on a dedicated mTLS gRPC listener
(`GATEWAY_BOTLINK_ADDR`, internal-only in the test contour, published in prod) plus a
plaintext relay (`GATEWAY_BOTLINK_RELAY_ADDR`) the backend admin console calls.
The full contour (`deploy/docker-compose.yml`) runs one `gateway`, one `backend`, The full contour (`deploy/docker-compose.yml`) runs one `gateway`, one `backend`,
one Postgres, the static `landing`, the connector (+ its VPN sidecar) and the **observability stack** one Postgres, the static `landing`, the Telegram `validator` and `bot` (+ the bot's VPN
OTel Collector (OTLP/gRPC ingest → Prometheus metrics + Tempo traces) and Grafana sidecar — the `bot`+`vpn` pair is gated to a `telegram-local` compose profile so the prod
with provisioned datasources and dashboards. All three services export OTLP to the main host can omit them) and the **observability stack**
collector; the connector shares the VPN sidecar's netns, so its `AWG_CONF` must not OTel Collector (OTLP/gRPC ingest → Prometheus metrics + Tempo traces), a `node_exporter`
for host CPU/memory (the prod main host's OOM signal), and Grafana
with provisioned datasources and dashboards. All services export OTLP to the
collector; the bot shares the VPN sidecar's netns, so its `AWG_CONF` must not
carry a `DNS=` directive (that would hijack resolv.conf and stop it resolving carry a `DNS=` directive (that would hijack resolv.conf and stop it resolving
`otelcol`; without it the netns uses Docker's resolver, which resolves both `otelcol` / `gateway`; without it the netns uses Docker's resolver, which resolves
`otelcol` and `api.telegram.org`). Inter-service traffic uses a private `internal` `otelcol`, `gateway` and `api.telegram.org`). Inter-service traffic uses a private `internal`
network (project-scoped DNS); only caddy joins the shared external `edge` network network (project-scoped DNS); only caddy joins the shared external `edge` network
(alias `scrabble`). (alias `scrabble`).
@@ -990,10 +1076,41 @@ Two contours, two secret/variable prefixes (`TEST_` / `PROD_`):
private-range upstreams** (`trusted_proxies private_ranges`), so the real client IP — private-range upstreams** (`trusted_proxies private_ranges`), so the real client IP —
used for chat-moderation logging and the gateway's per-IP rate limiting — survives the used for chat-moderation logging and the gateway's per-IP rate limiting — survives the
host-caddy hop; in prod (no host caddy) public clients are untrusted and Caddy uses the host-caddy hop; in prod (no host caddy) public clients are untrusted and Caddy uses the
real peer, so the single config is correct and spoof-safe in both contours. real peer, so the single config is correct and spoof-safe in both contours. The
- **Prod**: a manual SSH deploy after `development → master`. There is no **bot-link mTLS material** (a private CA + gateway/bot leaves, CN=`gateway`) is
host caddy, so the contour ships its own caddy terminating TLS — set generated by `deploy/gen-certs.sh` before `compose up`; the bot keeps its VPN sidecar
`CADDY_SITE_ADDRESS` to the domain and the caddy does its own ACME. for Telegram egress and dials the gateway by its internal name, so the bot-link stays
on the internal network.
- **Prod**: a **manual** rollout — `.gitea/workflows/prod-deploy.yaml`, `workflow_dispatch`
only (from `master`, `confirm=deploy`), run after `development → master` is merged green.
It builds and pushes the images to the registry (`docker.iliadenisov.ru`), then deploys
over SSH onto **two hosts** provisioned by `deploy/ansible/` (docker, a non-sudo `deploy`
service account holding a dedicated CI key, key-only sshd, default-deny ufw, fail2ban):
the **main host** runs the full stack (`docker-compose.yml` + `docker-compose.prod.yml`),
the **bot host** runs only the bot (`docker-compose.bot.yml`, no VPN — native Bot API
egress, telemetry off). There is no host caddy, so the contour caddy terminates TLS —
`CADDY_SITE_ADDRESS` is the domain and caddy does its own ACME. The gateway **publishes**
the bot-link `:9443`; the remote bot dials it over mTLS (certs from `PROD_BOTLINK_*`,
ServerName `gateway`, so TLS validation is independent of the public dial address), holds
no inbound port, and login is unaffected if that host or the link is down.
`deploy/prod-deploy.sh` rolls the main stack **one service at a time in dependency order**
(postgres → backend → gateway → landing → validator → caddy), health-checking after each;
any failure **rolls the whole stack back to the previous image tag**. A **schema migration**
adds a maintenance window: the backend (the sole writer) is stopped for a consistent
`pg_dump` before the new backend migrates forward — image rollback stays DB-safe under the
expand-contract migration rule, and the dump is kept for a manual restore. The workflow runs
four visible jobs (build → deploy-main → deploy-bot → verify). Releases are git tags
`vX.Y.Z`; the version is stamped into the image tag, every binary (`-ldflags``pkg/version`
→ the `service.version` telemetry attribute) and the SPA About screen. A separate manual
**`prod-rollback`** workflow re-deploys any prior release tag (blank input = the previous
deployed version, tracked on the host) over the same rolling, health-gated path — image-only,
no DB migration. The main host is
intentionally **launch-sized** (2 vCPU / 1.9 GiB): the prod overlay trims the baseline limits
(`GOMAXPROCS=2`, smaller caps, 7d Prometheus retention) and a **node_exporter** feeds
host-memory metrics to Grafana so it can be resized reactively as players arrive.
`GATEWAY_ABUSE_BAN_ENABLED=true` in prod (the per-IP ban is meaningful only with real
client IPs). The `vpn`+`bot` pair is gated to a `telegram-local` compose profile the test
contour activates; the prod main host omits it.
## 14. CI & branches ## 14. CI & branches
@@ -1011,11 +1128,11 @@ Two contours, two secret/variable prefixes (`TEST_` / `PROD_`):
branch-protection required check (`CI / gate`), so a path-skipped job never blocks branch-protection required check (`CI / gate`), so a path-skipped job never blocks
a merge. a merge.
- A gated **`deploy`** job auto-rolls the **test contour** on a PR into — or a push - A gated **`deploy`** job auto-rolls the **test contour** on a PR into — or a push
to — `development` (`docker compose up -d --build` on the runner host), then probes to — `development` (it generates the bot-link certs, then `docker compose up -d
the gateway (`GET /`) **and the Telegram connector's liveness** (via --build` on the runner host), then probes the gateway (`GET /`) **and the Telegram
`docker inspect`: running, not restarting, stable restart count, with a validator's and bot's liveness** (via `docker inspect`: running, not restarting,
VPN-handshake grace period, since the connector has no public ingress and a stable restart count, with a VPN-handshake grace period, since neither has public
crash-loop is otherwise invisible). A PR into `master` is test-only; the prod ingress and a crash-loop is otherwise invisible). A PR into `master` is test-only; the prod
deploy is the manual workflow. Secrets/variables are prefixed deploy is the manual workflow. Secrets/variables are prefixed
`TEST_`/`PROD_` per contour. `TEST_`/`PROD_` per contour.
- The engine consumes `scrabble-solver` as a **published, versioned module** - The engine consumes `scrabble-solver` as a **published, versioned module**
+34 -6
View File
@@ -31,9 +31,15 @@ ephemeral guest. The gateway validates the credential once and mints a thin
session token; the backend resolves it to an internal `user_id`. A **Telegram Mini session token; the backend resolves it to an internal `user_id`. A **Telegram Mini
App** launch authenticates from the platform's signed `initData`, themes the UI to App** launch authenticates from the platform's signed `initData`, themes the UI to
the Telegram colours, and — on first contact — seeds the new account's interface the Telegram colours, and — on first contact — seeds the new account's interface
language from the Telegram client. Telegram runs a **single bot**: every player uses language from the Telegram client. If a launch cannot reach the backend (for example during a
deployment), the Mini App retries quietly and then shows a small "couldn't load" screen with a
**Retry** button, rather than dropping to the web sign-in, which has no place inside Telegram.
Telegram runs a **single bot**: every player uses
the same bot, and all of its chat and out-of-app notifications are written in the the same bot, and all of its chat and out-of-app notifications are written in the
player's own **interface language** (en/ru). Guests are session-only with restricted features player's own **interface language** (en/ru). A separate optional **promo bot** can run alongside the
main one — its only job is to answer `/start` with a short message and a button that opens the
**main** bot's app, where the player picks their game variant; it is an onboarding entry point that
touches nothing else. Guests are session-only with restricted features
(auto-match only; no friends, stats or history); an abandoned guest that never (auto-match only; no friends, stats or history); an abandoned guest that never
joined a game and has been idle past the retention window is garbage-collected. While the app is open the client joined a game and has been idle past the retention window is garbage-collected. While the app is open the client
keeps a live stream and receives in-app updates in real time — the opponent's move, keeps a live stream and receives in-app updates in real time — the opponent's move,
@@ -53,6 +59,10 @@ reconnect), and pending reads resume on their own — the interface stays usable
flashing a red banner each time. flashing a red banner each time.
### Accounts, linking & merge ### Accounts, linking & merge
_Sign-in is currently provider-only, so the in-profile linking UI is temporarily hidden; it
returns once the anonymous `/app/` guest (whose upgrade path this is) ships. The flow below
describes it for when it does._
First platform contact auto-provisions a durable account. From the profile a player First platform contact auto-provisions a durable account. From the profile a player
links an email (via a confirm code) or their Telegram (via the web sign-in); a guest links an email (via a confirm code) or their Telegram (via the web sign-in); a guest
who links their first identity becomes a durable account. The "already taken" status who links their first identity becomes a durable account. The "already taken" status
@@ -65,6 +75,10 @@ account is kept and the guest's games move into it. A merge is blocked only whil
two accounts share a game still in progress. two accounts share a game still in progress.
### Lobby & matchmaking ### Lobby & matchmaking
On a cold open the lobby greets the player with a brief **loading splash** — Scrabble tiles
spelling **ЭРУДИТ / ЗАГРУЗКА / ОЖИДАНИЕ** as a small crossword — that clears the moment the
games list is ready, so the list never flashes an "empty" state on a slow connection.
The lobby lists **my games** and offers a bottom tab bar — new game, statistics, and a The lobby lists **my games** and offers a bottom tab bar — new game, statistics, and a
**⚙️ settings** tab opening the settings hub (settings, profile, friends, about). The **⚙️ settings** tab opening the settings hub (settings, profile, friends, about). The
**my games** list groups games into three **my games** list groups games into three
@@ -285,7 +299,7 @@ release archive, preview the words added and removed per variant against the act
dictionary, then install — which writes the version, loads it and makes it active; dictionary, then install — which writes the version, loads it and makes it active;
versions are immutable and games in progress keep their own), and the **pending versions are immutable and games in progress keep their own), and the **pending
wordlist changes** derived from accepted complaints (which feed the offline rebuild wordlist changes** derived from accepted complaints (which feed the offline rebuild
and are marked applied after an update). When a Telegram connector is configured an operator can also and are marked applied after an update). When the Telegram bot channel is configured an operator can also
**message a user** (by their Telegram identity) or **post to the game channel**. **message a user** (by their Telegram identity) or **post to the game channel**.
State-changing actions are protected by a same-origin check; the console tracks no State-changing actions are protected by a same-origin check; the console tracks no
operator identity. operator identity.
@@ -295,8 +309,14 @@ recently throttled users/IPs the gateway reported (an in-memory window — it re
a backend restart) and the accounts currently carrying the soft **high-rate flag**. An a backend restart) and the accounts currently carrying the soft **high-rate flag**. An
account sustaining rejections past a tunable threshold is flagged automatically — account sustaining rejections past a tunable threshold is flagged automatically —
the marker is reversible, shown as a badge in the user list and on the user card, and the marker is reversible, shown as a badge in the user list and on the user card, and
**never blocks play**; the operator reviews and clears it from the user card. There is **never blocks play**; the operator reviews and clears it from the user card. The
no automatic ban. account flag itself is never a ban. In **production** the same page also lists the
**active IP bans** the gateway is enforcing: a temporary block of a client IP that
floods the service past a threshold, or trips a hidden **honeypot** path or a planted
**honeytoken** — a high-confidence sign of a scanner or hostile bot, never a normal
player. Each ban shows its reason and expiry with an **Unban** action; bans auto-expire
and the operator can lift one early. IP bans are a production-only safeguard — the
shared test environment cannot tell its clients apart, so it does not enforce them.
The console also lets an operator **manually block** an account — the hard counterpart to the The console also lets an operator **manually block** an account — the hard counterpart to the
soft high-rate flag. From the user card the operator blocks the account **permanently** or soft high-rate flag. From the user card the operator blocks the account **permanently** or
@@ -310,6 +330,14 @@ plus the reason when one was given, and the app stops all background traffic wit
temporary block lifts itself when it expires; the operator can also **unblock** from the user card temporary block lifts itself when it expires; the operator can also **unblock** from the user card
at any time (games already lost stay lost). at any time (games already lost stay lost).
Where the bot manages a channel's **linked discussion chat**, everyone may write by default and the
bot **mutes** a player who is **not registered** or is **blocked**, un-muting them once they register
or are unblocked. So an unregistered newcomer who comments is muted (the promo bot points them at the
game to register, after which the bot restores their voice), and a registered, unblocked player simply
writes. An operator can also **mute a player in the chat only** — a `chat_muted` role on the user card —
without a full account block; an account block mutes them in the chat regardless. Muting and unmuting
take effect for a player already in the chat; one who is not in it is unaffected until they next join.
From the user card the operator can also **top up a player's hint wallet**: an additive grant From the user card the operator can also **top up a player's hint wallet**: an additive grant
(1100 hints per action) that raises the balance shown on the card. Grants are **raise-only** (1100 hints per action) that raises the balance shown on the card. Grants are **raise-only**
the console can never lower a wallet (a player only loses hints by spending them in a game), so an the console can never lower a wallet (a player only loses hints by spending them in a game), so an
@@ -317,7 +345,7 @@ over-grant cannot be reversed there.
The console works a **feedback** queue too (`/_gm/feedback`): the messages players sent, filtered The console works a **feedback** queue too (`/_gm/feedback`): the messages players sent, filtered
**unread / read / archived** with per-user search, each shown with its sender, source, channel **unread / read / archived** with per-user search, each shown with its sender, source, channel
(with the connector bot language — en/ru — for a Telegram message), the sender's interface (with the bot language — en/ru — for a Telegram message), the sender's interface
language, IP and any attachment. The operator can mark a message read, **reply** to the player (delivered language, IP and any attachment. The operator can mark a message read, **reply** to the player (delivered
in-app), archive it, delete it, or delete every message from that player — and, alongside a delete, in-app), archive it, delete it, or delete every message from that player — and, alongside a delete,
**bar the player from feedback** (a `feedback_banned` role, distinct from a full account block: it **bar the player from feedback** (a `feedback_banned` role, distinct from a full account block: it
+35 -5
View File
@@ -32,9 +32,15 @@ top-1 подсказку, безлимитную проверку слова с
session-токен; backend сопоставляет его с внутренним `user_id`. Запуск **Telegram session-токен; backend сопоставляет его с внутренним `user_id`. Запуск **Telegram
Mini App** авторизует по подписанным `initData` платформы, перекрашивает интерфейс Mini App** авторизует по подписанным `initData` платформы, перекрашивает интерфейс
в цвета Telegram и — при первом контакте — задаёт язык интерфейса нового аккаунта по в цвета Telegram и — при первом контакте — задаёт язык интерфейса нового аккаунта по
языку Telegram-клиента. Telegram держит **единого бота**: все игроки пользуются одним языку Telegram-клиента. Если запуск не может достучаться до бэкенда (например, во время
деплоя), Mini App тихо повторяет попытки, а затем показывает небольшой экран «не удалось
загрузить» с кнопкой **Повторить**, вместо того чтобы сбрасывать на веб-вход, которому внутри
Telegram не место. Telegram держит **единого бота**: все игроки пользуются одним
и тем же ботом, а весь его чат и внеприложенческие уведомления пишутся на **языке и тем же ботом, а весь его чат и внеприложенческие уведомления пишутся на **языке
интерфейса** самого игрока (en/ru). Гость — только сессия, с урезанными функциями (только интерфейса** самого игрока (en/ru). Рядом с основным может работать отдельный опциональный
**промо-бот** — его единственная задача отвечать на `/start` коротким сообщением и кнопкой,
открывающей приложение **основного** бота, где игрок выбирает нужный вариант игры; это точка входа
для онбординга, не затрагивающая больше ничего. Гость — только сессия, с урезанными функциями (только
авто-подбор; без друзей, статистики и истории); заброшенный гость, не вошедший ни авто-подбор; без друзей, статистики и истории); заброшенный гость, не вошедший ни
в одну игру и простаивавший дольше окна удержания, удаляется сборщиком. Пока приложение открыто, клиент в одну игру и простаивавший дольше окна удержания, удаляется сборщиком. Пока приложение открыто, клиент
держит живой стрим и получает обновления в реальном времени — ход соперника, ваш ход, держит живой стрим и получает обновления в реальном времени — ход соперника, ваш ход,
@@ -54,6 +60,10 @@ Mini App** авторизует по подписанным `initData` плат
рабочим вместо красного баннера каждый раз. рабочим вместо красного баннера каждый раз.
### Аккаунты, привязка и слияние ### Аккаунты, привязка и слияние
_Вход сейчас только через провайдера, поэтому UI привязки в профиле временно скрыт; он
вернётся, когда появится анонимный `/app/`-гость (для апгрейда которого он и нужен). Описание
ниже — на этот случай._
Первый контакт с платформы заводит постоянный аккаунт. Из профиля игрок Первый контакт с платформы заводит постоянный аккаунт. Из профиля игрок
привязывает email (по confirm-коду) или свой Telegram (через веб-вход); гость, привязывает email (по confirm-коду) или свой Telegram (через веб-вход); гость,
привязавший первую личность, становится постоянным аккаунтом. Факт «личность уже привязавший первую личность, становится постоянным аккаунтом. Факт «личность уже
@@ -66,6 +76,10 @@ Mini App** авторизует по подписанным `initData` плат
запрещено, только пока у аккаунтов есть общая незавершённая игра. запрещено, только пока у аккаунтов есть общая незавершённая игра.
### Лобби и подбор ### Лобби и подбор
При холодном запуске лобби встречает игрока короткой **заставкой загрузки** — фишки Scrabble
складывают небольшой кроссворд из слов **ЭРУДИТ / ЗАГРУЗКА / ОЖИДАНИЕ** — и она исчезает, как
только список игр готов, поэтому на медленном соединении список не мигает «пустым» состоянием.
В лобби — список **мои игры** и нижний tab-bar (новая игра, статистика и вкладка В лобби — список **мои игры** и нижний tab-bar (новая игра, статистика и вкладка
**⚙️ настройки**, открывающая хаб настроек — настройки, профиль, друзья, о программе). **⚙️ настройки**, открывающая хаб настроек — настройки, профиль, друзья, о программе).
Список **мои игры** разбит на три секции — Список **мои игры** разбит на три секции —
@@ -293,7 +307,7 @@ identity, их игры) и **игры** — сводка, места, запи
активной; версии неизменяемы, а идущие партии остаются на своей) и **список ожидающих активной; версии неизменяемы, а идущие партии остаются на своей) и **список ожидающих
правок**, выведенный из принятых жалоб (он питает офлайн-пересборку и отмечается правок**, выведенный из принятых жалоб (он питает офлайн-пересборку и отмечается
применённым после обновления). Если применённым после обновления). Если
подключён Telegram-коннектор, оператор также может **написать пользователю** (по его подключён Telegram-бот, оператор также может **написать пользователю** (по его
Telegram-identity) или **отправить пост в игровой канал**. Изменяющие действия Telegram-identity) или **отправить пост в игровой канал**. Изменяющие действия
защищены проверкой same-origin; личность оператора не отслеживается. защищены проверкой same-origin; личность оператора не отслеживается.
@@ -303,7 +317,14 @@ Telegram-identity) или **отправить пост в игровой кан
флагом**. Аккаунт, устойчиво превышающий настраиваемый порог отказов, помечается флагом**. Аккаунт, устойчиво превышающий настраиваемый порог отказов, помечается
автоматически — маркер обратим, виден бейджем в списке пользователей и на карточке автоматически — маркер обратим, виден бейджем в списке пользователей и на карточке
аккаунта и **никогда не блокирует игру**; оператор рассматривает и снимает его с аккаунта и **никогда не блокирует игру**; оператор рассматривает и снимает его с
карточки пользователя. Автоматического бана нет. карточки пользователя. Сам флаг аккаунта баном не является. В **проде** та же
страница дополнительно перечисляет **активные баны по IP**, которые применяет gateway:
временную блокировку IP клиента, превысившего порог наплыва, либо задевшего скрытую
**honeypot**-ловушку или подброшенный **honeytoken** — высокодостоверный признак
сканера или враждебного бота, но не нормального игрока. У каждого бана показаны причина
и срок, рядом действие **Unban**; баны истекают сами, а оператор может снять бан раньше.
Баны по IP — защита только для прода: общий тестовый контур не различает своих клиентов,
поэтому там не применяется.
Консоль также позволяет оператору **вручную заблокировать** аккаунт — жёсткий аналог мягкого Консоль также позволяет оператору **вручную заблокировать** аккаунт — жёсткий аналог мягкого
high-rate флага. С карточки пользователя оператор блокирует аккаунт **навсегда** или **до даты** high-rate флага. С карточки пользователя оператор блокирует аккаунт **навсегда** или **до даты**
@@ -318,6 +339,15 @@ high-rate флага. С карточки пользователя операт
истечении срока; оператор также может **разблокировать** с карточки пользователя в любой момент истечении срока; оператор также может **разблокировать** с карточки пользователя в любой момент
(уже проигранные партии не возвращаются). (уже проигранные партии не возвращаются).
Там, где бот ведёт **привязанный к каналу чат-обсуждение**, по умолчанию писать может каждый, а бот
**глушит** игрока, который **не зарегистрирован** или **заблокирован**, и снимает мьют, как только тот
зарегистрируется или будет разблокирован. То есть незарегистрированного новичка, написавшего в чат,
бот глушит (промо-бот направляет его в игру зарегистрироваться, после чего бот возвращает голос), а
зарегистрированный незаблокированный игрок просто пишет. Оператор также может **замьютить игрока только
в чате** — роль `chat_muted` на карточке пользователя — без полной блокировки аккаунта; блокировка
аккаунта всё равно мьютит его в чате. Мьют и размьют срабатывают для игрока, уже находящегося в чате;
того, кого в чате нет, это не затрагивает до его следующего входа.
С карточки пользователя оператор также может **пополнить кошелёк подсказок** игрока: аддитивное С карточки пользователя оператор также может **пополнить кошелёк подсказок** игрока: аддитивное
начисление (1–100 подсказок за раз), которое **только увеличивает** баланс на карточке. Начисления начисление (1–100 подсказок за раз), которое **только увеличивает** баланс на карточке. Начисления
**только в плюс** — понизить кошелёк из консоли нельзя (игрок теряет подсказки только тратя их в **только в плюс** — понизить кошелёк из консоли нельзя (игрок теряет подсказки только тратя их в
@@ -325,7 +355,7 @@ high-rate флага. С карточки пользователя операт
Консоль ведёт и очередь **обратной связи** (`/_gm/feedback`): присланные игроками сообщения с фильтром Консоль ведёт и очередь **обратной связи** (`/_gm/feedback`): присланные игроками сообщения с фильтром
**непрочитанные / прочитанные / архив** и поиском по пользователю, каждое — с отправителем, источником, **непрочитанные / прочитанные / архив** и поиском по пользователю, каждое — с отправителем, источником,
каналом (и языком бота-коннектора — en/ru — для сообщения из Telegram), языком интерфейса отправителя, каналом (и языком бота — en/ru — для сообщения из Telegram), языком интерфейса отправителя,
IP и вложением. Оператор может пометить сообщение прочитанным, **ответить** игроку (доставка IP и вложением. Оператор может пометить сообщение прочитанным, **ответить** игроку (доставка
в приложение), отправить в архив, удалить или удалить все сообщения этого игрока — и вместе с удалением в приложение), отправить в архив, удалить или удалить все сообщения этого игрока — и вместе с удалением
**запретить игроку обратную связь** (роль `feedback_banned`, отличная от полной блокировки аккаунта: **запретить игроку обратную связь** (роль `feedback_banned`, отличная от полной блокировки аккаунта:
+8 -9
View File
@@ -121,7 +121,7 @@ tests or touching CI.
Postgres-backed `inttest` drives the **guest reaper** end to end (an abandoned guest is Postgres-backed `inttest` drives the **guest reaper** end to end (an abandoned guest is
reaped; a too-young guest, a seated guest and a durable account are kept). reaped; a too-young guest, a seated guest and a durable account are kept).
- **Load test & resource baseline** — a reusable `loadtest/` module - **Load test & resource baseline** — a reusable `loadtest/` module
(`scrabble/loadtest`) is the pre-release stress harness. It **seeds** a large account (`scrabble/loadtest`) is the stress/load harness. It **seeds** a large account
population with pre-created sessions directly in Postgres (token hashes matching population with pre-created sessions directly in Postgres (token hashes matching
`backend/internal/session`), **drives** virtual players through the edge protocol — `backend/internal/session`), **drives** virtual players through the edge protocol —
real games assembled via invitations, **mid-ranked** legal moves generated locally by real games assembled via invitations, **mid-ranked** legal moves generated locally by
@@ -133,9 +133,9 @@ tests or touching CI.
engine tests do). It is **not** part of the per-PR suite's behavioural assertions: it engine tests do). It is **not** part of the per-PR suite's behavioural assertions: it
runs ad hoc as a one-shot container against the contour, producing a trip report (bugs runs ad hoc as a one-shot container against the contour, producing a trip report (bugs
+ a per-container resource profile) read off the **otelcol `docker_stats` + + a per-container resource profile) read off the **otelcol `docker_stats` +
postgres_exporter** Grafana dashboard on the contour. Two passes are recorded — the postgres_exporter** Grafana dashboard on the contour. The findings — including the
early [`REPORT-R2.md`](../loadtest/REPORT-R2.md) and the final, tuned `game.evaluate` hot-path model and the gateway→backend connection-pool fix — are written
[`REPORT-R7.md`](../loadtest/REPORT-R7.md). See [`../loadtest/README.md`](../loadtest/README.md). up in [`REPORT.md`](../loadtest/REPORT.md). See [`../loadtest/README.md`](../loadtest/README.md).
- **User feedback** — `internal/feedback` unit tests cover the attachment allow-list / - **User feedback** — `internal/feedback` unit tests cover the attachment allow-list /
content-type and the channel normaliser; the UI covers `detectChannel`, the attachment gate and content-type and the channel normaliser; the UI covers `detectChannel`, the attachment gate and
the feedback wire round-trip (`channel` / `feedback` / `codec` tests) plus a Playwright e2e the feedback wire round-trip (`channel` / `feedback` / `codec` tests) plus a Playwright e2e
@@ -154,13 +154,12 @@ tests or touching CI.
- No network or real platform calls in unit tests; validate platform - No network or real platform calls in unit tests; validate platform
credentials behind an interface seam and test with fixtures. credentials behind an interface seam and test with fixtures.
## Per-stage CI gate ## CI gate
Every completed stage is exercised on `gitea.iliadenisov.ru` before it is marked Every change is exercised on `gitea.iliadenisov.ru` before it is merged:
done in [`../PLAN.md`](../PLAN.md):
1. Commit the stage on its `feature/*` branch. 1. Commit the change on its `feature/*` branch.
2. Push to `origin`. 2. Push to `origin`.
3. Watch the run to completion — never hand-roll a poll loop: 3. Watch the run to completion — never hand-roll a poll loop:
`python3 ~/.claude/bin/gitea-ci-watch.py` (launch in the background). `python3 ~/.claude/bin/gitea-ci-watch.py` (launch in the background).
4. Only after every workflow that fired is green may the stage be marked done. 4. Only after every workflow that fired is green may the change be merged.
+27 -2
View File
@@ -20,6 +20,29 @@ the game** (`growNav`) does the nav bar grow to absorb spare height, so the stri
the title while the board and controls pin to the **bottom** for thumb reach. Every screen the title while the board and controls pin to the **bottom** for thumb reach. Every screen
except Login uses `Screen`. except Login uses `Screen`.
## Loading splash (`components/Splash.svelte`)
On a **cold app open** the lobby is the landing screen, but its game list arrives over the
network — on a slow link the empty "no games yet" line would flash before the games load. A
full-screen **tile splash** covers that gap: it lays a small Scrabble crossword out of the
words **ЭРУДИТ** / **ЗАГРУЗКА** / **ОЖИДАНИЕ**, tile by tile, until the lobby's first load
settles, then removes itself to reveal the populated list. The tiles carry their **Эрудит
point values** (hardcoded in `lib/splash.ts`, since the alphabet table the board's
`valueForLetter` reads is not cached yet at boot) and mirror a placed board tile's look
(cream stock, bottom edge, drop shadow). The words form a 6×8 crossword: ЭРУДИТ horizontal,
ЗАГРУЗКА and ОЖИДАНИЕ vertical, crossing it through the shared **Р** and **Д** (laid once).
It is an **App-level overlay** shown while `routeIsLobby && !app.splashDone` (so it also
covers the session bootstrap; a deep-link to another screen is not covered). The lobby sets
`app.lobbyReady` when its first load settles (success **or** error). Each word is laid, then
**held ~0.25 s** so it stays readable, and only after that hold does the readiness check fire —
so a word never blinks away the instant it finishes. ЭРУДИТ lays + holds over ~1.25 s, then the
splash loops ЗАГРУЗКА → ОЖИДАНИЕ (clearing back to ЭРУДИТ between rounds) until ready, so even a
fast load shows it for ~1.25 s. Each tile **drops in** with a brief scale + fade. Under **reduced motion**
(or the mock build, to keep the Playwright smoke unblocked) it shows a static ЭРУДИТ and
dismisses as soon as the lobby is ready. The pure layout and timing live in `lib/splash.ts`
(unit-tested); `Splash.svelte` is the renderer.
## Navigation ## Navigation
- **Back**: a thin, compact `<` drawn from two rotated CSS borders (`Header.svelte` - **Back**: a thin, compact `<` drawn from two rotated CSS borders (`Header.svelte`
@@ -240,8 +263,10 @@ on the right: Victory 🏆 / Defeat 🥈 / Draw 🏅, and for 34-player games
IV 🏅; active games show Your move 🟢 / Opponent's move ⏳; invitations use 💌. The score line IV 🏅; active games show Your move 🟢 / Opponent's move ⏳; invitations use 💌. The score line
lists seats in **seat-number order** (matching the over-the-board scoreboard) in a **bold**, lists seats in **seat-number order** (matching the over-the-board scoreboard) in a **bold**,
slightly smaller line; on an **in-progress** game the viewer's **own** number is tinted `--ok` slightly smaller line; on an **in-progress** game the viewer's **own** number is tinted `--ok`
when leading or tied and `--danger` when trailing (other numbers stay muted), a quick "am I when leading or tied and `--danger` when trailing, and an opponent's number is tinted `--ok` only
ahead" read that finished games leave to the place emoji. When a listed when it **ties the viewer for the lead** (so an equal non-zero score paints both numbers green);
otherwise numbers stay muted, as does a fresh **0:0** board where nobody has scored yet — a quick
"am I ahead" read that finished games leave to the place emoji. When a listed
game **becomes your turn or finishes** while the lobby is open, that status emoji **blinks game **becomes your turn or finishes** while the lobby is open, that status emoji **blinks
twice** (a two-cycle opacity fade, ~2 s; suppressed under reduce-motion) to draw the eye — the twice** (a two-cycle opacity fade, ~2 s; suppressed under reduce-motion) to draw the eye — the
opponent's-turn change is silent. Each card's blink is keyed by game id, so overlapping opponent's-turn change is silent. Each card's blink is keyed by game id, so overlapping
+3 -1
View File
@@ -70,7 +70,9 @@ RUN rm gateway/internal/webui/dist/landing.html
# Reduce the workspace to what the gateway needs: gateway + pkg (loadtest is not in # Reduce the workspace to what the gateway needs: gateway + pkg (loadtest is not in
# this context; its scrabble/gateway replace targets ./gateway, which is present here). # this context; its scrabble/gateway replace targets ./gateway, which is present here).
RUN go work edit -dropuse=./backend -dropuse=./platform/telegram -dropuse=./loadtest RUN go work edit -dropuse=./backend -dropuse=./platform/telegram -dropuse=./loadtest
RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -o /out/gateway ./gateway/cmd/gateway # VERSION (the deploy passes the git tag) is stamped into the binary via the linker.
ARG VERSION=dev
RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -ldflags "-X scrabble/pkg/version.Version=${VERSION}" -o /out/gateway ./gateway/cmd/gateway
# --- runtime ----------------------------------------------------------------- # --- runtime -----------------------------------------------------------------
FROM gcr.io/distroless/static-debian12:nonroot AS gateway FROM gcr.io/distroless/static-debian12:nonroot AS gateway
+33 -7
View File
@@ -23,7 +23,7 @@ proto/edge/v1/ # Connect envelope contract (committed generated Go)
internal/config/ # GATEWAY_* env config internal/config/ # GATEWAY_* env config
internal/backendclient/ # typed REST client (+ X-User-ID) and push gRPC client internal/backendclient/ # typed REST client (+ X-User-ID) and push gRPC client
internal/session/ # in-memory session cache (LRU/TTL, backend fallback) internal/session/ # in-memory session cache (LRU/TTL, backend fallback)
internal/ratelimit/ # token-bucket limiter (golang.org/x/time/rate) + the rejection tracker internal/ratelimit/ # token-bucket limiter (golang.org/x/time/rate) + the rejection tracker + the temporary IP banlist
internal/connector/ # gRPC client to the Telegram connector (initData validate, out-of-app push) + routing internal/connector/ # gRPC client to the Telegram connector (initData validate, out-of-app push) + routing
internal/push/ # live-event fan-out hub (per-user client streams) internal/push/ # live-event fan-out hub (per-user client streams)
internal/transcode/ # FlatBuffers<->REST bridge + message_type registry internal/transcode/ # FlatBuffers<->REST bridge + message_type registry
@@ -45,10 +45,14 @@ operations are unauthenticated and return the minted token. A unary domain
outcome rides back in `ExecuteResponse.result_code` (HTTP 200); only edge outcome rides back in `ExecuteResponse.result_code` (HTTP 200); only edge
failures become Connect error codes. failures become Connect error codes.
`auth.telegram` validates the Mini App `initData` by calling the **Telegram connector** `auth.telegram` validates the Mini App `initData` by calling the **Telegram validator**
(`GATEWAY_CONNECTOR_ADDR`), which holds the bot token; the gateway also routes (`GATEWAY_VALIDATOR_ADDR`), which holds the bot token (HMAC); out-of-app push for
out-of-app push to that connector for recipients with no live in-app stream recipients with no live in-app stream goes to the remote **bot** over the reverse **mTLS
(ARCHITECTURE.md §10). When `GATEWAY_CONNECTOR_ADDR` is unset, both are disabled. bot-link** (`GATEWAY_BOTLINK_ADDR`, fire-and-forget), and the backend admin broadcasts
arrive on the gateway's plaintext **relay** (`GATEWAY_BOTLINK_RELAY_ADDR`) which forwards
them down the same link and awaits the bot's ack (ARCHITECTURE.md §10/§12). When
`GATEWAY_VALIDATOR_ADDR` is unset Telegram auth is disabled; when `GATEWAY_BOTLINK_ADDR`
is unset the bot channel (out-of-app push + admin relay) is disabled.
The message-type catalog: `auth.telegram`, `auth.guest`, The message-type catalog: `auth.telegram`, `auth.guest`,
`auth.email.request`, `auth.email.login`, `profile.get`, `game.submit_play`, `auth.email.request`, `auth.email.login`, `profile.get`, `game.submit_play`,
@@ -63,7 +67,7 @@ refetch). The social/account/history ops —
transcode pattern (`transcode_social.go`). Account linking & merge transcode pattern (`transcode_social.go`). Account linking & merge
`link.email.request/confirm/merge` and `link.telegram.confirm/merge` `link.email.request/confirm/merge` and `link.telegram.confirm/merge`
(`transcode_link.go`); the telegram ops validate the **Login Widget** payload via the (`transcode_link.go`); the telegram ops validate the **Login Widget** payload via the
connector (`ValidateLoginWidget`) and forward the trusted `external_id`. These validator (`ValidateLoginWidget`) and forward the trusted `external_id`. These
**superseded** the former `email.bind.*` ops, which were removed. **superseded** the former `email.bind.*` ops, which were removed.
## Configuration ## Configuration
@@ -76,11 +80,20 @@ connector (`ValidateLoginWidget`) and forward the trusted `external_id`. These
| `GATEWAY_BACKEND_GRPC_ADDR` | `localhost:9090` | backend push gRPC address | | `GATEWAY_BACKEND_GRPC_ADDR` | `localhost:9090` | backend push gRPC address |
| `GATEWAY_BACKEND_TIMEOUT` | `5s` | per backend REST call | | `GATEWAY_BACKEND_TIMEOUT` | `5s` | per backend REST call |
| `GATEWAY_ADMIN_USER` / `GATEWAY_ADMIN_PASSWORD` | unset | enable + guard the admin console at `/_gm` | | `GATEWAY_ADMIN_USER` / `GATEWAY_ADMIN_PASSWORD` | unset | enable + guard the admin console at `/_gm` |
| `GATEWAY_CONNECTOR_ADDR` | unset | Telegram connector gRPC address (enables initData validation + out-of-app push) | | `GATEWAY_VALIDATOR_ADDR` | unset | Telegram validator gRPC address (enables initData / Login Widget validation) |
| `GATEWAY_BOTLINK_ADDR` | unset | reverse mTLS bot-link listener the remote bot dials (enables out-of-app push + admin relay) |
| `GATEWAY_BOTLINK_RELAY_ADDR` | unset | plaintext internal listener serving the backend admin `SendToUser`/`SendToGameChannel` relay |
| `GATEWAY_BOTLINK_TLS_CERT` / `_KEY` / `_CA` | unset | gateway server cert, its key, and the CA that signs accepted bot client certs (required when `GATEWAY_BOTLINK_ADDR` is set) |
| `GATEWAY_BOTLINK_SEND_TIMEOUT` | `5s` | admin relay wait for the bot ack before reporting not-delivered |
| `GATEWAY_SESSION_TTL` | `10m` | cached session lifetime | | `GATEWAY_SESSION_TTL` | `10m` | cached session lifetime |
| `GATEWAY_SESSION_CACHE_MAX` | `50000` | cached session cap | | `GATEWAY_SESSION_CACHE_MAX` | `50000` | cached session cap |
| `GATEWAY_PUSH_HEARTBEAT_INTERVAL` | `10s` | live-stream keep-alive (an immediate heartbeat also fires on open, under the ~15s edge idle timeout) | | `GATEWAY_PUSH_HEARTBEAT_INTERVAL` | `10s` | live-stream keep-alive (an immediate heartbeat also fires on open, under the ~15s edge idle timeout) |
| `GATEWAY_MAX_BODY_BYTES` | `1048576` | caps one request body and one Connect message read; an oversized Execute is refused with `resource_exhausted` | | `GATEWAY_MAX_BODY_BYTES` | `1048576` | caps one request body and one Connect message read; an oversized Execute is refused with `resource_exhausted` |
| `GATEWAY_ABUSE_BAN_ENABLED` | `false` | enable the temporary IP ban (prod-only — keys by real client IP, off in the shared-NAT test contour) |
| `GATEWAY_ABUSE_BAN_THRESHOLD` | `100` | rate-limiter rejections within the window that ban an IP |
| `GATEWAY_ABUSE_BAN_WINDOW` | `2m` | rolling window the rejection strikes accumulate over |
| `GATEWAY_ABUSE_BAN_DURATION` | `15m` | length of a rejection-earned ban (tripwire 1h, honeytoken 24h are fixed) |
| `GATEWAY_HONEYTOKEN` | unset | planted bearer value; presenting it bans the caller and raises an alarm |
| `GATEWAY_SERVICE_NAME` | `scrabble-gateway` | OpenTelemetry `service.name` | | `GATEWAY_SERVICE_NAME` | `scrabble-gateway` | OpenTelemetry `service.name` |
| `GATEWAY_OTEL_TRACES_EXPORTER` | `none` | `none`, `stdout` or `otlp` (gRPC; endpoint from `OTEL_EXPORTER_OTLP_*`) | | `GATEWAY_OTEL_TRACES_EXPORTER` | `none` | `none`, `stdout` or `otlp` (gRPC; endpoint from `OTEL_EXPORTER_OTLP_*`) |
| `GATEWAY_OTEL_METRICS_EXPORTER` | `none` | `none`, `stdout` or `otlp` | | `GATEWAY_OTEL_METRICS_EXPORTER` | `none` | `none`, `stdout` or `otlp` |
@@ -96,6 +109,19 @@ per-key rejection tracker every 30 s, emits a Warn summary per throttled key and
posts the report to the backend (`/api/v1/internal/ratelimit/report`), feeding posts the report to the backend (`/api/v1/internal/ratelimit/report`), feeding
the admin console's throttled view and the high-rate auto-flag. the admin console's throttled view and the high-rate auto-flag.
Temporary IP ban (prod-only, `GATEWAY_ABUSE_BAN_ENABLED`): a fail2ban-style block
keyed by client IP, fed by sustained rate-limiter rejections (the IP-keyed classes),
a **honeypot** decoy-path hit (the contour caddy tags decoy paths with
`X-Scrabble-Honeypot`), and a **honeytoken** (`GATEWAY_HONEYTOKEN`). A banned IP is
refused with 429 by the `abuseGuard` edge middleware before any work — covering the
Connect edge, the live stream and the static SPA/landing. Each ban increments
`gateway_abuse_banned_total{reason}` (`rejections`/`tripwire`/`honeytoken`). The ban
is in-memory (resets on restart); it is **off by default** because it keys by the real
client IP, which the shared-NAT test contour does not expose (detection still logs
there, only the ban action is gated). The gateway syncs its active set to the backend
every 30 s (`/api/v1/internal/bans/sync`) for the console's **Active IP bans** panel
and applies the operator unbans the response returns.
## Run ## Run
```sh ```sh
+163 -17
View File
@@ -9,16 +9,23 @@ package main
import ( import (
"context" "context"
"errors" "errors"
"fmt"
"log" "log"
"net"
"net/http" "net/http"
"os/signal" "os/signal"
"syscall" "syscall"
"time" "time"
"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
"go.uber.org/zap" "go.uber.org/zap"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
"google.golang.org/grpc/keepalive"
"scrabble/gateway/internal/admin" "scrabble/gateway/internal/admin"
"scrabble/gateway/internal/backendclient" "scrabble/gateway/internal/backendclient"
"scrabble/gateway/internal/botlink"
"scrabble/gateway/internal/config" "scrabble/gateway/internal/config"
"scrabble/gateway/internal/connector" "scrabble/gateway/internal/connector"
"scrabble/gateway/internal/connectsrv" "scrabble/gateway/internal/connectsrv"
@@ -26,9 +33,23 @@ import (
"scrabble/gateway/internal/ratelimit" "scrabble/gateway/internal/ratelimit"
"scrabble/gateway/internal/session" "scrabble/gateway/internal/session"
"scrabble/gateway/internal/transcode" "scrabble/gateway/internal/transcode"
"scrabble/pkg/mtls"
botlinkv1 "scrabble/pkg/proto/botlink/v1"
telegramv1 "scrabble/pkg/proto/telegram/v1"
pkgtel "scrabble/pkg/telemetry" pkgtel "scrabble/pkg/telemetry"
) )
const (
// botLinkKeepaliveTime is how often the gateway pings an idle bot-link stream
// to hold the WAN connection open and detect a dead bot.
botLinkKeepaliveTime = 30 * time.Second
// botLinkKeepaliveTimeout bounds the wait for a keepalive ping reply.
botLinkKeepaliveTimeout = 10 * time.Second
// botLinkMinPingInterval is the smallest client ping interval the gateway
// tolerates before treating it as abuse.
botLinkMinPingInterval = 10 * time.Second
)
const ( const (
// shutdownTimeout bounds the graceful HTTP shutdown. // shutdownTimeout bounds the graceful HTTP shutdown.
shutdownTimeout = 10 * time.Second shutdownTimeout = 10 * time.Second
@@ -47,6 +68,9 @@ const (
// throttleReportInterval is the cadence of the rate-limiter rejection // throttleReportInterval is the cadence of the rate-limiter rejection
// summary: the Warn log per throttled key and the report to the backend. // summary: the Warn log per throttled key and the report to the backend.
throttleReportInterval = 30 * time.Second throttleReportInterval = 30 * time.Second
// banSyncInterval is the cadence of the active-ban sync to the backend (which
// feeds the admin-console view) and the operator-unban pull.
banSyncInterval = 30 * time.Second
) )
func main() { func main() {
@@ -98,19 +122,59 @@ func run(ctx context.Context, cfg config.Config, logger *zap.Logger) error {
sessions := session.NewCache(backend, cfg.SessionTTL, cfg.SessionCacheMax) sessions := session.NewCache(backend, cfg.SessionTTL, cfg.SessionCacheMax)
limiter := ratelimit.New() limiter := ratelimit.New()
tracker := ratelimit.NewTracker() tracker := ratelimit.NewTracker()
banlist := ratelimit.NewBanlist(ratelimit.BanConfig{
Enabled: cfg.Abuse.BanEnabled,
Threshold: cfg.Abuse.BanThreshold,
Window: cfg.Abuse.BanWindow,
Duration: cfg.Abuse.BanDuration,
})
hub := push.NewHub(0) hub := push.NewHub(0)
var conn *connector.Client
var validator transcode.TelegramValidator var validator transcode.TelegramValidator
if cfg.ConnectorAddr != "" { if cfg.ValidatorAddr != "" {
conn, err = connector.New(cfg.ConnectorAddr) conn, cerr := connector.New(cfg.ValidatorAddr)
if err != nil { if cerr != nil {
return err return cerr
} }
defer func() { _ = conn.Close() }() defer func() { _ = conn.Close() }()
validator = conn validator = conn
} else { } else {
logger.Warn("telegram disabled (GATEWAY_CONNECTOR_ADDR unset)") logger.Warn("telegram auth disabled (GATEWAY_VALIDATOR_ADDR unset)")
}
// The reverse bot-link: the remote Telegram bot dials this gateway over mTLS and
// the gateway pushes send commands down the stream. Out-of-app push is
// fire-and-forget; the backend admin relay (plaintext, internal) awaits the Ack.
var botHub *botlink.Hub
if cfg.BotLinkEnabled() {
botHub = botlink.NewHub(logger, tel.MeterProvider().Meter("scrabble/gateway/botlink"),
func(ctx context.Context, externalID string) (bool, bool, error) {
r, rerr := backend.ChatEligibility(ctx, externalID)
return r.Registered, r.Eligible, rerr
})
tlsCfg, terr := mtls.ServerConfig(cfg.BotLink.CertFile, cfg.BotLink.KeyFile, cfg.BotLink.CAFile)
if terr != nil {
return terr
}
botSrv := grpc.NewServer(
grpc.Creds(credentials.NewTLS(tlsCfg)),
grpc.StatsHandler(otelgrpc.NewServerHandler()),
grpc.KeepaliveParams(keepalive.ServerParameters{Time: botLinkKeepaliveTime, Timeout: botLinkKeepaliveTimeout}),
grpc.KeepaliveEnforcementPolicy(keepalive.EnforcementPolicy{MinTime: botLinkMinPingInterval, PermitWithoutStream: true}),
)
botlinkv1.RegisterBotLinkServer(botSrv, botHub)
if serr := serveGRPC(ctx, "botlink", cfg.BotLink.Addr, botSrv, logger); serr != nil {
return serr
}
if cfg.BotLink.RelayAddr != "" {
relaySrv := grpc.NewServer(grpc.StatsHandler(otelgrpc.NewServerHandler()))
telegramv1.RegisterTelegramServer(relaySrv, botlink.NewRelayServer(botHub, cfg.BotLink.SendTimeout))
if serr := serveGRPC(ctx, "botlink-relay", cfg.BotLink.RelayAddr, relaySrv, logger); serr != nil {
return serr
}
}
} else {
logger.Warn("telegram bot channel disabled (GATEWAY_BOTLINK_ADDR unset)")
} }
// The admin console (backend /_gm) is fronted on the public listener behind // The admin console (backend /_gm) is fronted on the public listener behind
@@ -132,6 +196,8 @@ func run(ctx context.Context, cfg config.Config, logger *zap.Logger) error {
Sessions: sessions, Sessions: sessions,
Limiter: limiter, Limiter: limiter,
Tracker: tracker, Tracker: tracker,
Banlist: banlist,
Honeytoken: cfg.Abuse.Honeytoken,
Hub: hub, Hub: hub,
RateLimit: cfg.RateLimit, RateLimit: cfg.RateLimit,
Heartbeat: cfg.PushHeartbeatInterval, Heartbeat: cfg.PushHeartbeatInterval,
@@ -142,10 +208,15 @@ func run(ctx context.Context, cfg config.Config, logger *zap.Logger) error {
}) })
// Bridge the backend push stream into the fan-out hub (and the out-of-app // Bridge the backend push stream into the fan-out hub (and the out-of-app
// channel via the connector). // channel via the bot-link).
go runPushPump(ctx, backend, hub, conn, logger) go runPushPump(ctx, backend, hub, botHub, logger)
// Periodically summarise rate-limiter rejections (Warn log + backend report). // Periodically summarise rate-limiter rejections (Warn log + backend report).
go runThrottleReporter(ctx, tracker, backend, logger) go runThrottleReporter(ctx, tracker, backend, logger)
// When the IP ban is enabled (prod), sync the active set to the backend (the
// admin-console view) and apply the operator unbans it returns.
if cfg.Abuse.BanEnabled {
go runBanSync(ctx, banlist, backend, logger)
}
public := &http.Server{Addr: cfg.HTTPAddr, Handler: edge.HTTPHandler(), ReadHeaderTimeout: readHeaderTimeout} public := &http.Server{Addr: cfg.HTTPAddr, Handler: edge.HTTPHandler(), ReadHeaderTimeout: readHeaderTimeout}
servers := []*namedServer{{name: "public", srv: public}} servers := []*namedServer{{name: "public", srv: public}}
@@ -195,6 +266,27 @@ func runServers(ctx context.Context, cancel context.CancelFunc, servers []*named
return first return first
} }
// serveGRPC starts a gRPC server on addr in the background and gracefully stops it
// when ctx is cancelled. It returns synchronously on a bind error so a
// misconfigured listener fails startup fast; a later Serve error is logged.
func serveGRPC(ctx context.Context, name, addr string, srv *grpc.Server, logger *zap.Logger) error {
lis, err := net.Listen("tcp", addr)
if err != nil {
return fmt.Errorf("gateway: listen %s (%s): %w", addr, name, err)
}
go func() {
<-ctx.Done()
srv.GracefulStop()
}()
go func() {
logger.Info("listener starting", zap.String("server", name), zap.String("addr", addr))
if err := srv.Serve(lis); err != nil && !errors.Is(err, grpc.ErrServerStopped) {
logger.Error("grpc listener failed", zap.String("server", name), zap.Error(err))
}
}()
return nil
}
// runThrottleReporter drains the rate-limiter rejection tracker on a fixed // runThrottleReporter drains the rate-limiter rejection tracker on a fixed
// cadence, emits one Warn summary per throttled key and forwards the report to // cadence, emits one Warn summary per throttled key and forwards the report to
// the backend (which feeds the admin throttled view and the high-rate // the backend (which feeds the admin throttled view and the high-rate
@@ -226,11 +318,36 @@ func runThrottleReporter(ctx context.Context, tracker *ratelimit.Tracker, backen
} }
} }
// runBanSync periodically reports the gateway's active IP bans to the backend (the
// admin-console view) and applies the operator unbans it returns, until the
// context is done. A failed sync is logged and dropped — the next tick reports
// fresh state, and a missed unban is retried on it.
func runBanSync(ctx context.Context, banlist *ratelimit.Banlist, backend *backendclient.Client, logger *zap.Logger) {
ticker := time.NewTicker(banSyncInterval)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
}
unban, err := backend.SyncBans(ctx, banlist.Active())
if err != nil {
logger.Warn("ban sync failed", zap.Error(err))
continue
}
for _, ip := range unban {
banlist.Unban(ip)
logger.Info("ban cleared by operator", zap.String("client_ip", ip))
}
}
}
// runPushPump keeps a backend push subscription open, forwarding every event to // runPushPump keeps a backend push subscription open, forwarding every event to
// the hub and re-subscribing after the stream ends, until the context is done. For // the hub and re-subscribing after the stream ends, until the context is done. For
// the out-of-app push kinds it also routes events whose recipient has no live // the out-of-app push kinds it also routes events whose recipient has no live
// in-app stream to the platform connector (a nil connector disables that channel). // in-app stream to the platform connector (a nil connector disables that channel).
func runPushPump(ctx context.Context, backend *backendclient.Client, hub *push.Hub, conn *connector.Client, logger *zap.Logger) { func runPushPump(ctx context.Context, backend *backendclient.Client, hub *push.Hub, bot *botlink.Hub, logger *zap.Logger) {
for ctx.Err() == nil { for ctx.Err() == nil {
stream, err := backend.SubscribePush(ctx, gatewayID) stream, err := backend.SubscribePush(ctx, gatewayID)
if err != nil { if err != nil {
@@ -248,6 +365,15 @@ func runPushPump(ctx context.Context, backend *backendclient.Client, hub *push.H
} }
break break
} }
// A chat-access-changed event is an infra signal, not an in-app event:
// resolve the recipient's Telegram identity and current eligibility and
// push the chat-gate command to the bot, without fanning it out to clients.
if ev.GetKind() == chatAccessChangedKind {
if bot != nil {
go deliverChatGate(ctx, backend, bot, ev.GetUserId(), logger)
}
continue
}
hub.Publish(push.Event{ hub.Publish(push.Event{
UserID: ev.GetUserId(), UserID: ev.GetUserId(),
Kind: ev.GetKind(), Kind: ev.GetKind(),
@@ -255,10 +381,10 @@ func runPushPump(ctx context.Context, backend *backendclient.Client, hub *push.H
EventID: ev.GetEventId(), EventID: ev.GetEventId(),
}) })
// Out-of-app fallback: when the recipient has no live in-app stream, // Out-of-app fallback: when the recipient has no live in-app stream,
// deliver the event over the platform push channel. Done in a goroutine // deliver the event over the bot-link. Done in a goroutine so a slow
// so a slow connector never stalls the in-app firehose. // target lookup never stalls the in-app firehose.
if conn != nil && connector.OutOfAppKind(ev.GetKind()) && !hub.HasSubscribers(ev.GetUserId()) { if bot != nil && connector.OutOfAppKind(ev.GetKind()) && !hub.HasSubscribers(ev.GetUserId()) {
go deliverOutOfApp(ctx, backend, conn, ev.GetUserId(), ev.GetKind(), ev.GetPayload(), logger) go deliverOutOfApp(ctx, backend, bot, ev.GetUserId(), ev.GetKind(), ev.GetPayload(), logger)
} }
} }
if !sleep(ctx, pushReconnectDelay) { if !sleep(ctx, pushReconnectDelay) {
@@ -271,7 +397,7 @@ func runPushPump(ctx context.Context, backend *backendclient.Client, hub *push.H
// Telegram identity and have not confined notifications to the app, asks the // Telegram identity and have not confined notifications to the app, asks the
// connector to deliver the event. It is best-effort: every failure is logged and // connector to deliver the event. It is best-effort: every failure is logged and
// dropped (the in-app stream remains the primary channel). // dropped (the in-app stream remains the primary channel).
func deliverOutOfApp(ctx context.Context, backend *backendclient.Client, conn *connector.Client, userID, kind string, payload []byte, logger *zap.Logger) { func deliverOutOfApp(ctx context.Context, backend *backendclient.Client, bot *botlink.Hub, userID, kind string, payload []byte, logger *zap.Logger) {
target, err := backend.PushTarget(ctx, userID) target, err := backend.PushTarget(ctx, userID)
if err != nil { if err != nil {
logger.Warn("push target lookup failed", zap.String("user_id", userID), zap.Error(err)) logger.Warn("push target lookup failed", zap.String("user_id", userID), zap.Error(err))
@@ -280,10 +406,30 @@ func deliverOutOfApp(ctx context.Context, backend *backendclient.Client, conn *c
if !connector.DeliverToTarget(target.ExternalID, target.NotificationsInAppOnly) { if !connector.DeliverToTarget(target.ExternalID, target.NotificationsInAppOnly) {
return return
} }
// The single bot renders the message in the recipient's interface language. // Fire-and-forget down the bot-link; the bot renders the message in the
if _, err := conn.Notify(ctx, target.ExternalID, kind, payload, target.Language); err != nil { // recipient's interface language and is dropped (best-effort) if no bot is up.
logger.Warn("out-of-app notify failed", zap.String("kind", kind), zap.Error(err)) bot.Send(botlink.NotifyCommand(target.ExternalID, kind, payload, target.Language))
}
// chatAccessChangedKind is the backend event signalling that a player's moderated-chat
// write eligibility may have changed; the gateway turns it into a bot-link chat-gate
// command rather than an in-app event (it mirrors notify.KindChatAccessChanged).
const chatAccessChangedKind = "chat_access_changed"
// deliverChatGate resolves a chat-access-changed event to the recipient's Telegram
// identity and current eligibility and pushes the chat-gate command to the bot. It is
// best-effort: a recipient with no Telegram identity is skipped, and a resolve failure
// is logged and dropped (the next moderation action, or a re-join, re-applies the gate).
func deliverChatGate(ctx context.Context, backend *backendclient.Client, bot *botlink.Hub, userID string, logger *zap.Logger) {
res, err := backend.ChatAccessByUser(ctx, userID)
if err != nil {
logger.Warn("chat-gate resolve failed", zap.String("user_id", userID), zap.Error(err))
return
} }
if res.ExternalID == "" {
return // no Telegram identity, nothing to gate
}
bot.Send(botlink.ChatGateCommand(res.ExternalID, res.Eligible))
} }
// sleep waits for d or until ctx is cancelled, reporting whether it waited the // sleep waits for d or until ctx is cancelled, reporting whether it waited the
+28
View File
@@ -215,6 +215,34 @@ func (c *Client) PushTarget(ctx context.Context, userID string) (PushTargetResp,
return out, err return out, err
} }
// ChatAccessResp is a user's moderated-chat write eligibility: ExternalID is their
// Telegram identity (empty when they have none, so the gateway has nothing to gate),
// Registered whether an account was found, and Eligible the final gate the bot applies
// (registered and neither admin-suspended nor chat-muted).
type ChatAccessResp struct {
ExternalID string `json:"external_id"`
Registered bool `json:"registered"`
Eligible bool `json:"eligible"`
}
// ChatEligibility resolves a Telegram identity to its moderated-chat write
// eligibility — the join path, when the bot sees a user enter the chat.
func (c *Client) ChatEligibility(ctx context.Context, externalID string) (ChatAccessResp, error) {
var out ChatAccessResp
err := c.do(ctx, http.MethodPost, "/api/v1/internal/chat-access", "", "",
map[string]string{"external_id": externalID}, &out)
return out, err
}
// ChatAccessByUser resolves an account id to its Telegram identity and current
// moderated-chat write eligibility — the change path, for a chat-access-changed event.
func (c *Client) ChatAccessByUser(ctx context.Context, userID string) (ChatAccessResp, error) {
var out ChatAccessResp
err := c.do(ctx, http.MethodPost, "/api/v1/internal/chat-access", "", "",
map[string]string{"user_id": userID}, &out)
return out, err
}
// GuestAuth provisions a guest account and mints a session. // GuestAuth provisions a guest account and mints a session.
func (c *Client) GuestAuth(ctx context.Context) (SessionResp, error) { func (c *Client) GuestAuth(ctx context.Context) (SessionResp, error) {
var out SessionResp var out SessionResp
+38 -1
View File
@@ -22,6 +22,19 @@ import (
pushv1 "scrabble/pkg/proto/push/v1" pushv1 "scrabble/pkg/proto/push/v1"
) )
// backendMaxIdleConns sizes the REST keep-alive pool to the single backend host. The
// default transport caps idle connections per host at 2 (http.DefaultMaxIdleConnsPerHost),
// which — since every synchronous client call proxies to that one host — forces a fresh
// TCP connection (and a lingering TIME_WAIT socket) for almost every request under load.
// That connection churn burns gateway CPU and exhausts ephemeral ports at scale, all
// while the backend itself sits near-idle. Pooling the connections lets them be reused.
//
// The stress harness measured the effect at 500 concurrent players: the churn collapsed
// from ~26 500 TIME_WAIT sockets to ~0 and peak gateway CPU from ~1.75 to ~0.26 cores,
// with the pool settling at ~225 live connections. 512 keeps ~2x headroom over that
// observed peak so a burst never re-caps the pool. See loadtest/REPORT.md.
const backendMaxIdleConns = 512
// Client calls the backend's REST API and opens its push gRPC stream. // Client calls the backend's REST API and opens its push gRPC stream.
type Client struct { type Client struct {
baseURL string baseURL string
@@ -41,9 +54,14 @@ func New(httpURL, grpcAddr string, timeout time.Duration) (*Client, error) {
if err != nil { if err != nil {
return nil, fmt.Errorf("backendclient: dial push %s: %w", grpcAddr, err) return nil, fmt.Errorf("backendclient: dial push %s: %w", grpcAddr, err)
} }
// Clone the default transport (keeping its proxy, dialer and timeouts) and widen the
// idle pool so REST calls to the backend reuse connections instead of churning them.
transport := http.DefaultTransport.(*http.Transport).Clone()
transport.MaxIdleConns = backendMaxIdleConns
transport.MaxIdleConnsPerHost = backendMaxIdleConns
return &Client{ return &Client{
baseURL: strings.TrimRight(httpURL, "/"), baseURL: strings.TrimRight(httpURL, "/"),
http: &http.Client{Timeout: timeout}, http: &http.Client{Timeout: timeout, Transport: transport},
conn: conn, conn: conn,
push: pushv1.NewPushClient(conn), push: pushv1.NewPushClient(conn),
}, nil }, nil
@@ -137,3 +155,22 @@ func (c *Client) ReportRateLimited(ctx context.Context, windowSeconds int, entri
}{WindowSeconds: windowSeconds, Entries: entries} }{WindowSeconds: windowSeconds, Entries: entries}
return c.do(ctx, http.MethodPost, "/api/v1/internal/ratelimit/report", "", "", body, nil) return c.do(ctx, http.MethodPost, "/api/v1/internal/ratelimit/report", "", "", body, nil)
} }
// SyncBans reports the gateway's currently-active IP bans to the backend and
// returns the IPs an operator has marked for unban since the previous sync. It is
// the ban mirror of ReportRateLimited plus the manual-unban backchannel: the
// backend renders the active set in the admin console and drains the operator's
// unban requests into the response. Like the rejection report it carries no user
// identity and rides the trusted internal segment.
func (c *Client) SyncBans(ctx context.Context, active []ratelimit.Ban) ([]string, error) {
body := struct {
Active []ratelimit.Ban `json:"active"`
}{Active: active}
var out struct {
Unban []string `json:"unban"`
}
if err := c.do(ctx, http.MethodPost, "/api/v1/internal/bans/sync", "", "", body, &out); err != nil {
return nil, err
}
return out.Unban, nil
}
@@ -0,0 +1,31 @@
package backendclient
import (
"net/http"
"testing"
"time"
)
// TestBackendTransportPoolsConnections guards the fix for the gateway->backend
// connection churn. Every synchronous client call proxies to the single backend host,
// so the REST client must widen the idle-connection pool past the default per-host cap
// of 2 (http.DefaultMaxIdleConnsPerHost) — otherwise almost every request under load
// opens a fresh TCP connection that then lingers in TIME_WAIT, burning gateway CPU and
// exhausting ephemeral ports. Reverting to the default transport (`&http.Client{...}`
// with no Transport) would silently reintroduce that, so assert the pool is widened.
func TestBackendTransportPoolsConnections(t *testing.T) {
c, err := New("http://backend.invalid", "localhost:9090", time.Second)
if err != nil {
t.Fatalf("New: %v", err)
}
defer func() { _ = c.Close() }()
tr, ok := c.http.Transport.(*http.Transport)
if !ok {
t.Fatalf("REST transport = %T, want a *http.Transport with a widened idle pool", c.http.Transport)
}
if tr.MaxIdleConnsPerHost <= http.DefaultMaxIdleConnsPerHost {
t.Errorf("MaxIdleConnsPerHost = %d, want > default %d (else per-call connection churn)",
tr.MaxIdleConnsPerHost, http.DefaultMaxIdleConnsPerHost)
}
}
@@ -46,3 +46,39 @@ func TestReportRateLimited(t *testing.T) {
t.Fatalf("backend received %+v, want window 30 + %+v", got, entries[0]) t.Fatalf("backend received %+v, want window 30 + %+v", got, entries[0])
} }
} }
// TestSyncBans verifies the gateway reports its active bans to the backend's
// internal endpoint and returns the operator unban list the backend replies with.
func TestSyncBans(t *testing.T) {
var got struct {
Active []ratelimit.Ban `json:"active"`
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost || r.URL.Path != "/api/v1/internal/bans/sync" {
t.Errorf("call = %s %s, want POST /api/v1/internal/bans/sync", r.Method, r.URL.Path)
}
if err := json.NewDecoder(r.Body).Decode(&got); err != nil {
t.Errorf("decode sync: %v", err)
}
_, _ = w.Write([]byte(`{"unban":["203.0.113.9"]}`))
}))
defer srv.Close()
c, err := backendclient.New(srv.URL, "localhost:9090", 2*time.Second)
if err != nil {
t.Fatalf("backendclient: %v", err)
}
defer func() { _ = c.Close() }()
active := []ratelimit.Ban{{IP: "198.51.100.4", Reason: ratelimit.ReasonTripwire}}
unban, err := c.SyncBans(context.Background(), active)
if err != nil {
t.Fatalf("SyncBans: %v", err)
}
if len(got.Active) != 1 || got.Active[0].IP != "198.51.100.4" || got.Active[0].Reason != ratelimit.ReasonTripwire {
t.Fatalf("backend received active = %+v, want one tripwire ban for 198.51.100.4", got.Active)
}
if len(unban) != 1 || unban[0] != "203.0.113.9" {
t.Fatalf("unban = %v, want [203.0.113.9]", unban)
}
}
+50
View File
@@ -0,0 +1,50 @@
package botlink
import (
botlinkv1 "scrabble/pkg/proto/botlink/v1"
telegramv1 "scrabble/pkg/proto/telegram/v1"
)
// NotifyCommand builds an out-of-app push command rendered by the bot in the
// recipient's interface language.
func NotifyCommand(externalID, kind string, payload []byte, language string) *botlinkv1.Command {
return &botlinkv1.Command{
Payload: &botlinkv1.Command_Notify{Notify: &telegramv1.NotifyRequest{
ExternalId: externalID,
Kind: kind,
Payload: payload,
Language: language,
}},
}
}
// SendToUserCommand builds an admin text message addressed to one user.
func SendToUserCommand(externalID, text string) *botlinkv1.Command {
return &botlinkv1.Command{
Payload: &botlinkv1.Command_SendToUser{SendToUser: &telegramv1.SendToUserRequest{
ExternalId: externalID,
Text: text,
}},
}
}
// SendToGameChannelCommand builds an admin text message for the bot's game channel.
func SendToGameChannelCommand(text string) *botlinkv1.Command {
return &botlinkv1.Command{
Payload: &botlinkv1.Command_SendToChannel{SendToChannel: &telegramv1.SendToGameChannelRequest{
Text: text,
}},
}
}
// ChatGateCommand builds a chat-gate command that sets whether the Telegram user
// identified by externalID may write in the moderated discussion chat. The bot
// applies it only to a member currently in the chat (guarded on getChatMember).
func ChatGateCommand(externalID string, allow bool) *botlinkv1.Command {
return &botlinkv1.Command{
Payload: &botlinkv1.Command_ChatGate{ChatGate: &botlinkv1.ChatGateCommand{
ExternalId: externalID,
Allow: allow,
}},
}
}
+286
View File
@@ -0,0 +1,286 @@
// Package botlink is the gateway side of the reverse Telegram bot channel: a remote
// bot dials the gateway and opens one long-lived mTLS gRPC stream
// (pkg/proto/botlink/v1), over which the gateway pushes send Commands and the bot
// returns an Ack per command. The Hub registers connected bots and routes commands:
// out-of-app push is fire-and-forget (Send), admin sends await the bot Ack
// (SendAwait). Delivery is best-effort, at-most-once — a command lost across a
// reconnect is not replayed. See docs/ARCHITECTURE.md.
package botlink
import (
"context"
"errors"
"strconv"
"sync"
"sync/atomic"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/metric"
"go.uber.org/zap"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
botlinkv1 "scrabble/pkg/proto/botlink/v1"
)
// ErrNoBot is returned by SendAwait when no bot is currently connected.
var ErrNoBot = errors.New("botlink: no bot connected")
// outboundBuffer is the per-link command queue depth; a full queue drops commands
// (at-most-once under backpressure).
const outboundBuffer = 64
// EligibilityResolver answers a Telegram identity's moderated-chat write eligibility
// for the bot's join-time ResolveChatEligibility query: registered reports whether the
// identity maps to an account, eligible is the final gate the bot acts on (registered
// and neither admin-suspended nor chat-muted). The gateway backs it with the backend
// chat-access endpoint.
type EligibilityResolver func(ctx context.Context, externalID string) (registered, eligible bool, err error)
// Hub registers connected bots and routes send commands to them. A single bot is
// expected today; the registry already holds a set so adding more later needs no
// rewrite.
type Hub struct {
botlinkv1.UnimplementedBotLinkServer
log *zap.Logger
eligibility EligibilityResolver
mu sync.Mutex
links map[*link]struct{}
pending map[string]chan *botlinkv1.Ack
seq atomic.Uint64
connected metric.Int64UpDownCounter
commands metric.Int64Counter
}
// link is one connected bot's outbound queue and identity.
type link struct {
instanceID string
ownsUpdates bool
out chan *botlinkv1.ToBot
}
// NewHub builds a Hub. resolve answers the bot's join-time chat-eligibility query
// (nil rejects it as unavailable). A nil meter disables metrics; a nil logger is
// tolerated.
func NewHub(log *zap.Logger, meter metric.Meter, resolve EligibilityResolver) *Hub {
if log == nil {
log = zap.NewNop()
}
h := &Hub{
log: log,
eligibility: resolve,
links: make(map[*link]struct{}),
pending: make(map[string]chan *botlinkv1.Ack),
}
if meter != nil {
h.connected, _ = meter.Int64UpDownCounter("botlink_connected_bots",
metric.WithDescription("Number of Telegram bots currently connected to the gateway bot-link."))
h.commands, _ = meter.Int64Counter("botlink_commands_total",
metric.WithDescription("Bot-link send commands by result (delivered, not_delivered, dropped, error)."))
}
return h
}
// Link implements botlinkv1.BotLinkServer: it registers the dialing bot, drains
// queued commands to it, and resolves Acks until the stream ends.
func (h *Hub) Link(stream grpc.BidiStreamingServer[botlinkv1.FromBot, botlinkv1.ToBot]) error {
first, err := stream.Recv()
if err != nil {
return err
}
hello := first.GetHello()
if hello == nil {
return status.Error(codes.InvalidArgument, "first bot-link message must be Hello")
}
l := &link{
instanceID: hello.GetInstanceId(),
ownsUpdates: hello.GetOwnsUpdates(),
out: make(chan *botlinkv1.ToBot, outboundBuffer),
}
h.register(l)
defer h.unregister(l)
ctx := stream.Context()
// A dedicated goroutine owns stream.Send; the Recv loop below owns stream.Recv.
go func() {
for {
select {
case <-ctx.Done():
return
case msg := <-l.out:
if err := stream.Send(msg); err != nil {
return
}
}
}
}()
for {
msg, err := stream.Recv()
if err != nil {
return err
}
if ack := msg.GetAck(); ack != nil {
h.resolve(ack)
}
}
}
// ResolveChatEligibility serves the bot's join-time query: whether the Telegram user
// identified in the request may write in the moderated discussion chat. It delegates
// to the configured resolver (the backend chat-access endpoint), unlike the streamed
// Commands it is a plain request/response over the same mTLS channel.
func (h *Hub) ResolveChatEligibility(ctx context.Context, req *botlinkv1.ChatEligibilityRequest) (*botlinkv1.ChatEligibilityResponse, error) {
if h.eligibility == nil {
return nil, status.Error(codes.Unavailable, "chat eligibility resolver not configured")
}
registered, eligible, err := h.eligibility(ctx, req.GetExternalId())
if err != nil {
h.log.Warn("resolve chat eligibility failed", zap.String("external_id", req.GetExternalId()), zap.Error(err))
return nil, status.Error(codes.Internal, "resolve chat eligibility")
}
return &botlinkv1.ChatEligibilityResponse{Registered: registered, Eligible: eligible}, nil
}
// register adds a connected bot.
func (h *Hub) register(l *link) {
h.mu.Lock()
h.links[l] = struct{}{}
n := len(h.links)
h.mu.Unlock()
if h.connected != nil {
h.connected.Add(context.Background(), 1)
}
h.log.Info("bot connected",
zap.String("instance_id", l.instanceID),
zap.Bool("owns_updates", l.ownsUpdates),
zap.Int("connected", n))
}
// unregister removes a bot. l.out is left for the GC; its sender goroutine has
// already exited via the stream context, and stale enqueues simply never send
// (at-most-once).
func (h *Hub) unregister(l *link) {
h.mu.Lock()
delete(h.links, l)
n := len(h.links)
h.mu.Unlock()
if h.connected != nil {
h.connected.Add(context.Background(), -1)
}
h.log.Info("bot disconnected", zap.String("instance_id", l.instanceID), zap.Int("connected", n))
}
// pick returns one connected bot.
func (h *Hub) pick() (*link, bool) {
h.mu.Lock()
defer h.mu.Unlock()
for l := range h.links {
return l, true
}
return nil, false
}
// Send enqueues a fire-and-forget command to a connected bot, dropping it (with a
// warning) when no bot is connected or the queue is full.
func (h *Hub) Send(cmd *botlinkv1.Command) {
l, ok := h.pick()
if !ok {
h.count("dropped")
h.log.Warn("bot-link send dropped: no bot connected")
return
}
cmd.CommandId = h.nextID()
select {
case l.out <- &botlinkv1.ToBot{Command: cmd}:
default:
h.count("dropped")
h.log.Warn("bot-link send dropped: outbound queue full", zap.String("instance_id", l.instanceID))
}
}
// SendAwait enqueues a command and waits for the bot's Ack or until ctx is done.
// It returns ErrNoBot when no bot is connected, the delivered flag on a clean Ack,
// or (false, nil) when ctx (the deadline) fires before an Ack arrives.
func (h *Hub) SendAwait(ctx context.Context, cmd *botlinkv1.Command) (bool, error) {
l, ok := h.pick()
if !ok {
h.count("dropped")
return false, ErrNoBot
}
id := h.nextID()
cmd.CommandId = id
ackc := make(chan *botlinkv1.Ack, 1)
h.mu.Lock()
h.pending[id] = ackc
h.mu.Unlock()
defer func() {
h.mu.Lock()
delete(h.pending, id)
h.mu.Unlock()
}()
select {
case l.out <- &botlinkv1.ToBot{Command: cmd}:
case <-ctx.Done():
h.count("dropped")
return false, ctx.Err()
}
select {
case ack := <-ackc:
if e := ack.GetError(); e != "" {
h.count("error")
return false, errors.New(e)
}
h.count(deliveredLabel(ack.GetDelivered()))
return ack.GetDelivered(), nil
case <-ctx.Done():
h.count("error")
return false, nil
}
}
// resolve routes an Ack to its waiting SendAwait, if any.
func (h *Hub) resolve(ack *botlinkv1.Ack) {
h.mu.Lock()
c, ok := h.pending[ack.GetCommandId()]
h.mu.Unlock()
if ok {
select {
case c <- ack:
default:
}
}
}
// nextID returns a process-unique command id.
func (h *Hub) nextID() string {
return strconv.FormatUint(h.seq.Add(1), 10)
}
// count records one command result, when metrics are enabled.
func (h *Hub) count(result string) {
if h.commands == nil {
return
}
h.commands.Add(context.Background(), 1, metric.WithAttributes(resultAttr(result)))
}
// deliveredLabel maps the delivered flag to a metric result label.
func deliveredLabel(delivered bool) string {
if delivered {
return "delivered"
}
return "not_delivered"
}
// resultAttr is the metric attribute carrying a command result label.
func resultAttr(result string) attribute.KeyValue {
return attribute.String("result", result)
}
+235
View File
@@ -0,0 +1,235 @@
package botlink
import (
"context"
"errors"
"net"
"testing"
"time"
"google.golang.org/grpc"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/credentials/insecure"
"google.golang.org/grpc/status"
"google.golang.org/grpc/test/bufconn"
botlinkv1 "scrabble/pkg/proto/botlink/v1"
telegramv1 "scrabble/pkg/proto/telegram/v1"
)
// fakeBot is a test bot that dials the hub, registers, and acks commands with a
// fixed delivered flag (or never, when ack is false).
type fakeBot struct {
delivered bool
ack bool
received chan *botlinkv1.Command
}
// startHub registers a Hub (no chat-eligibility resolver) on an in-memory gRPC
// server and returns the hub plus a dialer for fake bots.
func startHub(t *testing.T) (*Hub, func(t *testing.T) botlinkv1.BotLinkClient) {
return startHubWith(t, nil)
}
// startHubWith is startHub with an explicit chat-eligibility resolver, for the
// ResolveChatEligibility tests.
func startHubWith(t *testing.T, resolve EligibilityResolver) (*Hub, func(t *testing.T) botlinkv1.BotLinkClient) {
t.Helper()
lis := bufconn.Listen(1 << 20)
hub := NewHub(nil, nil, resolve)
srv := grpc.NewServer()
botlinkv1.RegisterBotLinkServer(srv, hub)
go func() { _ = srv.Serve(lis) }()
t.Cleanup(srv.Stop)
dial := func(t *testing.T) botlinkv1.BotLinkClient {
t.Helper()
conn, err := grpc.NewClient("passthrough:///bufnet",
grpc.WithContextDialer(func(context.Context, string) (net.Conn, error) { return lis.Dial() }),
grpc.WithTransportCredentials(insecure.NewCredentials()),
)
if err != nil {
t.Fatalf("dial: %v", err)
}
t.Cleanup(func() { _ = conn.Close() })
return botlinkv1.NewBotLinkClient(conn)
}
return hub, dial
}
// connect runs a fake bot against client until ctx is cancelled, returning once the
// hub has registered it.
func (f *fakeBot) connect(t *testing.T, ctx context.Context, hub *Hub, client botlinkv1.BotLinkClient) {
t.Helper()
f.received = make(chan *botlinkv1.Command, 8)
stream, err := client.Link(ctx)
if err != nil {
t.Fatalf("link: %v", err)
}
if err := stream.Send(&botlinkv1.FromBot{Msg: &botlinkv1.FromBot_Hello{Hello: &botlinkv1.Hello{InstanceId: "test"}}}); err != nil {
t.Fatalf("hello: %v", err)
}
go func() {
for {
msg, err := stream.Recv()
if err != nil {
return
}
cmd := msg.GetCommand()
f.received <- cmd
if !f.ack {
continue
}
_ = stream.Send(&botlinkv1.FromBot{Msg: &botlinkv1.FromBot_Ack{Ack: &botlinkv1.Ack{
CommandId: cmd.GetCommandId(),
Delivered: f.delivered,
}}})
}
}()
waitConnected(t, hub, 1)
}
// waitConnected blocks until the hub reports n connected bots.
func waitConnected(t *testing.T, hub *Hub, n int) {
t.Helper()
deadline := time.Now().Add(2 * time.Second)
for time.Now().Before(deadline) {
hub.mu.Lock()
got := len(hub.links)
hub.mu.Unlock()
if got == n {
return
}
time.Sleep(5 * time.Millisecond)
}
t.Fatalf("hub did not reach %d connected bots", n)
}
func TestHubSendAwaitDelivered(t *testing.T) {
hub, dial := startHub(t)
ctx := t.Context()
bot := &fakeBot{delivered: true, ack: true}
bot.connect(t, ctx, hub, dial(t))
delivered, err := hub.SendAwait(ctx, SendToUserCommand("42", "hi"))
if err != nil {
t.Fatalf("SendAwait: %v", err)
}
if !delivered {
t.Fatal("delivered = false, want true")
}
select {
case cmd := <-bot.received:
if cmd.GetSendToUser().GetExternalId() != "42" {
t.Errorf("received external_id = %q, want 42", cmd.GetSendToUser().GetExternalId())
}
case <-time.After(time.Second):
t.Fatal("bot received no command")
}
}
func TestHubSendAwaitNoBot(t *testing.T) {
hub, _ := startHub(t)
_, err := hub.SendAwait(context.Background(), SendToUserCommand("42", "hi"))
if !errors.Is(err, ErrNoBot) {
t.Errorf("err = %v, want ErrNoBot", err)
}
}
func TestHubSendAwaitDeadline(t *testing.T) {
hub, dial := startHub(t)
ctx := t.Context()
bot := &fakeBot{ack: false} // receives but never acks
bot.connect(t, ctx, hub, dial(t))
awaitCtx, awaitCancel := context.WithTimeout(ctx, 100*time.Millisecond)
defer awaitCancel()
delivered, err := hub.SendAwait(awaitCtx, SendToUserCommand("42", "hi"))
if err != nil {
t.Fatalf("SendAwait err = %v, want nil on deadline", err)
}
if delivered {
t.Error("delivered = true, want false on deadline")
}
}
func TestHubSendAsync(t *testing.T) {
hub, dial := startHub(t)
ctx := t.Context()
bot := &fakeBot{ack: false}
bot.connect(t, ctx, hub, dial(t))
hub.Send(NotifyCommand("42", "your_turn", nil, "en"))
select {
case cmd := <-bot.received:
if cmd.GetNotify().GetExternalId() != "42" {
t.Errorf("received external_id = %q, want 42", cmd.GetNotify().GetExternalId())
}
case <-time.After(time.Second):
t.Fatal("bot received no command")
}
}
func TestHubSendChatGate(t *testing.T) {
hub, dial := startHub(t)
ctx := t.Context()
bot := &fakeBot{ack: false}
bot.connect(t, ctx, hub, dial(t))
hub.Send(ChatGateCommand("42", true))
select {
case cmd := <-bot.received:
cg := cmd.GetChatGate()
if cg.GetExternalId() != "42" || !cg.GetAllow() {
t.Errorf("chat_gate = %+v, want external_id=42 allow=true", cg)
}
case <-time.After(time.Second):
t.Fatal("bot received no command")
}
}
func TestHubResolveChatEligibility(t *testing.T) {
var gotExt string
_, dial := startHubWith(t, func(_ context.Context, ext string) (bool, bool, error) {
gotExt = ext
return true, ext == "good", nil
})
client := dial(t)
ctx := t.Context()
resp, err := client.ResolveChatEligibility(ctx, &botlinkv1.ChatEligibilityRequest{ExternalId: "good"})
if err != nil {
t.Fatalf("ResolveChatEligibility: %v", err)
}
if gotExt != "good" {
t.Errorf("resolver external_id = %q, want good", gotExt)
}
if !resp.GetRegistered() || !resp.GetEligible() {
t.Errorf("resp = %+v, want registered+eligible", resp)
}
resp, err = client.ResolveChatEligibility(ctx, &botlinkv1.ChatEligibilityRequest{ExternalId: "muted"})
if err != nil {
t.Fatalf("ResolveChatEligibility(muted): %v", err)
}
if !resp.GetRegistered() || resp.GetEligible() {
t.Errorf("resp = %+v, want registered but not eligible", resp)
}
}
func TestHubResolveChatEligibilityUnconfigured(t *testing.T) {
_, dial := startHub(t) // nil resolver
client := dial(t)
_, err := client.ResolveChatEligibility(t.Context(), &botlinkv1.ChatEligibilityRequest{ExternalId: "x"})
if status.Code(err) != codes.Unavailable {
t.Fatalf("err = %v, want Unavailable", err)
}
}
func TestRelayServerNoBot(t *testing.T) {
hub, _ := startHub(t)
relay := NewRelayServer(hub, 200*time.Millisecond)
if _, err := relay.SendToUser(context.Background(), &telegramv1.SendToUserRequest{ExternalId: "42", Text: "hi"}); err == nil {
t.Fatal("expected an error with no bot connected")
}
}
+160
View File
@@ -0,0 +1,160 @@
package botlink
import (
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"encoding/pem"
"math/big"
"net"
"os"
"path/filepath"
"testing"
"time"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
"scrabble/pkg/mtls"
botlinkv1 "scrabble/pkg/proto/botlink/v1"
)
// mintCerts writes a CA, a server leaf (IP SAN 127.0.0.1) and a client leaf into a
// temp dir, returning their file paths. It exercises the real pkg/mtls loaders.
func mintCerts(t *testing.T) (caFile, srvCert, srvKey, cliCert, cliKey string) {
t.Helper()
dir := t.TempDir()
caKey, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
caTmpl := &x509.Certificate{
SerialNumber: big.NewInt(1),
Subject: pkix.Name{CommonName: "test-ca"},
NotBefore: time.Now().Add(-time.Hour),
NotAfter: time.Now().Add(time.Hour),
IsCA: true,
KeyUsage: x509.KeyUsageCertSign,
BasicConstraintsValid: true,
}
caDER, err := x509.CreateCertificate(rand.Reader, caTmpl, caTmpl, &caKey.PublicKey, caKey)
if err != nil {
t.Fatalf("ca: %v", err)
}
caCert, _ := x509.ParseCertificate(caDER)
leaf := func(cn string, eku x509.ExtKeyUsage, ips []net.IP) (certPath, keyPath string) {
key, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
tmpl := &x509.Certificate{
SerialNumber: big.NewInt(time.Now().UnixNano()),
Subject: pkix.Name{CommonName: cn},
NotBefore: time.Now().Add(-time.Hour),
NotAfter: time.Now().Add(time.Hour),
KeyUsage: x509.KeyUsageDigitalSignature,
ExtKeyUsage: []x509.ExtKeyUsage{eku},
IPAddresses: ips,
}
der, err := x509.CreateCertificate(rand.Reader, tmpl, caCert, &key.PublicKey, caKey)
if err != nil {
t.Fatalf("leaf %s: %v", cn, err)
}
certPath = filepath.Join(dir, cn+".crt")
keyPath = filepath.Join(dir, cn+".key")
writePEM(t, certPath, "CERTIFICATE", der)
keyDER, _ := x509.MarshalPKCS8PrivateKey(key)
writePEM(t, keyPath, "PRIVATE KEY", keyDER)
return certPath, keyPath
}
caFile = filepath.Join(dir, "ca.crt")
writePEM(t, caFile, "CERTIFICATE", caDER)
srvCert, srvKey = leaf("server", x509.ExtKeyUsageServerAuth, []net.IP{net.ParseIP("127.0.0.1")})
cliCert, cliKey = leaf("client", x509.ExtKeyUsageClientAuth, nil)
return caFile, srvCert, srvKey, cliCert, cliKey
}
func writePEM(t *testing.T, path, typ string, der []byte) {
t.Helper()
if err := os.WriteFile(path, pem.EncodeToMemory(&pem.Block{Type: typ, Bytes: der}), 0o600); err != nil {
t.Fatalf("write %s: %v", path, err)
}
}
// startMTLSHub starts the Hub behind a real mTLS gRPC listener and returns its
// address and the CA/client cert paths.
func startMTLSHub(t *testing.T) (hub *Hub, addr, caFile, cliCert, cliKey string) {
t.Helper()
caFile, srvCert, srvKey, cliCert, cliKey := mintCerts(t)
tlsCfg, err := mtls.ServerConfig(srvCert, srvKey, caFile)
if err != nil {
t.Fatalf("server config: %v", err)
}
lis, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("listen: %v", err)
}
hub = NewHub(nil, nil, nil)
srv := grpc.NewServer(grpc.Creds(credentials.NewTLS(tlsCfg)))
botlinkv1.RegisterBotLinkServer(srv, hub)
go func() { _ = srv.Serve(lis) }()
t.Cleanup(srv.Stop)
return hub, lis.Addr().String(), caFile, cliCert, cliKey
}
// TestMTLSValidClientDelivers verifies a CA-signed bot connects and receives a
// pushed command.
func TestMTLSValidClientDelivers(t *testing.T) {
hub, addr, caFile, cliCert, cliKey := startMTLSHub(t)
tlsCfg, err := mtls.ClientConfig(cliCert, cliKey, caFile, "127.0.0.1")
if err != nil {
t.Fatalf("client config: %v", err)
}
conn, err := grpc.NewClient(addr, grpc.WithTransportCredentials(credentials.NewTLS(tlsCfg)))
if err != nil {
t.Fatalf("dial: %v", err)
}
t.Cleanup(func() { _ = conn.Close() })
ctx := t.Context()
bot := &fakeBot{delivered: true, ack: true}
bot.connect(t, ctx, hub, botlinkv1.NewBotLinkClient(conn))
delivered, err := hub.SendAwait(ctx, SendToUserCommand("42", "hi"))
if err != nil || !delivered {
t.Fatalf("SendAwait = (%v, %v), want (true, nil)", delivered, err)
}
}
// TestMTLSRejectsClientWithoutCert verifies the gateway refuses a client that
// presents no CA-signed certificate — mTLS is the sole guard on the bot-link.
func TestMTLSRejectsClientWithoutCert(t *testing.T) {
hub, addr, caFile, _, _ := startMTLSHub(t)
caPEM, _ := os.ReadFile(caFile)
pool := x509.NewCertPool()
pool.AppendCertsFromPEM(caPEM)
// Trusts the server, but presents no client certificate.
noCert := credentials.NewTLS(&tls.Config{RootCAs: pool, ServerName: "127.0.0.1", MinVersion: tls.VersionTLS13})
conn, err := grpc.NewClient(addr, grpc.WithTransportCredentials(noCert))
if err != nil {
t.Fatalf("dial: %v", err)
}
t.Cleanup(func() { _ = conn.Close() })
stream, err := botlinkv1.NewBotLinkClient(conn).Link(t.Context())
if err == nil {
err = stream.Send(&botlinkv1.FromBot{Msg: &botlinkv1.FromBot_Hello{Hello: &botlinkv1.Hello{InstanceId: "rogue"}}})
if err == nil {
_, err = stream.Recv()
}
}
if err == nil {
t.Fatal("expected the handshake to be rejected without a client certificate")
}
hub.mu.Lock()
n := len(hub.links)
hub.mu.Unlock()
if n != 0 {
t.Errorf("connected bots = %d, want 0 (rogue must not register)", n)
}
}
+67
View File
@@ -0,0 +1,67 @@
package botlink
import (
"context"
"errors"
"time"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
botlinkv1 "scrabble/pkg/proto/botlink/v1"
telegramv1 "scrabble/pkg/proto/telegram/v1"
)
// RelayServer lets the backend's admin console reach the remote bot through the
// gateway: it implements the two admin delivery methods of the Telegram service by
// forwarding them onto the bot-link and awaiting the bot's Ack. The validation and
// out-of-app push methods are intentionally unimplemented here — login validation
// runs against the home validator and out-of-app push goes straight to the Hub.
type RelayServer struct {
telegramv1.UnimplementedTelegramServer
hub *Hub
deadline time.Duration
}
// NewRelayServer builds the relay over hub. deadline bounds the wait for the bot's
// Ack before reporting the send as not delivered.
func NewRelayServer(hub *Hub, deadline time.Duration) *RelayServer {
return &RelayServer{hub: hub, deadline: deadline}
}
// SendToUser forwards an admin message for one user to the bot and reports whether
// it was delivered. A missing bot maps to Unavailable (parity with an unreachable
// connector); a deadline before the Ack reports delivered=false.
func (r *RelayServer) SendToUser(ctx context.Context, req *telegramv1.SendToUserRequest) (*telegramv1.SendResponse, error) {
delivered, err := r.await(ctx, SendToUserCommand(req.GetExternalId(), req.GetText()))
if err != nil {
return nil, err
}
return &telegramv1.SendResponse{Delivered: delivered}, nil
}
// SendToGameChannel forwards an admin message for the game channel to the bot and
// reports whether it was delivered.
func (r *RelayServer) SendToGameChannel(ctx context.Context, req *telegramv1.SendToGameChannelRequest) (*telegramv1.SendResponse, error) {
delivered, err := r.await(ctx, SendToGameChannelCommand(req.GetText()))
if err != nil {
return nil, err
}
return &telegramv1.SendResponse{Delivered: delivered}, nil
}
// await sends cmd with the relay deadline and translates the Hub outcome into the
// gRPC contract the backend client expects.
func (r *RelayServer) await(ctx context.Context, cmd *botlinkv1.Command) (bool, error) {
cctx, cancel := context.WithTimeout(ctx, r.deadline)
defer cancel()
delivered, err := r.hub.SendAwait(cctx, cmd)
switch {
case errors.Is(err, ErrNoBot):
return false, status.Error(codes.Unavailable, "no telegram bot connected")
case err != nil:
return false, status.Error(codes.Internal, err.Error())
default:
return delivered, nil
}
}
+118 -5
View File
@@ -28,10 +28,13 @@ type Config struct {
// checks before proxying admin traffic to the backend. Empty disables admin. // checks before proxying admin traffic to the backend. Empty disables admin.
AdminUser string AdminUser string
AdminPassword string AdminPassword string
// ConnectorAddr is the gRPC address of the Telegram connector side-service. The // ValidatorAddr is the gRPC address of the Telegram validator side-service (home,
// gateway calls it to validate Mini App initData and to deliver out-of-app push. // plaintext, internal). The gateway calls it to validate Mini App initData and
// Empty disables the telegram auth path and the out-of-app push channel. // Login Widget data. Empty disables the telegram auth path.
ConnectorAddr string ValidatorAddr string
// BotLink configures the reverse mTLS channel to the remote Telegram bot. An
// empty BotLink.Addr disables the bot channel (out-of-app push and admin relay).
BotLink BotLinkConfig
// SessionTTL bounds how long a resolved session stays cached; SessionCacheMax // SessionTTL bounds how long a resolved session stays cached; SessionCacheMax
// caps the number of cached sessions. // caps the number of cached sessions.
SessionTTL time.Duration SessionTTL time.Duration
@@ -43,10 +46,35 @@ type Config struct {
MaxBodyBytes int MaxBodyBytes int
// RateLimit configures the in-memory anti-abuse limiter. // RateLimit configures the in-memory anti-abuse limiter.
RateLimit RateLimitConfig RateLimit RateLimitConfig
// Abuse configures the temporary IP ban and the honeytoken (prod-only).
Abuse AbuseConfig
// Telemetry configures the OpenTelemetry providers (shared bootstrap). // Telemetry configures the OpenTelemetry providers (shared bootstrap).
Telemetry pkgtel.Config Telemetry pkgtel.Config
} }
// BotLinkConfig configures the gateway's reverse bot-link: the mTLS listener the
// remote Telegram bot dials, the plaintext listener the backend admin relay calls,
// and the mTLS material. The main host is already public, so exposing the bot-link
// listener on a dedicated port adds no static IP; the channel is guarded solely by
// mTLS (the bot has no fixed address to allow-list).
type BotLinkConfig struct {
// Addr is the mTLS gRPC listener the bot dials (e.g. ":9443"). Empty disables
// the whole bot channel.
Addr string
// RelayAddr is the plaintext internal gRPC listener that serves the backend
// admin SendToUser/SendToGameChannel relay (e.g. ":9092"). Empty disables it.
RelayAddr string
// CertFile, KeyFile and CAFile are the gateway server certificate, its key and
// the CA bundle that signs the accepted bot client certificates. Required when
// Addr is set.
CertFile string
KeyFile string
CAFile string
// SendTimeout bounds the admin relay's wait for the bot Ack before reporting
// the send as not delivered.
SendTimeout time.Duration
}
// RateLimitConfig holds the token-bucket limits per class. Public and admin are // RateLimitConfig holds the token-bucket limits per class. Public and admin are
// keyed per client IP; the authenticated class is keyed per user id; the email // keyed per client IP; the authenticated class is keyed per user id; the email
// sub-limit guards the costly email-code path per IP. // sub-limit guards the costly email-code path per IP.
@@ -61,8 +89,32 @@ type RateLimitConfig struct {
EmailBurst int EmailBurst int
} }
// AbuseConfig configures the gateway's temporary IP ban (fail2ban-style) and the
// honeytoken trap. BanEnabled gates the ban action and is off by default: it is
// only safe where the real client IP is visible (i.e. in prod, not behind the
// shared-NAT test contour). Detection of honeypot/honeytoken hits is logged
// regardless of BanEnabled — only the ban action is gated.
type AbuseConfig struct {
// BanEnabled turns the IP ban on. Off by default (prod-only).
BanEnabled bool
// BanThreshold is the rate-limiter rejection count within BanWindow that bans
// a client IP.
BanThreshold int
// BanWindow is the rolling window the rejection strikes accumulate over.
BanWindow time.Duration
// BanDuration is the length of a rejection-earned ban (tripwire and honeytoken
// bans use their own, longer, fixed durations).
BanDuration time.Duration
// Honeytoken, when non-empty, is a planted bearer value: presenting it bans the
// caller and raises a high-severity alarm. Empty disables the trap.
Honeytoken string
}
// Defaults applied when the corresponding environment variable is unset. // Defaults applied when the corresponding environment variable is unset.
const ( const (
defaultAbuseBanThreshold = 100
defaultAbuseBanWindow = 2 * time.Minute
defaultAbuseBanDuration = 15 * time.Minute
defaultHTTPAddr = ":8081" defaultHTTPAddr = ":8081"
defaultLogLevel = "info" defaultLogLevel = "info"
defaultBackendHTTPURL = "http://localhost:8080" defaultBackendHTTPURL = "http://localhost:8080"
@@ -72,6 +124,7 @@ const (
defaultSessionCacheMax = 50000 defaultSessionCacheMax = 50000
defaultPushHeartbeatInterval = 10 * time.Second // under the ~15 s edge idle timeout defaultPushHeartbeatInterval = 10 * time.Second // under the ~15 s edge idle timeout
defaultServiceName = "scrabble-gateway" defaultServiceName = "scrabble-gateway"
defaultBotLinkSendTimeout = 5 * time.Second
) )
// DefaultMaxBodyBytes is the default request-body cap (GATEWAY_MAX_BODY_BYTES): // DefaultMaxBodyBytes is the default request-body cap (GATEWAY_MAX_BODY_BYTES):
@@ -93,6 +146,17 @@ func DefaultRateLimit() RateLimitConfig {
} }
} }
// DefaultAbuse returns the built-in anti-abuse settings: the ban disabled
// (prod-only) with the agreed thresholds, and no honeytoken.
func DefaultAbuse() AbuseConfig {
return AbuseConfig{
BanEnabled: false,
BanThreshold: defaultAbuseBanThreshold,
BanWindow: defaultAbuseBanWindow,
BanDuration: defaultAbuseBanDuration,
}
}
// Load reads the configuration from the environment, applies defaults, and // Load reads the configuration from the environment, applies defaults, and
// validates the result. // validates the result.
func Load() (Config, error) { func Load() (Config, error) {
@@ -104,9 +168,17 @@ func Load() (Config, error) {
BackendGRPCAddr: envOr("GATEWAY_BACKEND_GRPC_ADDR", defaultBackendGRPCAddr), BackendGRPCAddr: envOr("GATEWAY_BACKEND_GRPC_ADDR", defaultBackendGRPCAddr),
AdminUser: os.Getenv("GATEWAY_ADMIN_USER"), AdminUser: os.Getenv("GATEWAY_ADMIN_USER"),
AdminPassword: os.Getenv("GATEWAY_ADMIN_PASSWORD"), AdminPassword: os.Getenv("GATEWAY_ADMIN_PASSWORD"),
ConnectorAddr: os.Getenv("GATEWAY_CONNECTOR_ADDR"), ValidatorAddr: os.Getenv("GATEWAY_VALIDATOR_ADDR"),
SessionCacheMax: defaultSessionCacheMax, SessionCacheMax: defaultSessionCacheMax,
RateLimit: DefaultRateLimit(), RateLimit: DefaultRateLimit(),
Abuse: DefaultAbuse(),
BotLink: BotLinkConfig{
Addr: os.Getenv("GATEWAY_BOTLINK_ADDR"),
RelayAddr: os.Getenv("GATEWAY_BOTLINK_RELAY_ADDR"),
CertFile: os.Getenv("GATEWAY_BOTLINK_TLS_CERT"),
KeyFile: os.Getenv("GATEWAY_BOTLINK_TLS_KEY"),
CAFile: os.Getenv("GATEWAY_BOTLINK_TLS_CA"),
},
} }
tel := pkgtel.DefaultConfig(defaultServiceName) tel := pkgtel.DefaultConfig(defaultServiceName)
tel.ServiceName = envOr("GATEWAY_SERVICE_NAME", tel.ServiceName) tel.ServiceName = envOr("GATEWAY_SERVICE_NAME", tel.ServiceName)
@@ -128,12 +200,31 @@ func Load() (Config, error) {
if c.MaxBodyBytes, err = envInt("GATEWAY_MAX_BODY_BYTES", DefaultMaxBodyBytes); err != nil { if c.MaxBodyBytes, err = envInt("GATEWAY_MAX_BODY_BYTES", DefaultMaxBodyBytes); err != nil {
return Config{}, err return Config{}, err
} }
c.Abuse.Honeytoken = os.Getenv("GATEWAY_HONEYTOKEN")
if c.Abuse.BanEnabled, err = envBool("GATEWAY_ABUSE_BAN_ENABLED", c.Abuse.BanEnabled); err != nil {
return Config{}, err
}
if c.Abuse.BanThreshold, err = envInt("GATEWAY_ABUSE_BAN_THRESHOLD", c.Abuse.BanThreshold); err != nil {
return Config{}, err
}
if c.Abuse.BanWindow, err = envDuration("GATEWAY_ABUSE_BAN_WINDOW", c.Abuse.BanWindow); err != nil {
return Config{}, err
}
if c.Abuse.BanDuration, err = envDuration("GATEWAY_ABUSE_BAN_DURATION", c.Abuse.BanDuration); err != nil {
return Config{}, err
}
if c.BotLink.SendTimeout, err = envDuration("GATEWAY_BOTLINK_SEND_TIMEOUT", defaultBotLinkSendTimeout); err != nil {
return Config{}, err
}
if err := c.validate(); err != nil { if err := c.validate(); err != nil {
return Config{}, err return Config{}, err
} }
return c, nil return c, nil
} }
// BotLinkEnabled reports whether the reverse bot-link channel is configured.
func (c Config) BotLinkEnabled() bool { return c.BotLink.Addr != "" }
// AdminEnabled reports whether the admin console proxy should be mounted (both // AdminEnabled reports whether the admin console proxy should be mounted (both
// Basic-Auth credentials are configured). // Basic-Auth credentials are configured).
func (c Config) AdminEnabled() bool { func (c Config) AdminEnabled() bool {
@@ -159,6 +250,14 @@ func (c Config) validate() error {
if c.MaxBodyBytes <= 0 { if c.MaxBodyBytes <= 0 {
return fmt.Errorf("config: GATEWAY_MAX_BODY_BYTES must be positive") return fmt.Errorf("config: GATEWAY_MAX_BODY_BYTES must be positive")
} }
if c.Abuse.BanEnabled && (c.Abuse.BanThreshold <= 0 || c.Abuse.BanWindow <= 0 || c.Abuse.BanDuration <= 0) {
return fmt.Errorf("config: GATEWAY_ABUSE_BAN_THRESHOLD/_WINDOW/_DURATION must be positive when GATEWAY_ABUSE_BAN_ENABLED")
}
if c.BotLink.Addr != "" {
if c.BotLink.CertFile == "" || c.BotLink.KeyFile == "" || c.BotLink.CAFile == "" {
return fmt.Errorf("config: GATEWAY_BOTLINK_ADDR requires GATEWAY_BOTLINK_TLS_CERT, _KEY and _CA")
}
}
if err := c.Telemetry.Validate(); err != nil { if err := c.Telemetry.Validate(); err != nil {
return fmt.Errorf("config: %w", err) return fmt.Errorf("config: %w", err)
} }
@@ -174,6 +273,20 @@ func envOr(key, fallback string) string {
return fallback return fallback
} }
// envBool parses the environment variable named key as a bool, returning fallback
// when it is unset and an error when it is set but malformed.
func envBool(key string, fallback bool) (bool, error) {
v := os.Getenv(key)
if v == "" {
return fallback, nil
}
b, err := strconv.ParseBool(v)
if err != nil {
return false, fmt.Errorf("config: %s: %w", key, err)
}
return b, nil
}
// envInt parses the environment variable named key as an int, returning fallback // envInt parses the environment variable named key as an int, returning fallback
// when it is unset and an error when it is set but malformed. // when it is unset and an error when it is set but malformed.
func envInt(key string, fallback int) (int, error) { func envInt(key string, fallback int) (int, error) {
+50
View File
@@ -2,6 +2,7 @@ package config
import ( import (
"testing" "testing"
"time"
pkgtel "scrabble/pkg/telemetry" pkgtel "scrabble/pkg/telemetry"
) )
@@ -45,3 +46,52 @@ func TestLoadMaxBodyBytes(t *testing.T) {
t.Fatal("Load: expected an error for a non-positive body cap, got nil") t.Fatal("Load: expected an error for a non-positive body cap, got nil")
} }
} }
// TestLoadAbuseDefaults verifies the anti-abuse ban defaults: disabled (prod-only),
// the agreed thresholds, and no honeytoken.
func TestLoadAbuseDefaults(t *testing.T) {
c, err := Load()
if err != nil {
t.Fatalf("Load: %v", err)
}
want := DefaultAbuse()
if c.Abuse != want {
t.Errorf("Abuse = %+v, want %+v", c.Abuse, want)
}
if c.Abuse.BanEnabled {
t.Error("ban must default to disabled (enabled only in prod)")
}
}
// TestLoadAbuseOverrides verifies the anti-abuse environment variables are parsed.
func TestLoadAbuseOverrides(t *testing.T) {
t.Setenv("GATEWAY_ABUSE_BAN_ENABLED", "true")
t.Setenv("GATEWAY_ABUSE_BAN_THRESHOLD", "50")
t.Setenv("GATEWAY_ABUSE_BAN_WINDOW", "90s")
t.Setenv("GATEWAY_ABUSE_BAN_DURATION", "30m")
t.Setenv("GATEWAY_HONEYTOKEN", "deadbeef")
c, err := Load()
if err != nil {
t.Fatalf("Load: %v", err)
}
want := AbuseConfig{
BanEnabled: true,
BanThreshold: 50,
BanWindow: 90 * time.Second,
BanDuration: 30 * time.Minute,
Honeytoken: "deadbeef",
}
if c.Abuse != want {
t.Errorf("Abuse = %+v, want %+v", c.Abuse, want)
}
}
// TestLoadAbuseRejectsBadThreshold verifies an enabled ban with a non-positive
// threshold fails validation.
func TestLoadAbuseRejectsBadThreshold(t *testing.T) {
t.Setenv("GATEWAY_ABUSE_BAN_ENABLED", "true")
t.Setenv("GATEWAY_ABUSE_BAN_THRESHOLD", "0")
if _, err := Load(); err == nil {
t.Fatal("Load: expected an error for an enabled ban with a zero threshold, got nil")
}
}
+6 -19
View File
@@ -1,7 +1,9 @@
// Package connector is the gateway's gRPC client for the Telegram connector // Package connector is the gateway's gRPC client for the Telegram validator
// side-service: it validates Mini App initData and delivers out-of-app push. The // side-service: it validates Mini App initData and Login Widget data. The validator
// connector lives on the trusted internal network, so the connection uses insecure // lives on the trusted internal network and holds the bot token only for HMAC, so
// (plaintext) transport credentials (ARCHITECTURE.md §12). // the connection uses insecure (plaintext) transport credentials (ARCHITECTURE.md
// §12). Out-of-app push no longer goes through this client; it is delivered to the
// remote bot over the reverse mTLS bot-link (gateway/internal/botlink).
package connector package connector
import ( import (
@@ -92,18 +94,3 @@ func (c *Client) ValidateLoginWidget(ctx context.Context, data string) (User, er
FirstName: resp.GetFirstName(), FirstName: resp.GetFirstName(),
}, nil }, nil
} }
// Notify delivers an out-of-app notification for a push event; delivered reports
// whether a message was actually sent.
func (c *Client) Notify(ctx context.Context, externalID, kind string, payload []byte, language string) (bool, error) {
resp, err := c.c.Notify(ctx, &telegramv1.NotifyRequest{
ExternalId: externalID,
Kind: kind,
Payload: payload,
Language: language,
})
if err != nil {
return false, err
}
return resp.GetDelivered(), nil
}
+219
View File
@@ -0,0 +1,219 @@
package connectsrv_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
"connectrpc.com/connect"
"scrabble/gateway/internal/backendclient"
"scrabble/gateway/internal/config"
"scrabble/gateway/internal/connectsrv"
"scrabble/gateway/internal/push"
"scrabble/gateway/internal/ratelimit"
"scrabble/gateway/internal/session"
"scrabble/gateway/internal/transcode"
edgev1 "scrabble/gateway/proto/edge/v1"
"scrabble/gateway/proto/edge/v1/edgev1connect"
)
const honeypotHeader = "X-Scrabble-Honeypot"
// guardedEdge wires an edge with an explicit banlist and honeytoken over a fake
// backend, returning the front URL, a Connect client and a cleanup func.
func guardedEdge(t *testing.T, bl *ratelimit.Banlist, honeytoken string, limits config.RateLimitConfig, backendHandler http.HandlerFunc) (string, edgev1connect.GatewayClient, func()) {
t.Helper()
backendSrv := httptest.NewServer(backendHandler)
backend, err := backendclient.New(backendSrv.URL, "localhost:9090", 2*time.Second)
if err != nil {
t.Fatalf("backendclient: %v", err)
}
edge := connectsrv.NewServer(connectsrv.Deps{
Registry: transcode.NewRegistry(backend, nil),
Sessions: session.NewCache(backend, time.Minute, 100),
Limiter: ratelimit.New(),
Banlist: bl,
Honeytoken: honeytoken,
Hub: push.NewHub(0),
RateLimit: limits,
Heartbeat: 15 * time.Second,
})
edgeSrv := httptest.NewServer(edge.HTTPHandler())
client := edgev1connect.NewGatewayClient(http.DefaultClient, edgeSrv.URL)
return edgeSrv.URL, client, func() {
edgeSrv.Close()
_ = backend.Close()
backendSrv.Close()
}
}
// noRedirect is an HTTP client that surfaces a redirect instead of following it,
// so the test can tell a 308 (passed the guard) from a 429 (blocked).
func noRedirect() *http.Client {
return &http.Client{CheckRedirect: func(*http.Request, []*http.Request) error {
return http.ErrUseLastResponse
}}
}
func enabledBanlist(threshold int) *ratelimit.Banlist {
return ratelimit.NewBanlist(ratelimit.BanConfig{
Enabled: true, Threshold: threshold, Window: time.Minute, Duration: time.Hour,
})
}
// TestAbuseGuardBlocksBannedIP verifies a banned client IP is refused with 429 at
// the HTTP layer, before any handler runs.
func TestAbuseGuardBlocksBannedIP(t *testing.T) {
bl := enabledBanlist(100)
bl.BanNow("127.0.0.1", ratelimit.ReasonTripwire)
url, _, cleanup := guardedEdge(t, bl, "", config.DefaultRateLimit(), func(w http.ResponseWriter, r *http.Request) {})
defer cleanup()
resp, err := noRedirect().Get(url + "/")
if err != nil {
t.Fatalf("get: %v", err)
}
_ = resp.Body.Close()
if resp.StatusCode != http.StatusTooManyRequests {
t.Fatalf("banned GET / = %d, want 429", resp.StatusCode)
}
}
// TestHoneypotHeaderTrips verifies a request carrying the honeypot header is 404'd
// and bans the client IP, so the next request is blocked.
func TestHoneypotHeaderTrips(t *testing.T) {
bl := enabledBanlist(100)
url, _, cleanup := guardedEdge(t, bl, "", config.DefaultRateLimit(), func(w http.ResponseWriter, r *http.Request) {
t.Error("backend must not be called for a honeypot hit")
})
defer cleanup()
req, _ := http.NewRequest(http.MethodGet, url+"/.env", nil)
req.Header.Set(honeypotHeader, "1")
resp, err := noRedirect().Do(req)
if err != nil {
t.Fatalf("honeypot get: %v", err)
}
_ = resp.Body.Close()
if resp.StatusCode != http.StatusNotFound {
t.Fatalf("honeypot hit = %d, want 404", resp.StatusCode)
}
if !bl.Banned("127.0.0.1") {
t.Fatal("a honeypot hit must ban the client IP")
}
follow, err := noRedirect().Get(url + "/")
if err != nil {
t.Fatalf("follow-up get: %v", err)
}
_ = follow.Body.Close()
if follow.StatusCode != http.StatusTooManyRequests {
t.Fatalf("post-trip GET / = %d, want 429", follow.StatusCode)
}
}
// TestHoneypotDetectsWithoutBanWhenDisabled verifies the prod-only gate: a disabled
// banlist still 404s the decoy (detection/logging) but bans nothing.
func TestHoneypotDetectsWithoutBanWhenDisabled(t *testing.T) {
bl := ratelimit.NewBanlist(ratelimit.BanConfig{}) // disabled
url, _, cleanup := guardedEdge(t, bl, "", config.DefaultRateLimit(), func(w http.ResponseWriter, r *http.Request) {})
defer cleanup()
req, _ := http.NewRequest(http.MethodGet, url+"/.env", nil)
req.Header.Set(honeypotHeader, "1")
resp, err := noRedirect().Do(req)
if err != nil {
t.Fatalf("honeypot get: %v", err)
}
_ = resp.Body.Close()
if resp.StatusCode != http.StatusNotFound {
t.Fatalf("decoy = %d, want 404", resp.StatusCode)
}
if bl.Banned("127.0.0.1") {
t.Fatal("a disabled banlist must not ban")
}
follow, err := noRedirect().Get(url + "/")
if err != nil {
t.Fatalf("follow-up: %v", err)
}
_ = follow.Body.Close()
if follow.StatusCode != http.StatusPermanentRedirect {
t.Fatalf("post-trip GET / = %d, want 308 (not banned)", follow.StatusCode)
}
}
// TestPublicRejectionStrikesBan verifies a public-class limiter rejection feeds the
// banlist: with a one-strike threshold the rejected IP is then banned.
func TestPublicRejectionStrikesBan(t *testing.T) {
bl := enabledBanlist(1)
limits := config.DefaultRateLimit()
limits.PublicPerMinute, limits.PublicBurst = 1, 1
_, client, cleanup := guardedEdge(t, bl, "", limits, func(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte(`{"token":"tok","user_id":"u-1","is_guest":true,"display_name":"Guest"}`))
})
defer cleanup()
if _, err := client.Execute(context.Background(), connect.NewRequest(&edgev1.ExecuteRequest{MessageType: transcode.MsgAuthGuest})); err != nil {
t.Fatalf("first execute: %v", err)
}
_, err := client.Execute(context.Background(), connect.NewRequest(&edgev1.ExecuteRequest{MessageType: transcode.MsgAuthGuest}))
if connect.CodeOf(err) != connect.CodeResourceExhausted {
t.Fatalf("second execute code = %v, want ResourceExhausted", connect.CodeOf(err))
}
if !bl.Banned("127.0.0.1") {
t.Fatal("a public rejection must strike the banlist")
}
}
// TestUserRejectionDoesNotBan verifies the user limiter class (keyed by account id,
// not IP) does not feed the IP banlist — that path is the backend's soft flag.
func TestUserRejectionDoesNotBan(t *testing.T) {
bl := enabledBanlist(1)
limits := config.DefaultRateLimit()
limits.UserPerMinute, limits.UserBurst = 1, 1
_, client, cleanup := guardedEdge(t, bl, "", limits, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/v1/internal/sessions/resolve":
_, _ = w.Write([]byte(`{"user_id":"u-1","is_guest":false}`))
case "/api/v1/user/feedback/unread":
_, _ = w.Write([]byte(`{"reply_unread":false}`))
default:
t.Errorf("unexpected backend path %s", r.URL.Path)
}
})
defer cleanup()
for i := range 2 {
req := connect.NewRequest(&edgev1.ExecuteRequest{MessageType: transcode.MsgFeedbackUnread})
req.Header().Set("Authorization", "Bearer tok")
_, err := client.Execute(context.Background(), req)
if i == 1 && connect.CodeOf(err) != connect.CodeResourceExhausted {
t.Fatalf("second execute code = %v, want ResourceExhausted", connect.CodeOf(err))
}
}
if len(bl.Active()) != 0 {
t.Fatalf("user-class rejection must not ban; active = %v", bl.Active())
}
}
// TestHoneytokenBansAndRejects verifies presenting the planted honeytoken bans the
// caller and returns the ordinary invalid-session error without a backend call.
func TestHoneytokenBansAndRejects(t *testing.T) {
bl := enabledBanlist(100)
_, client, cleanup := guardedEdge(t, bl, "s3cr3t-trap", config.DefaultRateLimit(), func(w http.ResponseWriter, r *http.Request) {
t.Error("backend must not be called for the honeytoken")
})
defer cleanup()
req := connect.NewRequest(&edgev1.ExecuteRequest{MessageType: transcode.MsgProfileGet})
req.Header().Set("Authorization", "Bearer s3cr3t-trap")
_, err := client.Execute(context.Background(), req)
if connect.CodeOf(err) != connect.CodeUnauthenticated {
t.Fatalf("honeytoken code = %v, want Unauthenticated", connect.CodeOf(err))
}
if !bl.Banned("127.0.0.1") {
t.Fatal("the honeytoken must ban the caller")
}
}
+13 -1
View File
@@ -26,6 +26,7 @@ var activeUserWindows = []struct {
type serverMetrics struct { type serverMetrics struct {
edge metric.Float64Histogram edge metric.Float64Histogram
rateLimited metric.Int64Counter rateLimited metric.Int64Counter
banned metric.Int64Counter
active *activeUsers active *activeUsers
} }
@@ -48,7 +49,12 @@ func newServerMetrics(meter metric.Meter) *serverMetrics {
if err != nil { if err != nil {
c, _ = noop.NewMeterProvider().Meter(meterName).Int64Counter("gateway_rate_limited_total") c, _ = noop.NewMeterProvider().Meter(meterName).Int64Counter("gateway_rate_limited_total")
} }
m := &serverMetrics{edge: h, rateLimited: c, active: newActiveUsers()} b, err := meter.Int64Counter("gateway_abuse_banned_total",
metric.WithDescription("Temporary IP bans applied at the edge, by reason (rejections, tripwire or honeytoken)."))
if err != nil {
b, _ = noop.NewMeterProvider().Meter(meterName).Int64Counter("gateway_abuse_banned_total")
}
m := &serverMetrics{edge: h, rateLimited: c, banned: b, active: newActiveUsers()}
gauge, err := meter.Int64ObservableGauge("active_users", gauge, err := meter.Int64ObservableGauge("active_users",
metric.WithDescription("Distinct accounts that performed an authenticated action within the window (in-memory, single gateway instance).")) metric.WithDescription("Distinct accounts that performed an authenticated action within the window (in-memory, single gateway instance)."))
@@ -86,3 +92,9 @@ func (m *serverMetrics) recordActive(uid string) {
func (m *serverMetrics) recordRateLimited(ctx context.Context, class string) { func (m *serverMetrics) recordRateLimited(ctx context.Context, class string) {
m.rateLimited.Add(ctx, 1, metric.WithAttributes(attribute.String("class", class))) m.rateLimited.Add(ctx, 1, metric.WithAttributes(attribute.String("class", class)))
} }
// recordBan counts one temporary IP ban under reason (rejections, tripwire or
// honeytoken).
func (m *serverMetrics) recordBan(ctx context.Context, reason string) {
m.banned.Add(ctx, 1, metric.WithAttributes(attribute.String("reason", reason)))
}
@@ -90,3 +90,41 @@ func TestRateLimitedMetric(t *testing.T) {
t.Errorf("rate_limited counts = %v, want user=2 public=1", counts) t.Errorf("rate_limited counts = %v, want user=2 public=1", counts)
} }
} }
// TestBannedMetric records ban events through a manual reader and asserts
// gateway_abuse_banned_total splits by reason.
func TestBannedMetric(t *testing.T) {
ctx := context.Background()
reader := sdkmetric.NewManualReader()
meter := sdkmetric.NewMeterProvider(sdkmetric.WithReader(reader)).Meter("test")
m := newServerMetrics(meter)
m.recordBan(ctx, "tripwire")
m.recordBan(ctx, "tripwire")
m.recordBan(ctx, "honeytoken")
var rm metricdata.ResourceMetrics
if err := reader.Collect(ctx, &rm); err != nil {
t.Fatalf("collect: %v", err)
}
counts := map[string]int64{}
for _, sm := range rm.ScopeMetrics {
for _, md := range sm.Metrics {
if md.Name != "gateway_abuse_banned_total" {
continue
}
sum, ok := md.Data.(metricdata.Sum[int64])
if !ok {
t.Fatalf("gateway_abuse_banned_total is not an int64 sum")
}
for _, dp := range sum.DataPoints {
reason, _ := dp.Attributes.Value(attribute.Key("reason"))
counts[reason.AsString()] += dp.Value
}
}
}
if counts["tripwire"] != 2 || counts["honeytoken"] != 1 {
t.Errorf("banned counts = %v, want tripwire=2 honeytoken=1", counts)
}
}
+72 -7
View File
@@ -8,6 +8,7 @@ package connectsrv
import ( import (
"context" "context"
"crypto/subtle"
"errors" "errors"
"net" "net"
"net/http" "net/http"
@@ -34,6 +35,12 @@ import (
// heartbeatKind is the live-stream keep-alive event kind. // heartbeatKind is the live-stream keep-alive event kind.
const heartbeatKind = "heartbeat" const heartbeatKind = "heartbeat"
// honeypotHeader marks a request the edge proxy routed from a honeypot decoy path;
// any request carrying it is treated as a scanner hit. The proxy strips any
// client-supplied value before setting its own, and a spoofed value only bans the
// spoofer, so trusting it is safe.
const honeypotHeader = "X-Scrabble-Honeypot"
// Limiter classes, the `class` attribute of gateway_rate_limited_total and the // Limiter classes, the `class` attribute of gateway_rate_limited_total and the
// class field of the periodic rejection report. // class field of the periodic rejection report.
const ( const (
@@ -63,6 +70,8 @@ type Server struct {
sessions *session.Cache sessions *session.Cache
limiter *ratelimit.Limiter limiter *ratelimit.Limiter
tracker *ratelimit.Tracker tracker *ratelimit.Tracker
banlist *ratelimit.Banlist
honeytoken string
hub *push.Hub hub *push.Hub
heartbeat time.Duration heartbeat time.Duration
log *zap.Logger log *zap.Logger
@@ -86,7 +95,13 @@ type Deps struct {
// Tracker accumulates limiter rejections for the periodic report; nil // Tracker accumulates limiter rejections for the periodic report; nil
// selects a private tracker (rejections are then only counted, never // selects a private tracker (rejections are then only counted, never
// reported). // reported).
Tracker *ratelimit.Tracker Tracker *ratelimit.Tracker
// Banlist enforces temporary IP bans on the hot path; nil selects a disabled
// (inert) banlist.
Banlist *ratelimit.Banlist
// Honeytoken, when non-empty, is the planted bearer value whose presentation
// bans the caller and raises a high-severity alarm.
Honeytoken string
Hub *push.Hub Hub *push.Hub
RateLimit config.RateLimitConfig RateLimit config.RateLimitConfig
Heartbeat time.Duration Heartbeat time.Duration
@@ -116,6 +131,10 @@ func NewServer(d Deps) *Server {
if limiter == nil { if limiter == nil {
limiter = ratelimit.New() limiter = ratelimit.New()
} }
banlist := d.Banlist
if banlist == nil {
banlist = ratelimit.NewBanlist(ratelimit.BanConfig{})
}
rl := d.RateLimit rl := d.RateLimit
if rl == (config.RateLimitConfig{}) { if rl == (config.RateLimitConfig{}) {
rl = config.DefaultRateLimit() rl = config.DefaultRateLimit()
@@ -125,6 +144,8 @@ func NewServer(d Deps) *Server {
sessions: d.Sessions, sessions: d.Sessions,
limiter: limiter, limiter: limiter,
tracker: tracker, tracker: tracker,
banlist: banlist,
honeytoken: d.Honeytoken,
hub: d.Hub, hub: d.Hub,
heartbeat: d.Heartbeat, heartbeat: d.Heartbeat,
log: log, log: log,
@@ -172,9 +193,11 @@ func (s *Server) HTTPHandler() http.Handler {
mux.Handle("/telegram/", webui.Handler("/telegram/", "index.html")) mux.Handle("/telegram/", webui.Handler("/telegram/", "index.html"))
mux.Handle("/app/", webui.Handler("/app/", "index.html")) mux.Handle("/app/", webui.Handler("/app/", "index.html"))
mux.Handle("/", http.RedirectHandler("/app/", http.StatusPermanentRedirect)) mux.Handle("/", http.RedirectHandler("/app/", http.StatusPermanentRedirect))
// Every request body on the public listener is capped (the admin proxy POSTs // abuseGuard is the outermost wrap (right under h2c) so a banned IP or a
// included); the h2c server carries explicit stream/idle sizing. // honeypot hit is turned away before the body cap and the mux. Every request
return h2c.NewHandler(maxBodyHandler(s.maxBodyBytes, mux), &http2.Server{ // body on the public listener is then capped (the admin proxy POSTs included);
// the h2c server carries explicit stream/idle sizing.
return h2c.NewHandler(s.abuseGuard(maxBodyHandler(s.maxBodyBytes, mux)), &http2.Server{
MaxConcurrentStreams: h2cMaxConcurrentStreams, MaxConcurrentStreams: h2cMaxConcurrentStreams,
IdleTimeout: h2cIdleTimeout, IdleTimeout: h2cIdleTimeout,
}) })
@@ -189,6 +212,32 @@ func maxBodyHandler(limit int, next http.Handler) http.Handler {
}) })
} }
// abuseGuard refuses a banned client IP with 429 before any work, and turns a
// honeypot decoy hit (the proxy-set honeypotHeader) into an instant ban plus a
// bland 404 that is indistinguishable from an ordinary miss. The ban check and the
// tripwire ban are inert on a disabled banlist (the prod-only gate); the tripwire
// hit is logged either way as scanner telemetry.
func (s *Server) abuseGuard(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
ip := peerIP(r.RemoteAddr, r.Header)
if s.banlist.Banned(ip) {
http.Error(w, "banned", http.StatusTooManyRequests)
return
}
if r.Header.Get(honeypotHeader) != "" {
s.log.Warn("honeypot tripwire",
zap.String("path", r.URL.Path),
zap.String("client_ip", ip))
if s.banlist.BanNow(ip, ratelimit.ReasonTripwire) {
s.metrics.recordBan(r.Context(), string(ratelimit.ReasonTripwire))
}
http.NotFound(w, r)
return
}
next.ServeHTTP(w, r)
})
}
// Execute runs one unary operation. Domain failures are returned in the envelope // Execute runs one unary operation. Domain failures are returned in the envelope
// (result_code != "ok", HTTP 200); only edge failures (rate limit, missing // (result_code != "ok", HTTP 200); only edge failures (rate limit, missing
// session, unknown type, internal) become Connect errors. // session, unknown type, internal) become Connect errors.
@@ -207,7 +256,7 @@ func (s *Server) Execute(ctx context.Context, req *connect.Request[edgev1.Execut
tr := transcode.Request{Payload: req.Msg.GetPayload(), ClientIP: clientIP} tr := transcode.Request{Payload: req.Msg.GetPayload(), ClientIP: clientIP}
if op.Auth { if op.Auth {
uid, isGuest, err := s.resolve(ctx, req.Header()) uid, isGuest, err := s.resolve(ctx, req.Header(), clientIP)
if err != nil { if err != nil {
result = "unauthenticated" result = "unauthenticated"
return nil, err return nil, err
@@ -263,7 +312,7 @@ func (s *Server) Execute(ctx context.Context, req *connect.Request[edgev1.Execut
// Subscribe streams the authenticated user's live events with a keep-alive // Subscribe streams the authenticated user's live events with a keep-alive
// heartbeat until the client disconnects. // heartbeat until the client disconnects.
func (s *Server) Subscribe(ctx context.Context, req *connect.Request[edgev1.SubscribeRequest], stream *connect.ServerStream[edgev1.Event]) error { func (s *Server) Subscribe(ctx context.Context, req *connect.Request[edgev1.SubscribeRequest], stream *connect.ServerStream[edgev1.Event]) error {
uid, _, err := s.resolve(ctx, req.Header()) uid, _, err := s.resolve(ctx, req.Header(), peerIP(req.Peer().Addr, req.Header()))
if err != nil { if err != nil {
return err return err
} }
@@ -311,6 +360,12 @@ func (s *Server) Subscribe(ctx context.Context, req *connect.Request[edgev1.Subs
func (s *Server) noteRateLimited(ctx context.Context, class, key, msgType string) { func (s *Server) noteRateLimited(ctx context.Context, class, key, msgType string) {
s.metrics.recordRateLimited(ctx, class) s.metrics.recordRateLimited(ctx, class)
s.tracker.Add(class, key) s.tracker.Add(class, key)
// IP-keyed rejections (public, email, admin — the key is the client IP) feed
// the ban; the user class is keyed by account id and is the backend soft-flag's
// concern, not the IP ban's.
if class != classUser && s.banlist.Strike(key) {
s.metrics.recordBan(ctx, string(ratelimit.ReasonRejections))
}
s.log.Debug("rate limited", s.log.Debug("rate limited",
zap.String("class", class), zap.String("class", class),
zap.String("key", key), zap.String("key", key),
@@ -344,11 +399,21 @@ func (s *Server) limitAdmin(next http.Handler) http.Handler {
// resolve extracts and resolves the Authorization bearer token to an account id // resolve extracts and resolves the Authorization bearer token to an account id
// and its guest flag, returning a Connect Unauthenticated error when it is missing // and its guest flag, returning a Connect Unauthenticated error when it is missing
// or unknown. // or unknown.
func (s *Server) resolve(ctx context.Context, h http.Header) (string, bool, error) { func (s *Server) resolve(ctx context.Context, h http.Header, clientIP string) (string, bool, error) {
token := bearerToken(h.Get("Authorization")) token := bearerToken(h.Get("Authorization"))
if token == "" { if token == "" {
return "", false, connect.NewError(connect.CodeUnauthenticated, errMissingToken) return "", false, connect.NewError(connect.CodeUnauthenticated, errMissingToken)
} }
// The honeytoken is a planted value no real client holds: presenting it is a
// high-confidence intrusion signal, so ban the caller and raise the alarm, then
// return the ordinary invalid-session error so the trap stays indistinguishable.
if s.honeytoken != "" && subtle.ConstantTimeCompare([]byte(token), []byte(s.honeytoken)) == 1 {
s.log.Warn("honeytoken presented", zap.String("client_ip", clientIP))
if s.banlist.BanNow(clientIP, ratelimit.ReasonHoneytoken) {
s.metrics.recordBan(ctx, string(ratelimit.ReasonHoneytoken))
}
return "", false, connect.NewError(connect.CodeUnauthenticated, errInvalidSession)
}
uid, isGuest, err := s.sessions.Resolve(ctx, token) uid, isGuest, err := s.sessions.Resolve(ctx, token)
if err != nil { if err != nil {
// An unknown or expired token (a backend 4xx) is the client's problem and // An unknown or expired token (a backend 4xx) is the client's problem and
+211
View File
@@ -0,0 +1,211 @@
package ratelimit
import (
"sort"
"sync"
"time"
)
// Reason labels why a client IP was banned; it is the reason attribute of the
// gateway_abuse_banned_total metric and a field of the ban report to the backend.
type Reason string
const (
// ReasonRejections is a ban earned by sustained rate-limiter rejections: the IP
// accumulated BanConfig.Threshold strikes within BanConfig.Window.
ReasonRejections Reason = "rejections"
// ReasonTripwire is an instant ban from a honeypot decoy-path hit.
ReasonTripwire Reason = "tripwire"
// ReasonHoneytoken is an instant ban from a planted credential being presented.
ReasonHoneytoken Reason = "honeytoken"
)
// Ban durations for the high-confidence reasons. A rejection ban uses the
// configured BanConfig.Duration; a tripwire or honeytoken hit is near
// zero-false-positive, so it earns a markedly longer ban.
const (
tripwireBanDuration = time.Hour
honeytokenBanDuration = 24 * time.Hour
)
// BanConfig tunes the temporary IP ban. Enabled gates the whole mechanism — kept
// off where the real client IP is not visible (e.g. every client arriving as one
// shared NAT address); when false every method is inert.
type BanConfig struct {
// Enabled turns enforcement on. While false Strike/BanNow record nothing,
// Banned is always false and Active is empty.
Enabled bool
// Threshold is the strike count within Window that earns a rejection ban.
Threshold int
// Window is the rolling window the strikes accumulate over.
Window time.Duration
// Duration is the length of a rejection ban (tripwire/honeytoken use their own).
Duration time.Duration
}
// Ban is a snapshot of one active ban for the periodic report and the admin view.
// Its JSON shape is the gateway→backend ban-sync wire contract.
type Ban struct {
IP string `json:"ip"`
Reason Reason `json:"reason"`
Since time.Time `json:"since"`
Expires time.Time `json:"expires"`
}
// Banlist is the gateway's in-memory temporary IP ban: a fail2ban-style block fed
// by sustained rate-limiter rejections (Strike) and by instant honeypot /
// honeytoken hits (BanNow), enforced on the hot path by Banned and cleared either
// by lapse or by an operator Unban. Entries are swept lazily so an expired ban
// does not leak memory. Like the rate limiter it is single-instance and resets on
// restart by design.
type Banlist struct {
cfg BanConfig
now func() time.Time
mu sync.Mutex
entries map[string]*banEntry
lastSweep time.Time
}
// banEntry is one IP's ban state: a live ban (until set) or, before the
// threshold, only the recent strike times.
type banEntry struct {
reason Reason
since time.Time
until time.Time // zero while only accumulating strikes
strikes []time.Time // strike times within the window; pruned on each strike
}
// NewBanlist constructs a Banlist with cfg.
func NewBanlist(cfg BanConfig) *Banlist {
return &Banlist{cfg: cfg, now: time.Now, entries: make(map[string]*banEntry)}
}
// Strike records one rate-limiter rejection for ip and reports whether it earned a
// ban (Threshold strikes within Window). It is a no-op on a disabled banlist.
func (b *Banlist) Strike(ip string) bool {
if !b.cfg.Enabled {
return false
}
b.mu.Lock()
defer b.mu.Unlock()
now := b.now()
b.sweepLocked(now)
e := b.entries[ip]
if e == nil {
e = &banEntry{}
b.entries[ip] = e
}
if now.Before(e.until) {
return false // already banned (a banned IP normally never reaches here)
}
cutoff := now.Add(-b.cfg.Window)
kept := e.strikes[:0]
for _, t := range e.strikes {
if t.After(cutoff) {
kept = append(kept, t)
}
}
e.strikes = append(kept, now)
if len(e.strikes) >= b.cfg.Threshold {
e.reason = ReasonRejections
e.since = now
e.until = now.Add(b.cfg.Duration)
e.strikes = nil
return true
}
return false
}
// BanNow bans ip immediately for reason and reports whether a ban is now in effect
// (true on an enabled banlist, false when disabled).
func (b *Banlist) BanNow(ip string, reason Reason) bool {
if !b.cfg.Enabled {
return false
}
b.mu.Lock()
defer b.mu.Unlock()
now := b.now()
b.sweepLocked(now)
e := b.entries[ip]
if e == nil {
e = &banEntry{}
b.entries[ip] = e
}
e.reason = reason
e.since = now
e.until = now.Add(banDuration(reason, b.cfg.Duration))
e.strikes = nil
return true
}
// Banned reports whether ip is currently banned. It is always false on a disabled
// banlist.
func (b *Banlist) Banned(ip string) bool {
if !b.cfg.Enabled {
return false
}
b.mu.Lock()
defer b.mu.Unlock()
e := b.entries[ip]
return e != nil && b.now().Before(e.until)
}
// Unban clears any ban and accumulated strikes for ip.
func (b *Banlist) Unban(ip string) {
b.mu.Lock()
defer b.mu.Unlock()
delete(b.entries, ip)
}
// Active returns a snapshot of the currently-banned IPs, most recently banned
// first. It is empty on a disabled banlist.
func (b *Banlist) Active() []Ban {
if !b.cfg.Enabled {
return nil
}
b.mu.Lock()
defer b.mu.Unlock()
now := b.now()
out := make([]Ban, 0, len(b.entries))
for ip, e := range b.entries {
if now.Before(e.until) {
out = append(out, Ban{IP: ip, Reason: e.reason, Since: e.since, Expires: e.until})
}
}
sort.Slice(out, func(i, j int) bool { return out[i].Since.After(out[j].Since) })
return out
}
// banDuration maps a reason to its ban length.
func banDuration(reason Reason, rejectionDuration time.Duration) time.Duration {
switch reason {
case ReasonTripwire:
return tripwireBanDuration
case ReasonHoneytoken:
return honeytokenBanDuration
default:
return rejectionDuration
}
}
// sweepLocked discards lapsed bans and stale strike-only entries, at most once per
// sweepInterval. The caller holds b.mu.
func (b *Banlist) sweepLocked(now time.Time) {
if now.Sub(b.lastSweep) < sweepInterval {
return
}
b.lastSweep = now
cutoff := now.Add(-b.cfg.Window)
for ip, e := range b.entries {
if !e.until.IsZero() {
if !now.Before(e.until) {
delete(b.entries, ip) // ban lapsed
}
continue
}
if len(e.strikes) == 0 || !e.strikes[len(e.strikes)-1].After(cutoff) {
delete(b.entries, ip) // strike-only entry gone stale
}
}
}
+143
View File
@@ -0,0 +1,143 @@
package ratelimit
import (
"testing"
"time"
)
// banlistAt builds an enabled banlist whose clock the test drives through clk.
func banlistAt(clk *time.Time, cfg BanConfig) *Banlist {
bl := NewBanlist(cfg)
bl.now = func() time.Time { return *clk }
return bl
}
func enabledCfg() BanConfig {
return BanConfig{Enabled: true, Threshold: 3, Window: time.Minute, Duration: 15 * time.Minute}
}
func TestBanlistStrikeThreshold(t *testing.T) {
clk := time.Date(2026, 6, 21, 12, 0, 0, 0, time.UTC)
bl := banlistAt(&clk, enabledCfg())
if bl.Strike("1.2.3.4") {
t.Fatal("first strike must not ban")
}
if bl.Strike("1.2.3.4") {
t.Fatal("second strike must not ban")
}
if bl.Banned("1.2.3.4") {
t.Fatal("must not be banned before the third strike")
}
if !bl.Strike("1.2.3.4") {
t.Fatal("third strike within the window must ban")
}
if !bl.Banned("1.2.3.4") {
t.Fatal("must be banned after the threshold strike")
}
}
func TestBanlistStrikeWindowResets(t *testing.T) {
clk := time.Date(2026, 6, 21, 12, 0, 0, 0, time.UTC)
bl := banlistAt(&clk, enabledCfg())
// Strikes spaced wider than the window never accumulate to a ban.
for i := range 5 {
if bl.Strike("9.9.9.9") {
t.Fatalf("strike %d should not ban: each falls outside the previous window", i)
}
clk = clk.Add(2 * time.Minute)
}
if bl.Banned("9.9.9.9") {
t.Fatal("strikes outside the rolling window must not ban")
}
}
func TestBanlistBanExpires(t *testing.T) {
clk := time.Date(2026, 6, 21, 12, 0, 0, 0, time.UTC)
bl := banlistAt(&clk, enabledCfg())
bl.BanNow("5.5.5.5", ReasonRejections)
if !bl.Banned("5.5.5.5") {
t.Fatal("must be banned right after BanNow")
}
clk = clk.Add(15*time.Minute + time.Second)
if bl.Banned("5.5.5.5") {
t.Fatal("ban must lapse after its duration")
}
}
func TestBanlistReasonDurations(t *testing.T) {
clk := time.Date(2026, 6, 21, 12, 0, 0, 0, time.UTC)
bl := banlistAt(&clk, enabledCfg())
bl.BanNow("a", ReasonTripwire) // 1h
bl.BanNow("b", ReasonHoneytoken) // 24h
clk = clk.Add(90 * time.Minute)
if bl.Banned("a") {
t.Fatal("tripwire ban (1h) must have lapsed after 90m")
}
if !bl.Banned("b") {
t.Fatal("honeytoken ban (24h) must still hold after 90m")
}
}
func TestBanlistUnban(t *testing.T) {
clk := time.Date(2026, 6, 21, 12, 0, 0, 0, time.UTC)
bl := banlistAt(&clk, enabledCfg())
bl.BanNow("7.7.7.7", ReasonTripwire)
bl.Unban("7.7.7.7")
if bl.Banned("7.7.7.7") {
t.Fatal("Unban must clear the ban")
}
}
func TestBanlistActiveSnapshot(t *testing.T) {
clk := time.Date(2026, 6, 21, 12, 0, 0, 0, time.UTC)
bl := banlistAt(&clk, enabledCfg())
bl.BanNow("a", ReasonTripwire)
bl.BanNow("b", ReasonHoneytoken)
active := bl.Active()
if len(active) != 2 {
t.Fatalf("Active = %d bans, want 2", len(active))
}
byIP := map[string]Ban{}
for _, b := range active {
byIP[b.IP] = b
}
if byIP["a"].Reason != ReasonTripwire || byIP["b"].Reason != ReasonHoneytoken {
t.Fatalf("Active reasons = %+v", byIP)
}
if !byIP["a"].Expires.After(byIP["a"].Since) {
t.Fatal("Expires must be after Since")
}
clk = clk.Add(2 * time.Hour) // tripwire (1h) lapses, honeytoken (24h) holds
if got := len(bl.Active()); got != 1 {
t.Fatalf("Active after 2h = %d, want 1 (only honeytoken)", got)
}
}
func TestBanlistDisabledIsInert(t *testing.T) {
clk := time.Date(2026, 6, 21, 12, 0, 0, 0, time.UTC)
cfg := enabledCfg()
cfg.Enabled = false
bl := banlistAt(&clk, cfg)
for range 10 {
if bl.Strike("1.1.1.1") {
t.Fatal("disabled banlist must never ban via Strike")
}
}
if bl.BanNow("2.2.2.2", ReasonHoneytoken) {
t.Fatal("disabled banlist must never ban via BanNow")
}
if bl.Banned("1.1.1.1") || bl.Banned("2.2.2.2") {
t.Fatal("disabled banlist must report nothing banned")
}
if len(bl.Active()) != 0 {
t.Fatal("disabled banlist must expose no active bans")
}
}
+1 -1
View File
@@ -12,7 +12,7 @@
# --- dictionary artifact ----------------------------------------------------- # --- dictionary artifact -----------------------------------------------------
FROM alpine:3.20 AS dawg FROM alpine:3.20 AS dawg
ARG DICT_VERSION=v1.0.0 ARG DICT_VERSION=v1.2.1
RUN apk add --no-cache curl tar RUN apk add --no-cache curl tar
RUN mkdir -p /dawg \ RUN mkdir -p /dawg \
&& curl -fsSL -o /tmp/dawg.tar.gz \ && curl -fsSL -o /tmp/dawg.tar.gz \
+15 -11
View File
@@ -1,6 +1,6 @@
# loadtest — stress harness # loadtest — stress harness
Reusable load harness for the pre-release stress pass. It Reusable load/stress harness. It
seeds a large account population with pre-created sessions, drives virtual players seeds a large account population with pre-created sessions, drives virtual players
through the **gateway edge protocol** in realistic games, hammers the rate limiter, through the **gateway edge protocol** in realistic games, hammers the rate limiter,
and prints a trip-report summary. It stays in the repo for repeats. and prints a trip-report summary. It stays in the repo for repeats.
@@ -15,10 +15,12 @@ and prints a trip-report summary. It stays in the repo for repeats.
2. **Drive** (edge protocol over h2c): assembles real 24 player games via the 2. **Drive** (edge protocol over h2c): assembles real 24 player games via the
invitation flow (`invitation.create``invitation.accept`, no robots), then runs invitation flow (`invitation.create``invitation.accept`, no robots), then runs
each player's turn loop — poll `game.state`, replay `game.history`, generate a legal each player's turn loop — poll `game.state`, replay `game.history`, generate a legal
**mid-ranked** move with the embedded `scrabble-solver`, and `game.submit_play` **mid-ranked** move with the embedded `scrabble-solver`, **compose it tile by tile with
(or pass/exchange). A fraction of turns exercise nudge / chat / check-word / draft / the debounced `game.evaluate` preview a real client fires** (the hottest gameplay call),
profile-update / stats. Each player also holds a live `Subscribe` stream. The persist a `draft.save`, and `game.submit_play` (or pass/exchange). A fraction of turns
moderate ramp is **50 → 200 → 500** concurrent players, ~12 min per step. exercise nudge / chat / check-word / draft / profile-update / stats. Each player also
holds a live `Subscribe` stream. The moderate ramp is **50 → 200 → 500** concurrent
players, ~12 min per step. `--eval=false` drops the evaluate model for an A/B baseline.
3. **Hammer**: drives `games.list` from one account far above the per-user rate limit 3. **Hammer**: drives `games.list` from one account far above the per-user rate limit
to verify the limiter holds (`rate_limited` results) and measure its cost. to verify the limiter holds (`rate_limited` results) and measure its cost.
4. **Report**: per-operation latency percentiles, throughput, result-code breakdown, 4. **Report**: per-operation latency percentiles, throughput, result-code breakdown,
@@ -72,6 +74,8 @@ Key `run` flags (env in parentheses):
| `--games-per-player` | `0` (random 35) | target concurrent games per player | | `--games-per-player` | `0` (random 35) | target concurrent games per player |
| `--tick` | `800ms` | per-player op cadence (keeps a player under the per-user limit) | | `--tick` | `800ms` | per-player op cadence (keeps a player under the per-user limit) |
| `--secondary-prob` | `0.08` | chance per tick of a non-move op | | `--secondary-prob` | `0.08` | chance per tick of a non-move op |
| `--eval` | `true` | model the per-tile `game.evaluate` preview (the gameplay hot path); `false` reproduces the pre-evaluate harness |
| `--eval-recon` | `1` | extra full-composition evaluate re-previews per play (reconsideration), beyond one per placed tile |
| `--hammer-workers` / `--hammer-dur` | `20` / `15s` | gateway-hammer (0 workers disables) | | `--hammer-workers` / `--hammer-dur` | `20` / `15s` | gateway-hammer (0 workers disables) |
| `--reset` / `--cleanup` | `false` | delete harness rows before / after the run | | `--reset` / `--cleanup` | `false` | delete harness rows before / after the run |
@@ -93,16 +97,16 @@ runs unconditionally. Use an **absolute** path (here via `$PWD`): `go test ./loa
runs each package from its own directory, so a relative `BACKEND_DICT_DIR` would not runs each package from its own directory, so a relative `BACKEND_DICT_DIR` would not
resolve. resolve.
## Trip reports ## Trip report
The two stress passes are written up in the repo: the early pass in The stress findings — the final run, the `game.evaluate` hot-path model, the
[`REPORT-R2.md`](REPORT-R2.md) and the final, tuned pass in gateway→backend connection-pool fix, and the revised sizing — are written up in
[`REPORT-R7.md`](REPORT-R7.md). [`REPORT.md`](REPORT.md).
## Caveat ## Caveat
The harness shares the host CPU with the contour, so its own `scrabble-loadtest` The harness shares the host CPU with the contour, so its own `scrabble-loadtest`
container series is read alongside the system under test; capping it with `--cpus` container series is read alongside the system under test; capping it with `--cpus`
keeps the contour's quota. Per-player transports (R7) removed the shared-transport keeps the contour's quota. Per-player transports removed the shared-transport
artifact that inflated R2's `transport_error`, so the figures reflect the system. A artifact that previously inflated `transport_error`, so the figures reflect the system. A
fully isolated ceiling on separate hardware remains future work. fully isolated ceiling on separate hardware remains future work.

Some files were not shown because too many files have changed in this diff Show More