Ilia Denisov
9e0f929e40
Add an integration test asserting the guest-until-confirmed squat fix (an email-login account is a reapable guest until the code is confirmed, then durable). Document the branded relay pipeline in ARCHITECTURE (go-mail, TLS by port, no client cert, PUBLIC_BASE_URL anti-injection, per-recipient send throttle, guest-until-confirmed), FUNCTIONAL (+ru), TESTING and the backend README config table (BACKEND_SMTP_* + BACKEND_PUBLIC_BASE_URL).
296 lines
21 KiB
Markdown
296 lines
21 KiB
Markdown
# backend
|
||
|
||
Internal-only domain service for the Scrabble platform (module `scrabble/backend`).
|
||
It owns identity/sessions, accounts, the lobby, game runtime, robot, chat, history
|
||
and administration. Its only network consumers are the `gateway` and the platform
|
||
side-services; it is never exposed publicly.
|
||
|
||
The backend provides the foundation: configuration, the HTTP listener with the
|
||
`/api/v1` route-group skeleton and probes, the Postgres pool with embedded goose
|
||
migrations, OpenTelemetry wiring, an in-memory session cache, and the durable
|
||
accounts / identities / sessions data model. The session and account REST
|
||
endpoints live in the `gateway`; the backend ships the store/service layer they
|
||
call.
|
||
|
||
`internal/engine` is the in-process bridge to the `scrabble-solver`
|
||
library: a versioned dictionary registry, a deterministic tile bag, and a pure
|
||
rules `Game` (legal plays, passes, exchanges, resignations and end-condition
|
||
detection) that emits dictionary-independent move records. It is a library only;
|
||
the game domain wires it into the process.
|
||
|
||
`internal/game` is the game domain over the engine. Active games are
|
||
event-sourced: a `games` row plus an append-only decoded move journal, with the
|
||
live `engine.Game` kept warm in a cache and rebuilt by replay on a miss. It
|
||
provides create, the play/pass/exchange/resign transitions, an unlimited
|
||
word/score/legality preview, the hint (per-game allowance plus a profile wallet), the
|
||
word-check tool with complaint capture, per-player game state, history and GCG
|
||
export, per-account statistics on finish, and a background turn-timeout sweeper
|
||
that auto-resigns overdue turns (honouring each player's daily away window). Like
|
||
the engine it is a service/store layer; the HTTP surface lives in the `gateway`.
|
||
|
||
The lobby and social fabric. `internal/lobby` runs **auto-match** — `Enqueue` opens a
|
||
real game seating the caller with an **empty opponent seat** (status `open`) or, when
|
||
another player already waits for the same variant and per-turn word rule, seats the
|
||
caller into that open game and starts it — and
|
||
friend-game invitations (invite → accept, starting a 2–4 player game once every
|
||
invitee accepts). A **simultaneous-game cap** (`game.MaxActiveQuickGames` = 10) limits a
|
||
player's active quick games — status `active`/`open`, excluding invitation-linked friend
|
||
games (`game.Service.CountActiveQuickGames`); the server refuses `lobby/enqueue` and
|
||
`invitations` creation with **409 `game_limit_reached`** at the cap (accepting an invitation
|
||
is exempt), and the `games.list` response carries an `at_game_limit` flag for the lobby.
|
||
`internal/social` owns the friend graph (request/accept),
|
||
per-user blocks, and per-game chat with nudges folded in as a message kind; chat
|
||
messages are length-capped, content-filtered (no links/emails/phone numbers,
|
||
including obfuscated forms) and stored with the sender's IP. Each message carries an
|
||
`unread_seats` read bitmask (a set bit per recipient seat still to read it); `MarkRead`
|
||
clears a reader's bit when they open the move history or chat, a wired `NudgeClearer`
|
||
clears a nudge when its recipient moves, and a wired `NudgeExpirer` clears **all** of a game's
|
||
nudges when it finishes (any completion path) — the first two record the publish-to-read latency,
|
||
the completion expiry does not (it is not a read); chat messages stay unread on completion.
|
||
A friend request (or block) aimed at a **disguised pooled robot** is recorded per game+seat
|
||
in `robot_friend_requests` / `robot_blocks`, never against the shared robot account; a
|
||
background reaper drops a robot friend request once its game has been finished for **7 days**.
|
||
`internal/account`
|
||
gains profile editing and the email confirm-code flow (a `Mailer` seam: SMTP or a
|
||
development log mailer). The engine now also handles **multi-player drop-out**: in
|
||
a 3–4 player game a resignation or timeout drops that seat and the rest play on
|
||
(the tile disposition is a per-game setting), the game ending when one active seat
|
||
remains. As before this is a service/store layer — chat and nudges are persisted
|
||
but their live delivery, and all REST endpoints, live in the `gateway`; the
|
||
services are exposed via `Server` accessors for those handlers.
|
||
|
||
The robot opponent (`internal/robot`). A pool of durable accounts —
|
||
each a `kind='robot'` identity, provisioned at startup with chat and friend
|
||
requests blocked — carries human-like names. The disguised reaper stamps a **fresh per-game name** on
|
||
the robot's seat (a `game_players.display_name` snapshot — which also freezes humans' names per game, so
|
||
a later rename never rewrites past games) drawn from a wide composed corpus (Western locales, native
|
||
Japanese/Chinese, a gender-agreed Russian pool, and handles); a Russian game stays Cyrillic with ≤20%
|
||
Latin and no CJK script, an English game uses the full corpus (`namevariety.go`, `PickNamed`). A
|
||
background driver plays the
|
||
robot's moves through the public game API as an ordinary seated player (so only
|
||
`internal/engine` imports the solver): it decides once per game whether to play to
|
||
win (≈ 40%), targets a small score margin — with an occasional off-strategy move that tapers to
|
||
none as the bag empties — and times its moves with a move-number-aware
|
||
right-skewed delay (quick openings, long endgames), a night-sleep window anchored to the opponent's timezone, and nudge
|
||
behaviour — all derived deterministically from the game seed, so it keeps no extra
|
||
state. In a dead-drawn endgame — the last two journal moves are both passes, so the robot is bound
|
||
to pass again — it shortens that delay to a `[0.8, 1.5]×` band around the human's last-move think
|
||
time (the gap between the last two moves), clamped to `[30 s, 8 min]` and `min`-ed with the normal
|
||
delay, so a decided game is not dragged out while the robot never moves slower than usual. A background **reaper** seats a pooled robot (matching the game's language) in any open
|
||
game whose wait window — a fixed **90 s** plus a random **0–90 s** (so **90–180 s**) — has
|
||
elapsed, and the waiting starter is told an opponent took the seat by an in-app
|
||
**opponent_joined** push (carrying their refreshed game state) that fills the opponent card and
|
||
re-enables resign and chat in place.
|
||
|
||
The same robot also backs an **honest-AI quick game** (`games.vs_ai`), the alternative to the random
|
||
path that the player chooses on New Game. `Matchmaker.StartVsAI` picks a pooled robot and creates a
|
||
game **already seated and active** (random seat order) — it never enters the open pool. The robot
|
||
driver has a `vs_ai` branch (no sleep, no proactive nudge, zero delay) plus a focused
|
||
`DriveGame`/`TriggerMove` fast path wired from the game service's after-create/after-commit hook
|
||
(`SetAITrigger`), so the robot replies the instant the player moves. AI games keep the same strength
|
||
(`playToWin`), have **no per-move timeout** (`turn_timeout_secs = AIInactivityTimeout`, 7 days, so an
|
||
abandoned game is lost after a week of inactivity — only the human is ever on the clock), record **no
|
||
statistics** (skipped in `commit`), and disable chat/nudge (`social` `VsAI` → `ErrGameVsAI`).
|
||
|
||
The backend opens to the edge. The route groups gain their first
|
||
handlers (`internal/server/handlers_*.go`): gateway-only session endpoints under
|
||
`/api/v1/internal` (Telegram/guest/email login → mint, resolve, revoke) and a
|
||
slice of authenticated `/api/v1/user` operations (profile, submit play, game
|
||
state, lobby enqueue, chat). The social/account/history operations under
|
||
`/api/v1/user`: `friends/*` (request/respond/cancel/unfriend,
|
||
list/incoming, the one-time `code` issue/redeem), `blocks/*`, `invitations/*`
|
||
(create/accept/decline/cancel/list), `PUT profile`, `email/{request,confirm}`,
|
||
`stats`, and `games/:id/gcg` (finished-only). The `internal/notify` hub feeds a
|
||
second listener — `internal/pushgrpc`, a gRPC server (`BACKEND_GRPC_ADDR`) streaming
|
||
live events (your-turn, opponent-moved, chat, nudge, match-found, notify) to the
|
||
gateway. The gateway-only `POST /api/v1/internal/push-target` (a user's
|
||
Telegram `external_id`, language and `notifications_in_app_only` flag) lets the gateway
|
||
route out-of-app push to the Telegram bot over the gateway bot-link; the Telegram login
|
||
seeds a new account's language and display name from the launch fields, and the
|
||
`accounts.notifications_in_app_only` flag (default true).
|
||
The gateway-only `POST /api/v1/internal/chat-access` resolves a Telegram identity (the
|
||
bot's join-time query) or an account id (a `chat_access_changed` event) to its
|
||
**moderated-chat write eligibility** — `registered AND NOT suspended AND NOT chat_muted`.
|
||
That event is emitted on an admin block/unblock, a `chat_muted` role grant/revoke, or — via
|
||
the `account.SuspensionSweeper` started in `cmd/backend` — a temporary block lapsing;
|
||
`chat_muted` is an `account.KnownRoles` entry, a chat-only mute distinct from the game
|
||
suspension (which dominates it).
|
||
`accounts.is_guest` marks an ephemeral guest — a durable row
|
||
with no identity, excluded from statistics. The server-rendered
|
||
**admin console** at `/_gm` (`internal/adminconsole` + `internal/server/handlers_admin_console.go`;
|
||
the gateway fronts it with Basic-Auth and a same-origin guard protects its POSTs), the
|
||
**complaint resolution** lifecycle (the `complaints` `disposition`/`resolution_note`/
|
||
`resolved_at`/`applied_in_version` columns + the `status` CHECK) feeding a dictionary-change
|
||
pipeline, the online **dictionary update** (upload the `scrabble-dawg-vX.Y.Z.tar.gz` release
|
||
archive, preview the per-variant word diff, then install + activate — `internal/dictadmin` +
|
||
`engine.DiffWords` / `Registry.LoadAvailable`, written to per-version subdirectories of the
|
||
`BACKEND_DICT_DIR` volume with the active version persisted in `dictionary_state`), and operator **broadcasts** via a
|
||
backend client (`internal/connector`, `BACKEND_CONNECTOR_ADDR`) that calls the gateway's
|
||
**bot-link relay** — each broadcast renders through the bot in an operator-chosen language
|
||
and the relay awaits the bot's delivery ack. There is one bot,
|
||
so `/internal/push-target` returns the recipient's `preferred_language` as the render
|
||
language for out-of-app push; no per-bot routing remains. The console also manages the **advertising banner** (`/_gm/banners` +
|
||
`/_gm/banner-settings`, `internal/ads`): operator campaigns with a percent weight, an optional
|
||
window and bilingual messages, plus the global display timings. `GET /api/v1/user/profile` attaches
|
||
the resolved, weighted campaign feed for an **eligible** viewer (`!paid_account && hint_balance == 0
|
||
&& !no_banner` role, the message language picked by `preferred_language`); changing those inputs
|
||
publishes a `notify` `banner` re-poll signal so the client shows/hides it in place. The shared wire
|
||
contracts live in the sibling [`../pkg`](../pkg) module.
|
||
|
||
**Account linking & merge** (`/api/v1/user/link/*`). `internal/link`
|
||
orchestrates it: an email confirm-code or a gateway-validated Telegram identity is
|
||
attached to the current account, and when the identity already has its own account
|
||
the two are merged in one transaction (`internal/accountmerge`) — stats and the hint
|
||
wallet summed, `paid_account` ORed, identities/games/chat/complaints transferred,
|
||
friends/blocks de-duplicated, the secondary kept as a `merged_into` tombstone (so a
|
||
shared finished game's foreign keys hold); a shared **active** game blocks the merge.
|
||
The current account is primary, except a guest initiator whose linked identity has a
|
||
durable owner — then the durable account wins and a fresh session is minted for it.
|
||
The `accounts.paid_account`/`merged_into`/`merged_at` columns back this. This supersedes the
|
||
former `email.bind.*` edge surface (the `RequestCode`/`ConfirmCode` primitives stay).
|
||
|
||
Rate-limit observability: the gateway posts its periodic rejection
|
||
summaries to `POST /api/v1/internal/ratelimit/report`; `internal/ratewatch` keeps a
|
||
bounded in-memory episode window for the console's **Throttled** page and applies the
|
||
conservative auto-flag — an account sustaining `BACKEND_HIGHRATE_FLAG_THRESHOLD`
|
||
rejected calls within `BACKEND_HIGHRATE_FLAG_WINDOW` gets the soft, reversible
|
||
`accounts.flagged_high_rate_at` marker (set-once; a badge in the user list and a
|
||
**Clear** action on the user card; never an automatic ban).
|
||
|
||
The gateway also syncs its active IP bans (prod-only — see ARCHITECTURE §11) to
|
||
`POST /api/v1/internal/bans/sync`; `internal/banview` mirrors them for the console's
|
||
**Throttled** page (an **Active IP bans** panel with an **Unban** action) and returns
|
||
the operator's pending unbans in the response, which the gateway applies on its next
|
||
sync. Like `ratewatch` it is in-memory and resets on restart — the enforced ban lives
|
||
in the gateway, not here.
|
||
|
||
## Package layout
|
||
|
||
```
|
||
cmd/backend/ # entrypoint: telemetry -> db+migrate -> registry -> cache -> game+sweeper -> robot pool+driver -> lobby+social -> server
|
||
cmd/jetgen/ # dev tool: regenerate go-jet code from a throwaway container
|
||
internal/config/ # env configuration (composes postgres + telemetry + game config)
|
||
internal/telemetry/ # OpenTelemetry providers + per-request timing middleware
|
||
internal/postgres/ # pgx-over-database/sql pool (otelsql), goose migrations
|
||
migrations/ # embedded *.sql (goose), schema `backend`
|
||
jet/ # generated go-jet models + table builders (committed)
|
||
internal/account/ # durable accounts + platform/email identities (store) + email/identity link primitives
|
||
internal/accountmerge/ # single-transaction merge of a secondary account into a primary
|
||
internal/link/ # link/merge orchestrator over account + accountmerge + session
|
||
internal/session/ # opaque tokens, sessions store, write-through cache, service (incl. RevokeAllForAccount)
|
||
internal/server/ # gin engine, route groups, X-User-ID middleware, probes
|
||
internal/engine/ # in-process scrabble-solver bridge: registry, bag, Game, replay
|
||
internal/game/ # game domain: lifecycle, journal+cache, hint, word-check, GCG, sweeper
|
||
internal/social/ # friend graph, per-user blocks, per-game chat + nudge, content filter
|
||
internal/feedback/ # user feedback: messages + attachment (bytea), anti-spam gate, admin review/reply
|
||
internal/lobby/ # auto-match (DB-backed open games + robot substitution) + friend-game invitations
|
||
internal/robot/ # human-like robot opponent: account pool, seed-derived strategy, move driver
|
||
internal/adminconsole/ # server-rendered admin console (Go templates + embedded CSS, view models), served at /_gm
|
||
internal/ads/ # advertising banner: campaigns + bilingual messages + display timings, weighted-rotation feed (ActiveSet)
|
||
internal/connector/ # backend gRPC client to the gateway bot-link relay (operator broadcasts)
|
||
internal/ratewatch/ # gateway rate-limit reports: episode window for the console + the high-rate auto-flag
|
||
internal/banview/ # gateway active-ban mirror: the console's Active IP bans panel + the operator unban backchannel
|
||
```
|
||
|
||
## Configuration (environment)
|
||
|
||
| Variable | Default | Notes |
|
||
| --- | --- | --- |
|
||
| `BACKEND_HTTP_ADDR` | `:8080` | HTTP (REST) listen address. |
|
||
| `BACKEND_GRPC_ADDR` | `:9090` | gRPC listen address for the live-event push stream to the gateway. |
|
||
| `BACKEND_LOG_LEVEL` | `info` | `debug` / `info` / `warn` / `error`. |
|
||
| `BACKEND_POSTGRES_DSN` | — | **Required.** pgx/libpq URL; must pin `search_path=backend`. |
|
||
| `BACKEND_POSTGRES_MAX_OPEN_CONNS` | `25` | Pool max open connections. |
|
||
| `BACKEND_POSTGRES_MAX_IDLE_CONNS` | `5` | Pool max idle connections. |
|
||
| `BACKEND_POSTGRES_CONN_MAX_LIFETIME` | `30m` | Max connection lifetime. |
|
||
| `BACKEND_POSTGRES_OPERATION_TIMEOUT` | `5s` | Connect attempt + `/readyz` ping bound. |
|
||
| `BACKEND_SERVICE_NAME` | `scrabble-backend` | OpenTelemetry `service.name`. |
|
||
| `BACKEND_OTEL_TRACES_EXPORTER` | `none` | `none`, `stdout` or `otlp` (gRPC; endpoint from the standard `OTEL_EXPORTER_OTLP_*`). |
|
||
| `BACKEND_OTEL_METRICS_EXPORTER` | `none` | `none`, `stdout` or `otlp`. |
|
||
| `BACKEND_DICT_DIR` | — | **Required.** Directory of committed `.dawg` dictionaries. |
|
||
| `BACKEND_DICT_VERSION` | `v1` | Version label for the flat dictionary dir. Recorded in a `.seed_version` marker on first boot and authoritative after: on a seeded volume a changed value is ignored (it seeds only a fresh volume) — the seed-drift guard (ARCHITECTURE.md §5). |
|
||
| `BACKEND_GAME_TIMEOUT_SWEEP_INTERVAL` | `1m` | How often the turn-timeout sweeper runs. |
|
||
| `BACKEND_GAME_CACHE_TTL` | `24h` | Idle window before a live game is evicted from cache. |
|
||
| `BACKEND_LOBBY_ROBOT_WAIT` | `10s` | Auto-match wait before a robot is substituted for a missing human. |
|
||
| `BACKEND_LOBBY_REAPER_INTERVAL` | `1s` | How often the substitution reaper scans for over-waited players. |
|
||
| `BACKEND_ROBOT_DRIVE_INTERVAL` | `30s` | How often the robot driver scans for due robot turns. |
|
||
| `BACKEND_SMTP_HOST` | — | Confirm-code relay host. **Empty selects the development log mailer** (the code is logged, not sent). |
|
||
| `BACKEND_SMTP_PORT` | `587` | Relay port. `465` selects implicit TLS; any other port uses STARTTLS. No client certificate is needed (the server cert is validated against the system roots). |
|
||
| `BACKEND_SMTP_USERNAME` | — | SMTP AUTH user; empty relays without authentication. |
|
||
| `BACKEND_SMTP_PASSWORD` | — | SMTP AUTH password. |
|
||
| `BACKEND_SMTP_FROM` | `no-reply@localhost` | From address. A deployed contour must use the prod domain (the relay only accepts its verified sender domain). |
|
||
| `BACKEND_PUBLIC_BASE_URL` | — | Canonical public origin (scheme + host) for links in the email. **Required when `BACKEND_SMTP_HOST` is set.** Never derived from a request Host header (anti-injection). |
|
||
| `BACKEND_CONNECTOR_ADDR` | — | the gateway bot-link relay gRPC address for admin-console operator broadcasts. Empty disables broadcasts. |
|
||
| `BACKEND_GUEST_REAP_INTERVAL` | `1h` | How often the abandoned-guest reaper sweeps. |
|
||
| `BACKEND_GUEST_RETENTION` | `720h` | Account age past which a guest with no game seat is deleted. |
|
||
| `BACKEND_HIGHRATE_FLAG_THRESHOLD` | `1000` | Gateway-reported rejected calls within the window past which an account is soft-flagged. |
|
||
| `BACKEND_HIGHRATE_FLAG_WINDOW` | `10m` | The rolling window those rejections accumulate over. |
|
||
|
||
## Run
|
||
|
||
```sh
|
||
docker run -d --name scrabble-pg -e POSTGRES_PASSWORD=dev -p 5432:5432 postgres:17-alpine
|
||
# DAWGs: extract the dictionary release artifact (or point at a local scrabble-solver/dawg):
|
||
mkdir -p /tmp/dawg && curl -fsSL https://gitea.iliadenisov.ru/developer/scrabble-dictionary/releases/download/v1.3.0/scrabble-dawg-v1.3.0.tar.gz | tar xz -C /tmp/dawg
|
||
BACKEND_POSTGRES_DSN='postgres://postgres:dev@localhost:5432/postgres?search_path=backend&sslmode=disable' \
|
||
BACKEND_DICT_DIR=/tmp/dawg \
|
||
GOPRIVATE='gitea.iliadenisov.ru/*' \
|
||
go run ./cmd/backend
|
||
```
|
||
|
||
On boot the backend opens the pool, creates the `backend` schema if needed,
|
||
applies the embedded migrations, loads the dictionaries into the engine registry
|
||
(a hard dependency — a missing dictionary aborts the boot), warms the session
|
||
cache and starts the game turn-timeout sweeper. `GET /healthz` reports liveness;
|
||
`GET /readyz` reports 200 only when the database answers and the session cache is
|
||
warmed.
|
||
|
||
## Migrations & generated code
|
||
|
||
Migrations are plain goose SQL under `internal/postgres/migrations` (sequential
|
||
`NNNNN_name.sql`), embedded and applied at startup. The incremental history was
|
||
squashed into a single `00001_baseline.sql` before the first production deploy
|
||
(there was no production data); new schema changes append as `00002_*` onward.
|
||
After changing the schema,
|
||
regenerate the committed go-jet code (needs Docker):
|
||
|
||
```sh
|
||
go run ./cmd/jetgen # rewrites internal/postgres/jet against a temp container
|
||
```
|
||
|
||
## Engine & dictionaries
|
||
|
||
`internal/engine` consumes `scrabble-solver` in-process as a **published, versioned
|
||
module** (`gitea.iliadenisov.ru/developer/scrabble-solver`, pinned in `go.mod`). Set
|
||
`GOPRIVATE=gitea.iliadenisov.ru/*` so go fetches it directly from this Gitea (skipping
|
||
the public proxy/checksum DB); no sibling checkout or `go.work` replace is needed (for
|
||
local solver co-development you may add a temporary replace — see `go.work`).
|
||
`github.com/iliadenisov/dafsa` (the DAWG loader) is a direct dependency. The dictionaries
|
||
(`en_sowpods.dawg`, `ru_scrabble.dawg`, `ru_erudit.dawg`) ship as a **release artifact**
|
||
from the [`scrabble-dictionary`](https://gitea.iliadenisov.ru/developer/scrabble-dictionary)
|
||
repo (one semver per set); the engine loads them by `(variant, dict_version)` from
|
||
`BACKEND_DICT_DIR`. The backend loads them at startup as a hard dependency
|
||
(a missing dictionary aborts the boot). The flat directory is the seed version,
|
||
labelled `BACKEND_DICT_VERSION`; uploaded versions live in `<version>/`
|
||
subdirectories the admin console writes and a restart re-loads. Because the DAWGs
|
||
carry no embedded version, the first boot records the seed in a `.seed_version`
|
||
marker that is authoritative after: on a seeded volume a changed `BACKEND_DICT_VERSION`
|
||
is ignored (it seeds only a fresh volume) — the seed-drift guard — so a live contour's
|
||
dictionary is changed through the console, never by bumping the build seed
|
||
(ARCHITECTURE.md §5).
|
||
|
||
## Tests
|
||
|
||
```sh
|
||
go test -count=1 ./... # unit tests (no Docker)
|
||
go test -tags=integration -count=1 -p=1 ./... # Postgres-backed (needs Docker)
|
||
```
|
||
|
||
Integration tests are guarded by the `integration` build tag and run against a
|
||
throwaway `postgres:17-alpine` container; they fail loudly when Docker is absent
|
||
rather than skipping. The `internal/engine` tests load the DAWGs from
|
||
`BACKEND_DICT_DIR` (CI sets it to the extracted dictionary release artifact; locally it
|
||
defaults to a `scrabble-solver/dawg` sibling checkout) and fail loudly when that directory
|
||
is absent. `GOPRIVATE=gitea.iliadenisov.ru/*` is needed for go to fetch the pinned solver
|
||
module.
|