Files
scrabble-game/docs/TESTING.md
T
Ilia Denisov 5689f7f6a3
CI / changes (pull_request) Successful in 3s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 15s
CI / ui (pull_request) Successful in 58s
CI / conformance (pull_request) Successful in 9s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m17s
feat: on-device move preview (local eval) with network fallback
Score and validate a tentative move on-device instead of a per-arrangement network
round trip. The dawg reader and the validate/score/direction slice of the
scrabble-solver engine are ported to TypeScript (ui/src/lib/dict), pinned
byte-for-byte to the Go engine by a `conformance` CI job (full-dictionary reader
parity plus a battery of plays across every variant and both cross-word rules,
including the inferred orientation). The server stays authoritative — submit_play
re-validates — so the local result is an advisory accelerator only.

- backend: Registry.DictBytes + an authed GET /api/v1/user/dict/{variant}/{version}
  (immutable) streaming the pinned per-game dawg; caddy routes /dict to the gateway.
- gateway: a session-gated /dict edge route proxying it; fetchDict on the transport.
- client: the dictionary loads on game open (low priority so it never starves the
  game on a slow link; aborted at a 5s cap or when leaving the game), is cached in
  IndexedDB (best-effort, self-healing on a rejected blob) and reused across
  sessions; a warm-up overlay covers a cold load, then the network preview is the
  fallback; a bad-connection breaker stops warming after repeated misses; the move
  preview cancels its in-flight request when the tiles change.
- parity generators backend/cmd/{dictgen,validategen} + gated Vitest suites, run in
  CI against the release dictionaries. A hidden debug readout lists the cached
  dictionaries + breaker state, and its reset clears the cache.
- docs: ARCHITECTURE §5, TESTING, UI_DESIGN, FUNCTIONAL (+ru).
2026-07-01 22:58:40 +02:00

175 lines
12 KiB
Markdown

# Scrabble Game — Testing
How the project is tested and the gate every stage must pass. Read before adding
tests or touching CI.
## Layers
- **Go unit tests** — table-driven where it helps; `testing` + standard library.
Every functional change ships with regression coverage. Run:
`go test -count=1 ./backend/... ./pkg/... ./gateway/...` (the module list grows
with the workspace).
- **Integration** — Postgres-backed tests behind the `integration`
build tag spin a throwaway `postgres:17-alpine` via `testcontainers-go`. They
live in `backend/internal/inttest` and run with
`go test -tags=integration -count=1 -p=1 ./backend/...` (needs Docker), guarded
by a separate CI workflow (`integration.yaml`; Ryuk disabled, serial). Slow.
- **UI** — Vitest (unit) + Playwright
(e2e), mirroring the chosen plain-Svelte + Vite toolchain. Vitest covers
the FlatBuffers codecs (friend list, invitation, stats), the win-rate
derivation and the GCG share/copy/download choice, plus Playwright specs against the
mock for the friends screen (code issue/redeem, accept a request), the lobby
invitations section, the stats screen, profile editing, and the GCG export's
finished-only visibility.
- **Local-eval conformance** — the client's on-device move preview (the ported
dawg reader + validator, `ui/src/lib/dict`) is checked byte-for-byte against the
authoritative Go engine. `backend/cmd/dictgen` and `backend/cmd/validategen` emit
golden vectors from the release dictionaries — every stored word plus a battery of
plays across both cross-word rules and all variants, each with the engine's
legality, score, words and inferred direction. The gated Vitest suites
(`ui/src/lib/dict/*.parity.test.ts`, skipped without their `DICT_*` env) replay
them and must agree exactly; the `conformance` CI job runs the whole loop, so a
drift in the port fails the merge.
- **Engine** — correctness of scoring and move generation is owned
by `scrabble-solver`'s own GCG-backed tests. `backend/internal/engine` adds, on
top of the embedded solver: per-variant smoke tests (load all three committed
DAWGs and validate a known word, including Эрудит), bag draw/return determinism
and exchange accounting, the `Game` end-conditions (empty bag with an empty
rack, and six scoreless turns) with end-game rack scoring, and
**dictionary-independent history replay** (`ReplayBoard` reproduces a full
greedy game's final board from decoded records alone), and the **resignation
win/loss rule** (the resigner keeps their score yet loses). The engine tests
read the DAWGs from `BACKEND_DICT_DIR` (or the sibling `scrabble-solver`
checkout) and fail loudly when it is absent.
- **Game domain** — `backend/internal/game` adds pure unit tests
(the GCG writer, the away-window / effective-deadline boundaries, the hint
budget, the live-game cache and per-game lock, payload round-trips) plus
Postgres-backed integration tests in `inttest` (full lifecycle to a natural
end, **journal-replay equivalence**, the turn-timeout sweep with away-window
grace, resign win/loss and statistics, the hint allowance-then-wallet policy,
word-check and complaint capture, and per-game-lock serialisation). It also covers
the engine's **multi-player drop-out** cases (continue after one resign,
last-survivor win, the tile-disposition bag effect) and a domain integration test
for a 3-player **timeout that continues**, and the engine's `Candidates`
ranked/decoded test.
- **Social & lobby** — `backend/internal/social` unit-tests the chat
**content filter** (links/emails/phones plus obfuscated forms) and
`backend/internal/lobby` unit-tests the in-memory **matchmaker** (FIFO pairing,
cancel, per-variant pools, plus the **robot substitution** reaper and
`Poll` delivery) with fake game-creator and robot-provider seams. Postgres-backed
`inttest` covers the friend request/accept lifecycle with the block/toggle guards,
the per-user block (and its severing of friendships), chat post/list with the IP,
content and block-visibility rules, the nudge turn/rate-limit rules, the
invitation flow (all-accept starts the game, decline cancels, lazy expiry,
inviter-only cancel), and the email confirm-code flow (request/confirm, taken
email, expiry and attempt-cap) with a fixture mailer. It also covers the
**befriend-an-opponent** gate (a request needs a shared game), the **permanent
decline** and 30-day re-send rule, the **one-time friend code** (issue/redeem,
self/single-use, decline-bypass), `ListInvitations`, the zero-value `GetStats`, and
the GCG **finished-only** gate.
- **Robot** — `backend/internal/robot` unit-tests the pure strategy:
the ≈ 40% play-to-win split over many seeds, the right-skewed move-delay
(bounds, ~10-min median, determinism), the margin selection (win/lose, in-band
and out-of-band fallbacks, no-play exchange/pass), the sleep window with drift
and the midnight wrap, and mix restart-stability. Postgres-backed `inttest`
drives a robot through a full auto-match to a natural end (asserting a robot
statistics row), the matchmaker substitution end-to-end (enqueue → reap →
`[human, robot]`, discoverable via `Poll`), and a proactive 12-hour nudge.
- **Gateway & contracts** — `backend/internal/notify` unit-tests the
hub fan-out (delivery, overflow drop, unsubscribe) and the FlatBuffers event
constructors (payload round-trip). `gateway/...` unit-tests are hermetic (no
real network — an `httptest` fake backend and fixtures): the Telegram initData
HMAC validator (genuine, tampered, wrong-token, stale), the session cache
(hit/miss/fallback, TTL re-resolve, invalidate), the rate limiter (burst,
per-key isolation, per-window), the push hub (per-user routing, overflow,
unsubscribe), the transcode round-trips (FlatBuffers↔JSON, X-User-ID
forwarding, nested GameView, domain-code surfacing), the admin Basic-Auth
reverse proxy (401 / forward), and a full Connect `Execute` path end to end
(guest auth, unauthenticated rejection, unknown message type). The
edge-hardening cases: an oversized `Execute` payload is refused
(`resource_exhausted`, the `GATEWAY_MAX_BODY_BYTES` cap), a limiter rejection
lands in `gateway_rate_limited_total{class}` and the rejection tracker
(drain/aggregate unit tests), the report POST reaches
`/api/v1/internal/ratelimit/report` with the agreed JSON shape, the `/_gm`
mount is 429-guarded by the per-IP admin class, and the gateway's `/`
308-redirects to `/app/` (the landing left the embed). The backend covers
the **guest** lifecycle (a guest plays an auto-match to a natural end yet accrues
no statistics) and the **email-as-login** flow (request/verify, returning user)
in `inttest`. Gateway transcode round-trips cover the social/account
operations (friends list, friend code issue/redeem, invitation create, stats, GCG,
the profile-update away round-trip) and a `notify`-event constructor round-trip.
- **Admin & dictionary ops** — `backend/internal/adminconsole` unit-tests
the template renderer over every page plus the embedded asset; `backend/internal/engine`
adds the **dictionary** cases (`LoadAvailable` loads only the present variants,
`OpenWithVersions` scans version subdirectories and skips the staging area, `DiffWords`
decodes the words added/removed between two versions); `backend/internal/dictadmin`
unit-tests the release-archive validation and hardened extraction (path-traversal,
symlink, oversize and entry-count rejection, immutable versions); `backend/internal/server` unit-tests the console's
**same-origin** CSRF guard; the gateway adds the **verbatim `/_gm` Basic-Auth proxy**
(401 / forward, path preserved) and the h2c **console mount** (routed when configured,
404 when not). Postgres-backed `inttest` drives the **complaint resolution →
dictionary-change pipeline** (file → resolve with a disposition → pending change → mark
applied), the **dictionary update flow** (upload a release archive over HTTP → preview →
install → the new version becomes active and resident, persists across a restart, pins new
games while in-progress games keep theirs, and is immutable), the admin **list/count** read
queries, and the **/_gm console over HTTP** (pages render; a resolve POST needs a
same-origin header). `ratewatch` has
unit tests (window accumulation, the auto-flag threshold + expiry, the bounded
episode map), the account-store **high-rate flag round-trip** (set-once / clear /
re-flag) and a console flow in `inttest`: a gateway report auto-flags the account,
the **Throttled** page shows the episode and the flagged queue, the user card
carries the marker and the CSRF-guarded **Clear** reverses it.
- **Observability & performance** — `pkg/telemetry` unit-tests the exporter
selection (`none`/`stdout`/`otlp` build providers; OTLP constructs with no collector;
the nil-runtime fallback). The domain metrics are exercised through a manual
`sdkmetric` reader: `backend/internal/game` and `…/social` assert the counters and
histograms record with the right `variant`/`kind` attributes, and
`gateway/internal/connectsrv` asserts `edge_request_duration` by `message_type`/
`result`. Config tests cover the new telemetry env vars (backend/gateway/connector —
`otlp` now accepted, an unsupported exporter rejected) and the guest-reaper knobs.
Postgres-backed `inttest` drives the **guest reaper** end to end (an abandoned guest is
reaped; a too-young guest, a seated guest and a durable account are kept).
- **Load test & resource baseline** — a reusable `loadtest/` module
(`scrabble/loadtest`) is the stress/load harness. It **seeds** a large account
population with pre-created sessions directly in Postgres (token hashes matching
`backend/internal/session`), **drives** virtual players through the edge protocol —
real games assembled via invitations, **mid-ranked** legal moves generated locally by
the embedded `scrabble-solver` (the edge carries no board, so the client replays
history) — plus a fraction of nudge/chat/check-word/draft/profile/stats ops, and a
**gateway-hammer** that verifies the rate limiter. Its own Go unit tests cover the pure
pieces (token hashing, board replay vs. `board.Parse`, rack reconstruction, mid-rank
selection, the report); the DAWG-backed move test runs under `BACKEND_DICT_DIR` (as the
engine tests do). It is **not** part of the per-PR suite's behavioural assertions: it
runs ad hoc as a one-shot container against the contour, producing a trip report (bugs
+ a per-container resource profile) read off the **otelcol `docker_stats` +
postgres_exporter** Grafana dashboard on the contour. The findings — including the
`game.evaluate` hot-path model and the gateway→backend connection-pool fix — are written
up in [`REPORT.md`](../loadtest/REPORT.md). See [`../loadtest/README.md`](../loadtest/README.md).
- **User feedback** — `internal/feedback` unit tests cover the attachment allow-list /
content-type and the channel normaliser; the UI covers `detectChannel`, the attachment gate and
the feedback wire round-trip (`channel` / `feedback` / `codec` tests) plus a Playwright e2e
(submit → confirm + resend blocked; an operator reply raises the badge and shows on the screen).
The gateway `connectsrv` test asserts the **guest gate** (a guest's `feedback.submit`
`guest_forbidden` with no backend call; a non-gated authenticated op still passes). Postgres-backed
`inttest` drives the **feedback lifecycle** end to end: the guest / ban / pending gates, the reply
read + one-week visibility window, the account-role grant/revoke, and the admin queue filters with
delete / delete-all. The admin-console render test renders the feedback list + detail pages.
## Principles
- A green run must not depend on cached state: use `-count=1` in CI.
- Tests that need infrastructure fail loudly (`t.Fatal`) when it is unavailable
rather than silently skipping coverage.
- No network or real platform calls in unit tests; validate platform
credentials behind an interface seam and test with fixtures.
## CI gate
Every change is exercised on `gitea.iliadenisov.ru` before it is merged:
1. Commit the change on its `feature/*` branch.
2. Push to `origin`.
3. Watch the run to completion — never hand-roll a poll loop:
`python3 ~/.claude/bin/gitea-ci-watch.py` (launch in the background).
4. Only after every workflow that fired is green may the change be merged.