Files
Ilia Denisov 04263a17ca R7: per-player transports + drop finished games in the load harness
Each virtual player now builds its own edge.Client (its own h2c connection
carrying both the Subscribe stream and the Execute calls), instead of every
player multiplexing over a single shared http2.Transport. The R2 trip report
traced the ~14% transport_error on game.state at 500 players to that single
shared transport; per-player connections mirror real clients and isolate the
artifact. The assembly burst and the gateway-hammer each get their own client.

playTurn now reports when a game has finished so playerLoop drops it from the
rotation (slices.DeleteFunc); once no active game remains the player idles while
still holding its stream. This stops secondary ops from hammering game_finished
on already-ended games (the other R2 harness finding).
2026-06-10 18:53:07 +02:00
..

loadtest — stress harness

Reusable load harness for the pre-release stress pass. It seeds a large account population with pre-created sessions, drives virtual players through the gateway edge protocol in realistic games, hammers the rate limiter, and prints a trip-report summary. It stays in the repo for repeats.

What it does

  1. Seed (direct Postgres, schema backend): inserts --durable durable accounts (each with a confirmed email identity) + --guest guest accounts and an active sessions row per account, then hands the plaintext bearer tokens to the driver. Token hashes match backend/internal/session (hex(sha256(token))), so the seeded sessions resolve. Every row carries a distinctive display-name marker for cleanup.
  2. Drive (edge protocol over h2c): assembles real 24 player games via the invitation flow (invitation.createinvitation.accept, no robots), then runs each player's turn loop — poll game.state, replay game.history, generate a legal mid-ranked move with the embedded scrabble-solver, and game.submit_play (or pass/exchange). A fraction of turns exercise nudge / chat / check-word / draft / profile-update / stats. Each player also holds a live Subscribe stream. The moderate ramp is 50 → 200 → 500 concurrent players, ~12 min per step.
  3. Hammer: drives games.list from one account far above the per-user rate limit to verify the limiter holds (rate_limited results) and measure its cost.
  4. Report: per-operation latency percentiles, throughput, result-code breakdown, live-event tally and the aggregate error rate.

The driver runs the solver locally because the edge protocol carries no board: the client reconstructs it from decoded history (the same invariant as the UI).

Connection model

The harness reaches Postgres and the gateway directly, so run it as a one-shot container on the contour's docker network (this bypasses the host→gateway hairpin):

# from the repo root
docker build -f loadtest/Dockerfile -t scrabble-loadtest .

docker run --rm --name scrabble-loadtest --network scrabble-internal \
  -e POSTGRES_PASSWORD="$TEST_POSTGRES_PASSWORD" \
  scrabble-loadtest run

Defaults assume the contour service names: postgres:5432 and gateway:8081. The DAWGs are baked into the image (/opt/dawg, pinned to the dictionary release). Run with --name scrabble-loadtest so the harness's own CPU/memory show up as a scrabble-* series in cAdvisor (keeping it separable from the system under test). Capture the resource baseline from the Grafana Scrabble — Resources dashboard (cAdvisor + postgres_exporter) while the run is in progress.

Commands & flags

loadtest run [flags]      seed, drive the ramp + hammer, print the report
loadtest cleanup [flags]  delete everything the harness seeded (matched by the display-name marker)

Key run flags (env in parentheses):

flag default meaning
--gateway (LOADTEST_GATEWAY_URL) http://gateway:8081 gateway base URL
--dsn (LOADTEST_DSN) from POSTGRES_* backend Postgres DSN (schema backend)
--dawg (LOADTEST_DAWG_DIR) /dawg (image: /opt/dawg) committed *.dawg directory
--durable / --guest 10000 / 1000 accounts to seed
--steps 50,200,500 concurrent-player ramp steps
--step-dur 12m hold time per step
--games-per-player 0 (random 35) target concurrent games per player
--tick 800ms per-player op cadence (keeps a player under the per-user limit)
--secondary-prob 0.08 chance per tick of a non-move op
--hammer-workers / --hammer-dur 20 / 15s gateway-hammer (0 workers disables)
--reset / --cleanup false delete harness rows before / after the run

run re-seeds every time (plaintext tokens are never stored), so pass --reset to clear a prior run's rows first. The authoritative hard reset of the contour remains the DB wipe (DROP SCHEMA backend CASCADE + backend restart).

Build & test

go build ./loadtest/...
go vet ./loadtest/...
BACKEND_DICT_DIR=../scrabble-solver/dawg go test -count=1 ./loadtest/...

The DAWG-backed moves test runs only when BACKEND_DICT_DIR is set (as the engine tests use); the pure logic (hashing, board replay, rack build, move selection, report) runs unconditionally.

Caveat

The harness shares the host CPU with the contour, so the early-pass resource baseline is read with the harness's own container series in mind; a cleaner number on separate hardware is future work. The moderate ramp keeps the generator from being the bottleneck.