perf(gateway): pool backend conns; loadtest evaluate hot path
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 18s
CI / ui (pull_request) Has been skipped
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m5s

The loadtest harness never modelled game.evaluate — the debounced per-tile
play preview a real client fires several times per turn, the hottest gameplay
call. Model it (one evaluate per placed tile + reconsideration re-previews +
draft.save, human-paced; --eval / --eval-recon toggle it).

That realistic load surfaced the real bottleneck: the gateway's backend HTTP
client used the default transport (MaxIdleConnsPerHost=2), so every sync call
to the single backend host churned a fresh TCP connection — ~26500 TIME_WAIT
sockets at 500 players (near the ephemeral-port ceiling), burning ~1.75 gateway
cores while the backend sat near-idle. It was the unfixed root of the residual
transport_error the earlier passes chased on the client side.

Widen the keep-alive pool (backendMaxIdleConns=512, ~2x the observed 225-conn
peak). At 500 players the churn collapses to ~0 and peak gateway CPU drops ~7x
(~1.75 -> ~0.26 cores); postgres (~1.65 cores) becomes the busiest service.
This overturns the earlier "gateway is the binding constraint, scale it
horizontally" sizing — that was sizing around this bug, not a real floor.

Consolidate the loadtest trip reports into one loadtest/REPORT.md (drop the
R2/R7 split) and bake the finding into README / PRERELEASE / ARCHITECTURE /
TESTING.
This commit is contained in:
Ilia Denisov
2026-06-21 19:55:57 +02:00
parent dec6fac013
commit e2771826fd
14 changed files with 378 additions and 396 deletions
+27
View File
@@ -63,6 +63,33 @@ func submitPlay(gameID string, tiles []PlayTile) []byte {
return b.FinishedBytes()
}
// evalReq builds an EvalRequest payload (game id plus the tentative newly-placed tiles).
// It mirrors submitPlay's shape — the backend infers the play's orientation the same way —
// so a preview previews exactly what submitting those tiles would score.
func evalReq(gameID string, tiles []PlayTile) []byte {
b := flatbuffers.NewBuilder(256)
gid := b.CreateString(gameID)
offs := make([]flatbuffers.UOffsetT, len(tiles))
for i, t := range tiles {
fb.PlayTileStart(b)
fb.PlayTileAddRow(b, int32(t.Row))
fb.PlayTileAddCol(b, int32(t.Col))
fb.PlayTileAddLetter(b, t.Letter)
fb.PlayTileAddBlank(b, t.Blank)
offs[i] = fb.PlayTileEnd(b)
}
fb.EvalRequestStartTilesVector(b, len(offs))
for i := len(offs) - 1; i >= 0; i-- {
b.PrependUOffsetT(offs[i])
}
tilesVec := b.EndVector(len(offs))
fb.EvalRequestStart(b)
fb.EvalRequestAddGameId(b, gid)
fb.EvalRequestAddTiles(b, tilesVec)
b.Finish(fb.EvalRequestEnd(b))
return b.FinishedBytes()
}
// exchange builds an ExchangeRequest payload swapping the listed rack tiles (alphabet
// indices; 255 a blank).
func exchange(gameID string, tiles []byte) []byte {