Ilia Denisov
fe5a3d6d3b
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 11s
CI / integration (pull_request) Successful in 22s
CI / ui (pull_request) Successful in 1m13s
CI / conformance (pull_request) Successful in 11s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 2m7s
A fresh email login that hit the per-IP email rate limit stranded the user in an unrecoverable "offline" state that survived a full PWA restart, even though the network was fine. Root cause: the gateway email class (auth.email.request + auth.email.login, keyed per IP, 5/10min burst 2) trips on the third event, which in the natural "request code -> wrong code -> correct code" sequence is the correct-code login. The gateway returns ResourceExhausted; the client mapped it to 'rate_limited', which retry.ts classified as retryable + a connection code, so exec() called reportOffline(). That pushes the net-state machine into connecting, whose recovery probe is an authenticated profile.get -- but the login screen has no session, so the probe can never succeed and the machine latches offline. The transport kill switch (assertOnline) then refuses the very login that would fix it, and because the IP's email bucket refills only 1 token / 120s, each fresh attempt after a restart is rate-limited again -> offline again. Fix (client, narrow -- the trigger): - A rate-limit is no longer treated as connectivity. retry.ts no longer marks 'rate_limited' retryable or a connection code, so exec() never reports it offline; it surfaces as the existing error.rate_limited "slow down" message. Not auto-retrying it also stops the ~20s button freeze and avoids feeding the gateway's ban tripwire with 6 extra rejections per attempt. Fix (server): - Raise the email-code burst 2 -> 4 so the honest request + a mistyped code + the correct one is not throttled mid-login. Defence-in-depth over the backend's own per-code guards (5-attempt cap + 15-min TTL + send throttle). Tests: retry classification units updated to the new semantics; a gateway regression guard asserts the honest three-event email flow passes under the default policy. gateway/README.md rate-limit note updated. The deeper gap (the net-state recovery probe is session-gated, so any real transport failure on the session-less login/confirm screens still cannot self-heal) is left as a known residual, deferred by the owner.
175 lines
11 KiB
Markdown
175 lines
11 KiB
Markdown
# gateway
|
|
|
|
The Scrabble platform's only public ingress (module `scrabble/gateway`). It
|
|
terminates the client's **Connect-RPC + FlatBuffers** traffic over HTTP/2
|
|
cleartext (`h2c`), authenticates the originating credential, mints/resolves a
|
|
thin opaque session, rate-limits, injects `X-User-ID` when forwarding to the
|
|
backend over REST/JSON, and bridges the backend's gRPC push stream to each
|
|
client's in-app live channel. It **embeds the static UI build** (`go:embed`, baked
|
|
in by the gateway image's node stage) and serves a **landing page** at `/` and the game
|
|
**SPA** at `/app/` (web), `/telegram/` (the Telegram Mini App) and `/vk/` (the VK Mini App) — the single-origin model.
|
|
The web SPA is an installable **PWA**: it serves `manifest.webmanifest` (unhashed from `ui/public/`,
|
|
with the `.webmanifest` MIME type registered in-process — the distroless image has no
|
|
`/etc/mime.types`) and the app-shell `sw.js` (built from `ui/src/sw.ts` by vite-plugin-pwa, which
|
|
precaches the shell + assets so an installed PWA cold-launches offline).
|
|
Hash-named `/assets/*` are served `immutable`; the HTML shells are `no-cache`. It can also serve the
|
|
backend's admin console at `/_gm` behind HTTP Basic-Auth for a local non-caddy run;
|
|
in the deployed contour the front caddy owns `/_gm` (see
|
|
[`../deploy`](../deploy)). See
|
|
[`../docs/ARCHITECTURE.md`](../docs/ARCHITECTURE.md) §2, §3, §10, §12, §13.
|
|
|
|
## Package layout
|
|
|
|
```
|
|
cmd/gateway/ # main: config -> backend client -> session cache ->
|
|
# push hub -> Connect h2c server (+ admin) -> serve
|
|
proto/edge/v1/ # Connect envelope contract (committed generated Go)
|
|
internal/config/ # GATEWAY_* env config
|
|
internal/backendclient/ # typed REST client (+ X-User-ID) and push gRPC client
|
|
internal/session/ # in-memory session cache (LRU/TTL, backend fallback)
|
|
internal/ratelimit/ # token-bucket limiter (golang.org/x/time/rate) + the rejection tracker + the temporary IP banlist
|
|
internal/connector/ # gRPC client to the Telegram connector (initData validate, out-of-app push) + routing
|
|
internal/push/ # live-event fan-out hub (per-user client streams)
|
|
internal/transcode/ # FlatBuffers<->REST bridge + message_type registry
|
|
internal/connectsrv/ # the Connect Gateway service over h2c (+ the in-memory active_users gauge)
|
|
internal/admin/ # Basic-Auth reverse proxy mounting the backend admin console at /_gm (verbatim)
|
|
internal/webui/ # embedded UI build (go:embed dist): landing at /, SPA at /app/ + /telegram/ + /vk/
|
|
```
|
|
|
|
The FlatBuffers payloads and the backend push proto are the shared wire
|
|
contracts in [`../pkg`](../pkg).
|
|
|
|
## Transport contract
|
|
|
|
A single `Gateway` Connect service: `Execute(message_type, payload, request_id)`
|
|
for unary operations and `Subscribe` for the live stream. The `payload` bytes are
|
|
FlatBuffers tables (`scrabble/pkg/fbs`); the gateway transcodes them to and from
|
|
the backend's JSON. The session token rides in `Authorization: Bearer`; `auth.*`
|
|
operations are unauthenticated and return the minted token. A unary domain
|
|
outcome rides back in `ExecuteResponse.result_code` (HTTP 200); only edge
|
|
failures become Connect error codes.
|
|
|
|
`auth.telegram` validates the Mini App `initData` by calling the **Telegram validator**
|
|
(`GATEWAY_VALIDATOR_ADDR`), which holds the bot token (HMAC); out-of-app push for
|
|
recipients with no live in-app stream goes to the remote **bot** over the reverse **mTLS
|
|
bot-link** (`GATEWAY_BOTLINK_ADDR`, fire-and-forget), and the backend admin broadcasts
|
|
arrive on the gateway's plaintext **relay** (`GATEWAY_BOTLINK_RELAY_ADDR`) which forwards
|
|
them down the same link and awaits the bot's ack (ARCHITECTURE.md §10/§12). When
|
|
`GATEWAY_VALIDATOR_ADDR` is unset Telegram auth is disabled; when `GATEWAY_BOTLINK_ADDR`
|
|
is unset the bot channel (out-of-app push + admin relay) is disabled.
|
|
|
|
`auth.vk` validates a VK Mini App launch **in-process** (`internal/vkauth`): it verifies the
|
|
signed `vk_*` launch parameters against the VK app's protected key (`GATEWAY_VK_APP_SECRET`) —
|
|
HMAC-SHA256 over the sorted params, base64url — a pure offline check with no side-service or VK API
|
|
round-trip, then forwards the trusted `vk_user_id` (and the client-read display name, which VK omits
|
|
from the signed params) to the backend. When `GATEWAY_VK_APP_SECRET` is unset VK auth is disabled
|
|
(`auth.vk` is unregistered).
|
|
|
|
`link.vk.confirm`/`merge` link a VK identity from a **browser** (not a Mini App) via **VK ID web
|
|
login** (`internal/vkid`): the SPA runs the VK ID raw OAuth 2.1 flow (PKCE, no `@vkid/sdk`) and
|
|
the gateway completes the **confidential code exchange** at `id.vk.com` under a SEPARATE VK "Web"
|
|
app's protected key — the only op here that calls VK (the Mini App path above is offline). All
|
|
three `GATEWAY_VK_ID_*` unset leaves the ops unregistered.
|
|
|
|
The message-type catalog: `auth.telegram`, `auth.vk`, `auth.guest`,
|
|
`auth.email.request`, `auth.email.login`, `profile.get`, `game.submit_play`,
|
|
`game.state`, `lobby.enqueue`, `lobby.poll`, `chat.post`, `chat.read` and the play-loop ops;
|
|
live events
|
|
`your_turn`, `opponent_moved`, `chat_message`, `nudge`, `match_found` (the game events —
|
|
and `game_over`/`notify` — carry the state delta the client applies without a `game.state`
|
|
refetch). The social/account/history ops —
|
|
`friends.*` (list/incoming/request/respond/cancel/unfriend/code.issue/code.redeem),
|
|
`blocks.*`, `invitation.*` (list/create/accept/decline/cancel), `profile.update`,
|
|
`stats.get`, `game.gcg`, and the `notify` live event — go through the identical
|
|
transcode pattern (`transcode_social.go`). Account linking & merge
|
|
— `link.email.request/confirm/merge`, `link.telegram.confirm/merge` and
|
|
`link.vk.confirm/merge` (`transcode_link.go`); the telegram ops validate the **Login
|
|
Widget** payload via the validator (`ValidateLoginWidget`), the vk ops complete the VK ID
|
|
web code exchange via `internal/vkid`, and both forward the trusted `external_id`. These
|
|
**superseded** the former `email.bind.*` ops, which were removed.
|
|
|
|
## Configuration
|
|
|
|
| Variable | Default | Notes |
|
|
| --- | --- | --- |
|
|
| `GATEWAY_HTTP_ADDR` | `:8081` | public Connect/h2c listener (also serves the admin console at `/_gm`) |
|
|
| `GATEWAY_LOG_LEVEL` | `info` | zap level |
|
|
| `GATEWAY_BACKEND_HTTP_URL` | `http://localhost:8080` | backend REST base URL |
|
|
| `GATEWAY_BACKEND_GRPC_ADDR` | `localhost:9090` | backend push gRPC address |
|
|
| `GATEWAY_BACKEND_TIMEOUT` | `5s` | per backend REST call |
|
|
| `GATEWAY_ADMIN_USER` / `GATEWAY_ADMIN_PASSWORD` | unset | enable + guard the admin console at `/_gm` |
|
|
| `GATEWAY_VALIDATOR_ADDR` | unset | Telegram validator gRPC address (enables initData / Login Widget validation) |
|
|
| `GATEWAY_VK_APP_SECRET` | unset | VK app protected key; enables in-process VK Mini App launch-signature verification (`auth.vk`) |
|
|
| `GATEWAY_VK_ID_APP_ID` / `GATEWAY_VK_ID_CLIENT_SECRET` / `GATEWAY_VK_ID_REDIRECT_URL` | unset | VK ID "Web" app credentials for VK web-login linking (`link.vk.*`): the server-side confidential OAuth 2.1 code exchange. A separate VK app from `GATEWAY_VK_APP_SECRET`; all three required to enable the ops |
|
|
| `GATEWAY_BOTLINK_ADDR` | unset | reverse mTLS bot-link listener the remote bot dials (enables out-of-app push + admin relay) |
|
|
| `GATEWAY_BOTLINK_RELAY_ADDR` | unset | plaintext internal listener serving the backend admin `SendToUser`/`SendToGameChannel` relay |
|
|
| `GATEWAY_BOTLINK_TLS_CERT` / `_KEY` / `_CA` | unset | gateway server cert, its key, and the CA that signs accepted bot client certs (required when `GATEWAY_BOTLINK_ADDR` is set) |
|
|
| `GATEWAY_BOTLINK_SEND_TIMEOUT` | `5s` | admin relay wait for the bot ack before reporting not-delivered |
|
|
| `GATEWAY_SESSION_TTL` | `10m` | cached session lifetime |
|
|
| `GATEWAY_SESSION_CACHE_MAX` | `50000` | cached session cap |
|
|
| `GATEWAY_PUSH_HEARTBEAT_INTERVAL` | `10s` | live-stream keep-alive (an immediate heartbeat also fires on open, under the ~15s edge idle timeout) |
|
|
| `GATEWAY_MAX_BODY_BYTES` | `1048576` | caps one request body and one Connect message read; an oversized Execute is refused with `resource_exhausted` |
|
|
| `GATEWAY_ABUSE_BAN_ENABLED` | `false` | enable the temporary IP ban (prod-only — keys by real client IP, off in the shared-NAT test contour) |
|
|
| `GATEWAY_ABUSE_BAN_THRESHOLD` | `100` | rate-limiter rejections within the window that ban an IP |
|
|
| `GATEWAY_ABUSE_BAN_WINDOW` | `2m` | rolling window the rejection strikes accumulate over |
|
|
| `GATEWAY_ABUSE_BAN_DURATION` | `15m` | length of a rejection-earned ban (tripwire 1h, honeytoken 24h are fixed) |
|
|
| `GATEWAY_HONEYTOKEN` | unset | planted bearer value; presenting it bans the caller and raises an alarm |
|
|
| `GATEWAY_SERVICE_NAME` | `scrabble-gateway` | OpenTelemetry `service.name` |
|
|
| `GATEWAY_OTEL_TRACES_EXPORTER` | `none` | `none`, `stdout` or `otlp` (gRPC; endpoint from `OTEL_EXPORTER_OTLP_*`) |
|
|
| `GATEWAY_OTEL_METRICS_EXPORTER` | `none` | `none`, `stdout` or `otlp` |
|
|
|
|
Rate-limit defaults (built-in): public 30/min·IP (burst 10), authenticated
|
|
300/min·user (burst 80, sized for multi-device play), admin
|
|
60/min·IP (burst 20, guarding the `/_gm` mount ahead of its Basic-Auth),
|
|
email-code 5/10 min·IP (burst 4, covering request + a mistyped code + the
|
|
correct one; defence-in-depth over the backend's per-code 5-attempt/15-min cap).
|
|
|
|
Every rejection increments `gateway_rate_limited_total{class}`
|
|
(`user`/`public`/`email`/`admin`) and logs one Debug line; a reporter drains the
|
|
per-key rejection tracker every 30 s, emits a Warn summary per throttled key and
|
|
posts the report to the backend (`/api/v1/internal/ratelimit/report`), feeding
|
|
the admin console's throttled view and the high-rate auto-flag.
|
|
|
|
Temporary IP ban (prod-only, `GATEWAY_ABUSE_BAN_ENABLED`): a fail2ban-style block
|
|
keyed by client IP, fed by sustained rate-limiter rejections (the IP-keyed classes),
|
|
a **honeypot** decoy-path hit (the contour caddy tags decoy paths with
|
|
`X-Scrabble-Honeypot`), and a **honeytoken** (`GATEWAY_HONEYTOKEN`). A banned IP is
|
|
refused with 429 by the `abuseGuard` edge middleware before any work — covering the
|
|
Connect edge, the live stream and the static SPA/landing. Each ban increments
|
|
`gateway_abuse_banned_total{reason}` (`rejections`/`tripwire`/`honeytoken`). The ban
|
|
is in-memory (resets on restart); it is **off by default** because it keys by the real
|
|
client IP, which the shared-NAT test contour does not expose (detection still logs
|
|
there, only the ban action is gated). The gateway syncs its active set to the backend
|
|
every 30 s (`/api/v1/internal/bans/sync`) for the console's **Active IP bans** panel
|
|
and applies the operator unbans the response returns.
|
|
|
|
## Run
|
|
|
|
```sh
|
|
GATEWAY_BACKEND_HTTP_URL=http://localhost:8080 \
|
|
GATEWAY_BACKEND_GRPC_ADDR=localhost:9090 \
|
|
go run ./gateway/cmd/gateway # Connect edge on :8081
|
|
```
|
|
|
|
## Generated code
|
|
|
|
The Connect envelope Go is committed under `proto/edge/v1`. Regenerate after
|
|
editing the `.proto` (dev-time, like `backend/cmd/jetgen`):
|
|
|
|
```sh
|
|
make -C gateway tools # go install protoc-gen-go + protoc-gen-connect-go
|
|
make -C gateway gen # buf generate (local plugins)
|
|
```
|
|
|
|
The FlatBuffers payloads are generated in [`../pkg`](../pkg) (`make -C pkg fbs`).
|
|
|
|
## Tests
|
|
|
|
```sh
|
|
go test -count=1 ./gateway/...
|
|
```
|
|
|
|
All gateway tests are hermetic: no real network, a fake backend (`httptest`) and
|
|
credential fixtures. There is no integration (Docker) suite — the gateway holds
|
|
no database.
|