# Galaxy Architecture Galaxy is a turn-based strategy platform. This document is the source of truth for the platform architecture and supersedes `ARCHITECTURE_deprecated.md`. The previous design factored the platform into nine independently deployed services. This design consolidates all business logic into a single `backend` service alongside the existing `gateway` and `game` components. ## 1. Overview The platform is composed of three executable units: - **`gateway`** — single public ingress. Owns transport security, request authentication via Ed25519-signed envelopes, anti-replay, response signing, and routing of authenticated traffic to `backend`. Stays as a separate process and is the only component reachable from the public internet. - **`backend`** — single internal service that owns every domain concern of the platform: identity, sessions, lobby, game runtime, mail, push and email notification delivery, geo signals, and administration. Talks to Postgres, the Docker daemon, an SMTP relay, and the GeoLite2 country database. The only consumer of `backend` over the network is `gateway`. - **`game`** — turn-engine container. One container per active game, managed exclusively by `backend`. The contract is the OpenAPI document shipped with the engine module; behaviour is unchanged by this architecture. ```mermaid flowchart LR Client((Client)) -- TLS + Ed25519 envelopes --> Gateway Gateway -- REST/JSON, X-User-ID --> Backend Backend -- gRPC stream (push) --> Gateway Backend -- REST/JSON --> Engine[(Game Engine\ncontainer)] Backend -- pgx --> Postgres[(Postgres)] Backend -- Docker API --> Docker[(Docker daemon)] Backend -- SMTP --> Mail[(SMTP relay)] Backend -- GeoLite2 lookup --> GeoIP[(GeoLite2 DB)] Gateway -- anti-replay reservations --> Redis[(Redis)] ``` The MVP runs `gateway` and `backend` as single-instance processes inside a trusted network. Horizontal scaling, distributed coordination, and mTLS-secured east-west traffic are explicit future work and are called out in `Deployment topology`. ## 2. Component Boundaries ### `backend` - Owns every persistent record of platform state in a Postgres schema named `backend`. No other process writes that schema. - Owns every Docker call to `galaxy-game-{game_id}` containers. - Owns the SMTP relationship and the durable email outbox. - Owns the in-memory caches that serve hot reads. - Exposes one HTTP listener and one gRPC listener. No public ingress. ### `gateway` - Public ingress. Performs TLS termination, request signature verification, freshness window enforcement, anti-replay reservations, and rate limiting before any request is forwarded to `backend`. - Forwards authenticated requests to `backend` over HTTP/REST with the resolved `user_id` carried as the `X-User-ID` header. Forwards unauthenticated public traffic verbatim. - Subscribes to `backend` over a long-lived gRPC server stream to receive client push events and session-invalidation notices, signs them, and delivers them to active client subscriptions. - Stops everything that can be stopped at the edge. Any check that does not require backend state — bad signature, stale timestamp, replayed request_id, malformed envelope, blocked-session shortcut — is enforced by `gateway` so that backend is not loaded with invalid traffic. ### `game` - A single game-engine instance per running game, packaged as a Docker container. Stateful only on its host bind-mounted state directory. - Reachable inside the trusted network at `http://galaxy-game-{game_id}:8080`. - Receives all administrative and player-action calls from `backend` only. ## 3. Backend API Surfaces `backend` exposes one HTTP listener with four route groups distinguished by middleware. The full contract lives in `backend/openapi.yaml`. | Prefix | Authentication | Audience | | --------------------- | ------------------------------------------------ | --------------------------------------- | | `/api/v1/public/*` | none | unauthenticated registration | | `/api/v1/user/*` | `X-User-ID` injected by `gateway` | authenticated end users | | `/api/v1/internal/*` | none (network-trusted) | gateway-only server-to-server endpoints | | `/api/v1/admin/*` | HTTP Basic Auth against `admin_accounts` | platform administrators | | `/healthz`, `/readyz` | none | infrastructure probes | `backend` derives user identity exclusively from the `X-User-ID` header on the user surface. Request bodies are never trusted to convey identity. The admin surface is on the same listener as the user surface; isolation between admin and the public is provided by Basic Auth and by the trust boundary described in [§15](#15-transport-security-model-gateway-boundary). The internal surface is part of that same trust boundary: it is network-locked rather than auth-locked, and only `gateway` is expected to call it. The internal surface is read-only with respect to device sessions — it carries the per-request lookup gateway needs to verify a signed envelope, and nothing else. Revocations are user-driven (through the user surface) or admin-driven (through in-process calls inside backend); see [`FUNCTIONAL.md` §1.5](FUNCTIONAL.md#15-revocation). JSON bodies use `snake_case` field names everywhere on the wire. Backend, gateway, and the shared `pkg/model` schemas are aligned on this convention; any future migration to `camelCase` must happen at the `pkg/model` boundary and propagate uniformly. Every error response follows the envelope `{"error": {"code": "", "message": ""}}`. The closed set of `code` values is enumerated in `components/schemas/ErrorBody` of `backend/openapi.yaml`. `409 Conflict` is the standard status when a request collides with existing state (duplicate admin username, duplicate `(template_id, idempotency_key)`, resend on a `sent` mail delivery, lobby state-machine collisions). ## 4. Backend Domain Modules Each module is a Go package under `backend/internal/`. Modules are wired by direct struct references; interfaces are introduced only where a test seam or an external system boundary justifies them. A few cross-module invariants survive consolidation and are surfaced here because they cross domain boundaries: - **`accounts.user_name`** is the immutable login handle assigned at first sign-in. Backend synthesises it as `Player-XXXXXXXX` (eight `crypto/rand`-backed alphanumerics, retried on UNIQUE collisions), so a fresh email always lands a unique account without a client-supplied name. The column is never overwritten on subsequent sign-ins. - **`accounts.permanent_block`** is the canonical permanent-block flag. When set, both `auth.SendEmailCode` and `auth.ConfirmEmailCode` reject with `400 invalid_request`. The send-time check stops fresh challenges for already-blocked addresses; the confirm-time check (re-run after the verification code matches) catches admin blocks applied in the window between send and confirm. Every other branch on send — including a `blocked_emails` row, a throttled email, a fresh email — returns the opaque `{challenge_id}` shape so the endpoint cannot be used to enumerate accounts. - **Public lobby games are admin-created** through `POST /api/v1/admin/games`. The user-facing `POST /api/v1/user/lobby/games` always emits `private` games owned by `X-User-ID`. Public games carry `owner_user_id IS NULL`; the partial index on `(owner_user_id) WHERE visibility = 'private'` keeps the private-owner lookup efficient. - **Authenticated lobby commands** flow through the gateway envelope by `message_type`. The catalog is `lobby.my.games.list`, `lobby.public.games.list`, `lobby.my.applications.list`, `lobby.my.invites.list`, `lobby.game.create`, `lobby.game.open-enrollment`, `lobby.application.submit`, `lobby.invite.redeem`, and `lobby.invite.decline`. Each lands on a REST handler under `/api/v1/user/lobby/*`; the gateway forces visibility to `private` on `lobby.game.create` before forwarding, matching the user-surface invariant above. | Package | Responsibility | | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `backend/internal/config` | Environment-variable loader and validator. | | `backend/internal/server` | gin engine, listeners, route groups, shared middleware (request id, panic recovery, metrics, tracing). | | `backend/internal/auth` | Email-code challenges, device sessions, Ed25519 client public keys, send/confirm, user-driven revoke (single + revoke-all), admin-driven revoke (sanctions, soft-delete, in-process), durable revocation audit in `session_revocations`, internal session lookup endpoint for gateway. | | `backend/internal/user` | User accounts, settings (`preferred_language`, `time_zone`, `declared_country`), entitlements, sanctions, limits, soft delete with in-process cascade. | | `backend/internal/lobby` | Games, applications, invites, memberships, enrollment state machine, turn schedule, Race Name Directory. | | `backend/internal/runtime` | Engine version registry, container lifecycle, turn scheduler, `(user_id ↔ race_name ↔ engine_player_uuid)` mapping per game, runtime snapshot publication into `lobby`. | | `backend/internal/mail` | Postgres outbox, SMTP delivery worker, retry/backoff, dead letters, admin resend. | | `backend/internal/notification` | Notification intent normalization, idempotency, per-route fan-out into push (gRPC) and email (outbox). | | `backend/internal/geo` | Per-session country observation, `(user_id, country)` counter, `declared_country` initialisation at registration. | | `backend/internal/admin` | `admin_accounts` table, env-driven bootstrap, Basic Auth verifier, admin-side operations across other modules. | | `backend/internal/push` | gRPC server hosting the `SubscribePush` stream consumed by gateway. | | `backend/internal/engineclient` | Thin REST client to running game engines. Reuses DTOs from `pkg/model/{order,report,rest}`. | | `backend/internal/dockerclient` | Wrapper around `github.com/docker/docker` for container start, stop, restart, patch, inspect, reconcile. | | `backend/internal/postgres` | pgx pool, embedded migrations, jet-generated query packages. | | `backend/internal/telemetry` | OpenTelemetry runtime, zap logger factory, trace-field helpers. | ## 5. Persistence - A single Postgres database, schema `backend`. `backend` is the only writer. Every `backend` table lives in this schema. - Migrations are kept in `backend/internal/postgres/migrations/`, embedded into the binary, and applied via `pressly/goose/v3` during startup before any listener opens. The DSN must include `?search_path=backend` so unqualified reads and writes resolve to the service-owned schema. - Queries are written through `go-jet/jet/v2`. Generated code lives in `backend/internal/postgres/jet/` and is regenerated by `make jet`. - Every domain identifier is a `uuid` primary key (`device_session_id`, `user_id`, `game_id`, `application_id`, `invite_id`, `membership_id`, `delivery_id`, `notification_id`, …). Identifiers that are not Postgres-side identities (`email`, `user_name`, `canonical`, `template_id`, `idempotency_key`, `race_name`) remain `text`. - Foreign keys are intra-domain only: `accounts → entitlement_*` / `sanction_*` / `limit_*`; `games → applications` / `invites` / `memberships` (with `ON DELETE CASCADE`); `mail_payloads → mail_deliveries → mail_recipients` / `mail_attempts` / `mail_dead_letters`; `notifications → notification_routes` / `notification_dead_letters`. Cross-domain references (`memberships.user_id`, `games.owner_user_id`, etc.) are kept as opaque `uuid` columns because each domain runs its own cleanup through the in-process cascade described in [§7](#7-in-process-async-patterns). Adding a database cascade would either duplicate that work or hide it behind opaque triggers. - `created_at`, `updated_at`, `deleted_at` are always `timestamptz`. UTC normalisation is applied on read and write. - Idempotency is enforced through UNIQUE indexes on durable tables (for example `(template_id, idempotency_key)` on `mail_deliveries`, `race_name_canonical` on registered race names, `(game_id, user_id)` on `memberships`). There is no separate idempotency table. - Worker pickup uses `SELECT ... FOR UPDATE SKIP LOCKED` ordered by `next_attempt_at`. This pattern serves the mail outbox, retry-able runtime jobs, and any future deferred work. - `session_revocations` is the append-only audit trail of every device session revocation, keyed by `revocation_id` (uuid) with `device_session_id`, `user_id`, `actor_kind`, the actor pair `actor_user_id uuid` + `actor_username text` (exactly one is non-NULL per row, enforced by a CHECK constraint), `reason`, and `revoked_at`. The row is inserted in the same transaction that flips `device_sessions.status` to `'revoked'`, so a successful revoke always leaves a matching audit row. The two-column actor pair is the canonical shape used by every audit-bearing table — `accounts.deleted_actor_*`, `entitlement_records`, `entitlement_snapshots`, `sanction_records.actor_*` + `removed_by_*`, and `limit_records.actor_*` + `removed_by_*` follow the same convention. `actor_kind` (or `actor_type` on the user-domain tables) values are `user`, `admin`, `system`. The Go layer hides the split behind `user.ActorRef{Type, ID string}`: `Type=="user"` requires `ID` to be a UUID, `Type=="admin"` stores `ID` as the operator username (passed to `actor_username`), and `Type=="system"` requires an empty `ID`. See `backend/internal/user/store.go` (`actorToColumnArgs`/`actorFromColumns`) for the SQL boundary. ## 6. In-Memory Cache Postgres is the cold store. In-memory caches in `backend` serve hot reads and are warmed at process start. | Cache | Population | Update path | | ------------------------------- | --------------------------------------------------------- | -------------------------------------------- | | Active device sessions | Full table read at startup. | Write-through on create/revoke. | | User entitlement snapshots | Latest snapshot per active user at startup. | Write-through on entitlement change. | | Engine version registry | Full table read at startup. | Write-through on admin update. | | Active runtime records | Full table read at startup. | Write-through on container ops. | | Active games + memberships | Full table read at startup. | Write-through inside lobby commands. | | Race Name Directory canonicals | Full table read at startup. | Write-through inside lobby commands. | | Admin accounts | Full table read at startup. | Write-through on admin CRUD. | Every cache is bounded to MVP-scale data sets that comfortably fit in process memory (10K accounts, 1000 active games, 100K device sessions, a few thousand directory entries — all together well under 100 MB). If a specific cache is observed to grow beyond a process budget at scale, moving that cache to Redis must be discussed and approved before implementation; the architecture leaves `backend` Redis-free by default. Cache writes happen *after* the matching Postgres mutation commits. A commit failure leaves the cache in sync with the prior database state. Each cache exposes a `Ready` flag flipped to `true` after the warm-up read finishes; the `/readyz` probe waits on every cache being ready before reporting ready, so the listener never serves a request that would spuriously miss because of a cold cache. `gateway` carries a separate, smaller cache: the in-memory session cache fronting every authenticated request. It is a bounded LRU (default 50 000 entries) with a safety-net TTL (default 10 minutes). Misses trigger a single synchronous REST call to backend's `/api/v1/internal/sessions/{id}` lookup; hits answer the hot path directly. The cache is kept consistent through the `session_invalidation` push events backend emits over `Push.SubscribePush`: each event flips the cached entry to `revoked` so subsequent authenticated requests bound to that session are rejected at the edge without another backend round-trip. The TTL covers the case of a missed event (cursor aged out, gateway restart) by forcing a refresh at most once per window. ## 7. In-Process Async Patterns Async work is implemented with goroutines and channels. There is no Redis pub/sub, no Redis Stream, and no message broker between domain modules. The following table records how previously inter-service streams are realised in process. The semantics — when each event fires, how many times, in which order — are preserved; the transport changes from a durable stream to an in-process function call or buffered channel. | Previous external stream | In-process realisation | | ----------------------------------------------------- | - | | User lifecycle (block / soft delete) → Lobby cascade | `lobby.OnUserBlocked(user_id)` and `lobby.OnUserDeleted(user_id)` invoked synchronously after `user` commits. | | Runtime snapshot updates → Lobby denormalisation | `lobby.OnRuntimeSnapshot(snapshot)` invoked from `runtime` after each engine status read. | | Game finished → Lobby promotion / cleanup | `lobby.OnGameFinished(game_id)`. | | Lobby start/stop jobs → Runtime container lifecycle | `runtime.StartGame(game_id)` / `runtime.StopGame(game_id)`. Long-running pull/start drained on a per-game worker goroutine, serialised by per-game mutex. | | Runtime job results → Lobby | Direct return value from `runtime.StartGame`, plus optional `lobby.OnRuntimeJobResult` callback for asynchronous progression. | | Runtime health events | `runtime` publishes onto an in-process channel; `lobby` and `admin` observers consume. | | Notification intents | Direct call `notification.Submit(intent)` by producers (lobby, runtime, geo). | | Mail delivery commands | Direct insert into `mail_deliveries` by producers; mail worker drains the table. | | Auth → Mail (login codes) | Direct call `mail.EnqueueLoginCode(...)` from `auth.confirmEmailCode`. | | Gateway client-events stream | Backend `push` server emits `client_event` on the gRPC stream consumed by gateway. | | Gateway session-events stream | Backend `push` server emits `session_invalidation` on the same gRPC stream. | Workers drain outstanding work on graceful shutdown in a deterministic order: stop accepting new HTTP/gRPC traffic → finish in-flight requests → flush mail outbox writes that already started → flush push events to gateway buffer → close the Docker client → close the database pool. The lobby state machine is the only domain whose transitions cross several producers and consumers. The closed transitions are `draft → enrollment_open → ready_to_start → starting → running ↔ paused → finished`, with `cancelled` reachable from every pre-`finished` state and `start_failed → ready_to_start` for retry. Owner-driven endpoints (or admin overrides for public games) trigger transitions; the `runtime` callback `OnRuntimeJobResult` is the only path that flips `starting → running` or `starting → start_failed`. `lobby.OnGameFinished` is invoked when the engine reports the game finished, after which the runtime container is torn down and Race Name Directory promotions run. ## 8. Backend ↔ Gateway Communication There are two channels between `gateway` and `backend`. **Sync REST (gateway → backend).** Every authenticated user request and every public auth request goes over plain HTTP/JSON. The gateway sends `X-User-ID` (when authenticated) and forwards the verified payload. The backend never re-derives user identity from the body. The session lookup hits backend's `/api/v1/internal/sessions/{id}` only on a cache miss in the gateway-side LRU described in [§6](#6-in-memory-cache); backend updates `device_sessions.last_seen_at` on every successful lookup so admin operators can observe when each session was last resolved at the edge. **gRPC stream (gateway ⇄ backend).** Backend exposes a single RPC `SubscribePush(GatewaySubscribeRequest) returns (stream PushEvent)`. The gateway opens this stream once at start and keeps it open. Each `PushEvent` carries a `oneof`: - `client_event` — opaque payload addressed to `(user_id [, device_session_id])`, which gateway signs and delivers to active client subscriptions. - `session_invalidation` — instructs gateway to immediately close any active streams for `(device_session_id)` or for all sessions of `user_id`, and to reject in-flight requests bound to those sessions. Backend keeps a small in-memory ring buffer of recent events keyed by cursor with TTL equal to the gateway freshness window. On reconnect, gateway sends its last consumed cursor; backend resumes from the next event or from a fresh cursor if the requested point has expired. `gateway` keeps using Redis for anti-replay request_id reservations. No other gateway↔backend interaction uses Redis. ### Edge enforcement `gateway` is responsible for stopping every check it can answer locally so that backend processes only well-shaped, fresh, authentic traffic: - TLS termination and pinning where applicable. - Request envelope parsing, payload hash verification, Ed25519 signature verification, freshness window enforcement, anti-replay reservation. - Public-facing rate limiting and basic policy. - Closing of streams marked invalid via `session_invalidation`. Backend assumes those checks have happened. It runs business validation, authorisation, and state transitions on top of that assumption. ## 9. Backend ↔ Game Engine Communication Backend is the only platform participant that talks to `galaxy-game-*` containers. The contract is the engine OpenAPI document; backend uses the existing typed DTOs in `pkg/model/{order,report,rest}` and a hand-written `net/http` client in `backend/internal/engineclient`. Authenticated client traffic for in-game operations crosses three serialisation boundaries: signed-gRPC FlatBuffers (client ↔ gateway), JSON over REST (gateway ↔ backend), and JSON over REST again (backend ↔ engine). Gateway owns the FB ↔ JSON transcoding for the four message types `user.games.command`, `user.games.order`, `user.games.order.get`, `user.games.report` (FB schemas in `pkg/schema/fbs/{order,report}`, encoders in `pkg/transcoder`). `user.games.order.get` reads back the player's stored order for a given turn — paired with the POST `user.games.order` so the client can hydrate its local draft after a cache loss without re-deriving from the report. Backend never touches FlatBuffers and never re-interprets the JSON beyond rebinding the actor field from the runtime player mapping (clients never carry a trusted actor). Container state is owned by `backend/internal/runtime`: - `runtime_records` is the persistent map from `game_id` to current container state. - `engine_versions` is the registry of allowed engine images and serves as the source for `image_ref` arbitration. Producers do not pick image references on their own. - Patch is semver-patch-only inside the same major/minor line; any major/minor change requires an explicit stop and start. - Reconciliation runs at startup and periodically: every container with the `galaxy.backend` label is matched against `runtime_records`; unrecorded containers with the label are adopted, missing recorded containers are marked removed and an internal event is emitted. - Container naming is fixed: `galaxy-game-{game_id}`; engine endpoint is always `http://galaxy-game-{game_id}:8080`. - Engine probes (`/healthz`) feed `runtime` health observations and turn generation status. ## 10. Geo Profile (reduced) The geo concern is intentionally minimal. - At registration (`/api/v1/public/auth/confirm-email-code`), backend looks up the source IP against the GeoLite2 country database via `pkg/geoip` and stores the resulting ISO country code in `accounts.declared_country`. This value is never updated afterwards; there is no version history. - On every authenticated user-facing request, a fire-and-forget goroutine performs the same lookup against the request IP and increments `user_country_counters` by `(user_id, country, count bigint)`. The request itself does not block on this update. - There is no aggregation, no automatic flagging, no review recommendations, no admin notifications, and no detection of account takeover. Counter data is only available to operators via the admin surface for manual inspection. - Geo work is fail-open: any geoip error is logged but never blocks the user request. - Source IP for both flows is read from the leftmost `X-Forwarded-For` entry, falling back to `RemoteAddr` when the header is absent. Backend trusts the value because the network segment between gateway and backend is the trust boundary ([§15](#15-transport-security-model-gateway-boundary)–[§16](#16-security-boundaries-summary)); duplicating the edge rate-limit / spoof checks here would be double work. - Email addresses are never written to logs verbatim. Backend modules emit a per-process HMAC-SHA256-truncated `email_hash` instead, so operators can correlate log lines within a single process lifetime without persisting PII. ## 11. Mail Outbox Email is delivered through a Postgres-backed outbox. - Producers (auth login codes, notification routes) write into `mail_deliveries` with a unique `(template_id, idempotency_key)` and the rendered payload bytes in `mail_payloads`. - A worker goroutine selects work from `mail_deliveries` with `SELECT ... FOR UPDATE SKIP LOCKED`, attempts SMTP delivery via `wneessen/go-mail`, records the attempt in `mail_attempts`, and either marks the delivery sent or schedules `next_attempt_at` for retry with exponential backoff and jitter. - After the configured maximum retry budget the delivery moves to `mail_dead_letters`. The `mail.dead_lettered` notification kind is reserved in the catalog but has no producer wired up yet, so no admin notification is emitted today — operator visibility comes from a log line and the `/api/v1/admin/mail/dead-letters` listing. - On startup the worker drains everything pending. There is no separate recovery procedure: starting backend is sufficient. - Operators can re-enqueue from `mail_dead_letters` through the admin surface. The auth path returns success as soon as the delivery row is durably committed; SMTP completion is asynchronous to the auth request. ## 12. Notification Pipeline Notifications are an in-process pipeline. The closed catalog is defined in `backend/internal/notification/catalog.go` and currently covers 13 kinds: 10 lobby kinds (invite received/revoked, application submitted/approved/rejected, membership removed/blocked, race name registered/pending/expired) and 3 admin-recipient runtime kinds (image pull failed, container start failed, start config invalid). Per-kind delivery channels (push, email, or both) and the admin-vs- per-user recipient routing live in the same file. For every intent, `notification.Submit` performs: 1. Idempotency check (UNIQUE on `(intent_kind, idempotency_key)`). 2. Recipient resolution against `user`. 3. Per-recipient route materialisation in `notification_routes` — `push`, `email`, or both — based on the type-specific policy table. 4. Push routes are emitted onto the gRPC `client_event` channel for the recipient. The dispatcher passes the producer's payload map through `notification.buildClientPushEvent(kind, payload)`, which maps the kind to the matching FlatBuffers schema in `pkg/schema/fbs/notification.fbs` (one table per catalog kind, 1:1 with the camel-case form of the kind plus the `Event` suffix) and returns a typed `push.Event`. `push.Service` invokes `Marshal` and places the bytes into `pushv1.ClientEvent.Payload`. An unknown kind falls back to `push.JSONEvent` so a misconfigured producer does not silently drop frames; new kinds must ship with a typed FB schema and a matching `buildClientPushEvent` case rather than relying on the fallback. 5. Email routes are inserted into `mail_deliveries` with the matching template id. 6. Malformed intents go to `notification_malformed_intents` and never block the producer. Notification persistence is the auditable record of "we tried to tell this user about this thing"; clients still derive their actual game state through normal user-facing reads. ## 13. Container Lifecycle (in-process) `backend/internal/runtime` owns the lifecycle of game-engine containers and is the only component permitted to issue Docker calls. - All Docker calls go through `dockerclient`, which is a thin wrapper over `github.com/docker/docker` configured against `BACKEND_DOCKER_HOST`. - Per-game container operations are serialised through a per-game mutex (held in memory) so that concurrent start/stop/patch attempts cannot race. `runtime_operation_log` records every operation for audit. - Long-running pulls and starts execute on worker goroutines; the calling path returns as soon as the operation is queued, then receives completion through a callback or a follow-up status read. - The turn scheduler uses `pkg/cronutil` (a wrapper over `robfig/cron/v3`) and schedules a tick per running game according to `games.turn_schedule`. Force-next-turn sets a skip-flag that advances the next scheduled tick by one cron step. - Snapshots are read from the engine on a schedule, after every successful command, and on health probe transitions; each read publishes a `runtime_snapshot_update` to `lobby` in process. Containers managed by `backend` carry the Docker label `galaxy.backend=1`. Reconciliation matches that label against `runtime_records` so a redeploy of `backend` re-attaches to running games rather than orphaning them. Future improvement (not in MVP): introduce a docker-socket-proxy sidecar (for example `tecnativa/docker-socket-proxy`) and connect `dockerclient` through it over TCP. Until then `backend` mounts `/var/run/docker.sock` directly. ## 14. Admin Surface - Admin authentication is HTTP Basic Auth. - Credentials live in the Postgres table `admin_accounts` with `username`, `password_hash` (bcrypt cost 12), `created_at`, `last_used_at`, `disabled_at`. - Bootstrap: at startup `backend` reads `BACKEND_ADMIN_BOOTSTRAP_USER` and `BACKEND_ADMIN_BOOTSTRAP_PASSWORD`; if no `admin_accounts` record with that username exists, it is inserted with the bcrypt hash. The insert is idempotent so restarts are safe. - Existing admins can manage other admins through the same `/api/v1/admin/admin-accounts` endpoints. - All other admin endpoints (`/api/v1/admin/users/*`, `/api/v1/admin/games/*`, `/api/v1/admin/runtimes/*`, `/api/v1/admin/mail/*`, `/api/v1/admin/notifications/*`) reuse the per-domain logic of the module they target. ## 15. Transport Security Model (gateway boundary) This section describes the secure exchange model between client and gateway. It applies at the public boundary and does not rely on backend behaviour for any of its guarantees. The authenticated edge listener is built on `connectrpc.com/connect` and natively serves the Connect, gRPC, and gRPC-Web protocols on a single HTTP/2 cleartext (`h2c`) port. Browser clients use Connect via `@connectrpc/connect-web`; native iOS / Android / desktop clients can use either Connect or raw gRPC framing against the same listener. Envelope, signature, freshness, and anti-replay rules below are protocol-agnostic — they apply identically to every supported wire framing. ### Principles - No browser cookies. - Authentication is device-session based. - Each device session is unique and independently revocable. - No short-lived access tokens or refresh-token flows. - Requests are authenticated by client signatures. - Responses and push events are authenticated by server signatures. - Transport integrity and freshness are verified before any payload is processed. ### Device session model After a successful email-code login: 1. The client generates an Ed25519 key pair. 2. The private key remains on the client. 3. The client public key is registered with `backend` as the standard base64-encoded raw 32-byte Ed25519 key. 4. `backend` creates a persistent device session. 5. The client persists `device_session_id` and the private key. `backend` stores at least `device_session_id`, `user_id`, the base64-encoded raw 32-byte Ed25519 client public key, session status, and revoke metadata. ### Key storage - Native clients use platform secure storage; private keys never leave the device. - Browser/WASM clients use WebCrypto with non-exportable storage where available. Loss of browser storage is acceptable and is recovered by re-login. The concrete browser baseline, IndexedDB schema, and keystore lifecycle live in [`ui/docs/storage.md`](../ui/docs/storage.md). ### Request envelope Each authenticated request carries `payload_bytes`, a `request_envelope`, and a signature. The envelope contains: - `protocol_version` (`v1`) - `device_session_id` - `message_type` - `timestamp_ms` - `request_id` - `payload_hash` (raw 32-byte SHA-256 of `payload_bytes`) The client signs canonical bytes built from: ```text "galaxy-request-v1" || protocol_version || device_session_id || message_type || timestamp_ms || request_id || payload_hash ``` with this binary encoding: - each `string` and `bytes` field is encoded as `uvarint(len(field_bytes))` followed by raw bytes; - `timestamp_ms` is encoded as an 8-byte big-endian unsigned integer; - fields are appended in the exact order listed. The signature scheme is Ed25519. The signature carries the raw 64-byte signature. ### Response envelope Each server response carries `payload_bytes`, a `response_envelope`, and a signature. The envelope contains: - `protocol_version` - `request_id` - `timestamp_ms` - `result_code` - `payload_hash` Canonical bytes: ```text "galaxy-response-v1" || protocol_version || request_id || timestamp_ms || result_code || payload_hash ``` The gateway signs with a PKCS#8 PEM-encoded Ed25519 private key. Clients verify with a trusted server public key. ### Push events Each server push event carries `payload_bytes`, an `event_envelope`, and a signature. Required envelope fields: `event_type`, `event_id`, `timestamp_ms`, `payload_hash`. Optional: `request_id`, `trace_id`. Canonical bytes: ```text "galaxy-event-v1" || event_type || event_id || timestamp_ms || request_id || trace_id || payload_hash ``` Gateway signs each event at delivery time using the same Ed25519 key as for responses. The bootstrap event delivered when a `SubscribeEvents` stream opens is `event_type = gateway.server_time`, reusing the opening `request_id` as `event_id` and carrying `server_time_ms` so clients can calibrate offset without a separate time request. ### Verification order at gateway Before any payload is forwarded to backend, gateway must: 1. Verify the transport envelope is present and supported. 2. Resolve `device_session_id` (against backend, sync REST). 3. Reject unknown or revoked sessions. 4. Verify the client signature using the stored public key. 5. Verify `payload_hash`. 6. Verify timestamp freshness (symmetric ±5 minutes around server time). 7. Verify anti-replay: reserve `(device_session_id, request_id)` until `timestamp_ms + freshness_window`. 8. Apply edge rate limits and basic policy. 9. Forward to backend with `X-User-ID` set. ### Verification order at client Before accepting a response payload, the client must verify the response signature, that `request_id` matches the corresponding request, the `payload_hash`, and where applicable the timestamp freshness. Before accepting a push payload, the client must verify the event signature, the `payload_hash`, the `request_id` when correlated, and where applicable the timestamp freshness. ### Anti-replay Anti-replay uses `(timestamp_ms, request_id)`. Recently seen `request_id` values are tracked per session in Redis until `timestamp_ms + freshness_window`. This protects transport freshness only; business idempotency is a separate concern enforced by backend domain tables. ### TLS and MITM Native clients should use TLS pinning (SPKI-based) in addition to the signed exchange. Browser clients rely on browser-managed TLS and the signed exchange. ### Threat model boundaries The transport model protects against tampering in transit, replay inside the freshness window, use of unknown or revoked sessions, forged server responses without the gateway signing key, and forged client requests without the client signing key. It does not prevent a legitimate user from generating their own valid requests; that is handled by backend business validation and authorisation. ## 16. Security Boundaries Summary | Concern | Enforced by | Notes | | -------------------------------------------------------- | ----------------------- | ----------------------------------------------------------------------------------------------- | | Public TLS termination, pinning | gateway | Native clients pin SPKI. | | Request signature, payload hash, freshness, anti-replay | gateway | See [§15](#15-transport-security-model-gateway-boundary). | | Session lookup | backend (sync REST) + gateway in-memory LRU | gateway-side LRU with TTL safety net ([§6](#6-in-memory-cache)) hits backend's `/api/v1/internal/sessions/{id}` only on miss; no Redis projection. | | Session revocation propagation | backend → gateway | `session_invalidation` over the gRPC push stream flips the gateway-side cache entry to revoked and closes any active push stream. | | Authorisation, ownership, state transitions | backend | `X-User-ID` is the sole identity input on the user surface. | | Edge rate limiting | gateway | Backend has no rate-limit responsibility in MVP. | | Admin authentication | backend | Basic Auth against `admin_accounts`. | | Engine API authentication | network | Engine listens only on the trusted network; backend is the only caller. | ### Backend ↔ Gateway trust The MVP does not require an additional authenticator between gateway and backend. Backend trusts `X-User-ID` from gateway and accepts gateway gRPC subscribers without authentication. The trust boundary is the network: deployment must ensure that only `gateway` can reach `backend`'s HTTP and gRPC listeners. This is an explicit, accepted risk. Compromise of the trusted network between gateway and backend would let any party impersonate any user or admin against backend. The risk is mitigated only by network isolation of the deploy. Adding mutual authentication (a pre-shared bearer token or mTLS between gateway and backend) is a future hardening step; backend is structured so that adding such a check is a single middleware addition. ## 17. Observability - **Tracing and metrics** flow through OpenTelemetry. The default exporter is OTLP (gRPC or HTTP/protobuf, configurable). Metrics may also be exposed via a Prometheus pull endpoint when configured. - **Logging** uses `go.uber.org/zap` in JSON mode. Trace and span ids are injected into every log entry written inside a request scope. - Every backend module emits the metrics relevant to its concern: HTTP request count and duration per route group, gRPC subscription count and push event throughput, mail outbox depth and per-attempt outcomes, notification fan-out counts, container operation counts and durations, Postgres pool stats, geo lookup count and error rate. - Health probes are unauthenticated `GET /healthz` (process liveness) and `GET /readyz` (Postgres reachable, migrations applied, gRPC listener bound). Probes are excluded from anti-replay and rate limiting. ## 18. CI and Environments The repository is monorepo and intentionally so — semver tags and per-service rollouts are achievable without splitting the code into multiple repositories. Branches: - `main` — production-track. Direct pushes are disallowed; the only way in is a PR merge from `development`. - `development` — long-lived dev integration branch. Every merge triggers an auto-deploy into the long-lived dev environment on the CI host, reachable through the host Caddy at `https://www.galaxy.lan` and `https://api.galaxy.lan`. - `feature/*` — short-lived branches off `development`. Merged back via PR; PRs run unit + integration checks before merge. Workflows under `.gitea/workflows/`: | File | Trigger | Purpose | |------|---------|---------| | `go-unit.yaml` | push + PR matching Go paths | Fast Go unit tests. | | `ui-test.yaml` | push + PR matching `ui/**` | Vitest + Playwright. | | `integration.yaml` | PR to `development` / `main`; push to `development` | testcontainers integration suite. | | `dev-deploy.yaml` | push to `development` | Build images, seed UI volume, `compose up` against `tools/dev-deploy/`. | | `prod-build.yaml` | push to `main` | Build production images and persist `docker save` bundles as artifacts. | | `deploy-prod.yaml` | manual `workflow_dispatch` | Placeholder for the future SSH-based production rollout. | Environments: - **`tools/local-dev/`** — single-developer playground. Bound to host ports, Vite dev server runs on the host. Not driven by CI. - **`tools/dev-deploy/`** — long-lived dev environment behind `*.galaxy.lan`, redeployed on every merge into `development`. - **production** — future. Images come from the `galaxy-images-commit-` artifact produced by `prod-build.yaml` and are shipped to the production host via `docker save` → `ssh prod docker load` → `docker compose up -d`. `tools/local-ci/` remains as an opt-in fallback runner for testing workflow changes without `gitea.lan`. It is no longer part of the per-stage CI gate; see `CLAUDE.md` for the gate definition. ## 19. Deployment Topology (informational) - MVP runs three executables: one `gateway` instance, one `backend` instance, and N `galaxy-game-{game_id}` containers managed by backend. - One Postgres database is shared by `backend` only. - One Redis instance is reachable from `gateway` only (anti-replay). - One SMTP relay is reachable from `backend`. - The Docker daemon socket is mounted into `backend`. - The GeoLite2 country database file is mounted at the path given by `BACKEND_GEOIP_DB_PATH`. Future scale-out hooks (not in MVP): - Distributed `backend` requires reintroducing Redis for shared session cache and runtime job leasing, plus leader election for the turn scheduler. - mTLS between gateway and backend. - Docker-socket-proxy sidecar fronting Docker daemon access. ## 20. Glossary - **device_session_id** — opaque identifier of an authenticated client device; primary key of the device session record. - **race_name** — in-game player display name. Three tiers in the Race Name Directory: registered (platform-unique), reservation (per-game), pending_registration (post-capable-finish). - **canonical key** — lowercased and confusable-folded form of a race name used for uniqueness checks, computed via `disciplinedware/go-confusables`. - **capable finish** — a finished game in which the player reached `max_planets > initial AND max_population > initial`. Only capable finishes promote a reservation to `pending_registration`. - **runtime snapshot** — engine-status read materialised into the lobby's denormalised view: `current_turn`, `runtime_status`, `engine_health_summary`, `player_turn_stats`. - **turn cutoff** — the `running → generation_in_progress` runtime-status flip performed by `backend/internal/runtime/scheduler.go` before each engine `/admin/turn` call. Commands and orders arriving while the flag is set are rejected by the user-games handlers with HTTP 409 `turn_already_closed`. The matching reopening flip (`generation_in_progress → running`) happens on a successful tick; a failing tick instead drives the lobby to `paused` and fans out `game.paused` (FUNCTIONAL.md §6.3, §6.5). - **auto-pause** — the lobby reaction to a failed runtime snapshot (`engine_unreachable` / `generation_failed`): the game flips `running → paused`, the order handlers refuse new submits with HTTP 409 `game_paused`, and `lobby.publishGamePaused` fans out the push event. Only an admin `/resume` followed by a successful tick recovers the game; the UI relies on the next `game.turn.ready` to clear the paused banner. - **outbox** — the durable queue of pending mail rows in `mail_deliveries`, drained by the mail worker. - **freshness window** — the symmetric ±5-minute interval around server time inside which a request `timestamp_ms` is accepted. - **trust boundary** — the network segment between gateway and backend. Compromise of this segment defeats backend authentication; deployment must isolate it.