The prod rolling deploy rolls each service with `compose up -d --no-deps <svc>`.
For caddy that is a no-op on a config-only release: its image is pinned
(caddy:2-alpine, no $TAG), so the compose definition is unchanged between
releases, compose treats the container as current and does not recreate it, and
admin is off so there is no hot reload. The new bind-mounted Caddyfile is seeded
to the host but never loaded -- the v1.2.2 `Alt-Svc: clear` edge fix deployed
green yet did not take effect until caddy was restarted by hand.
Force a recreate for caddy on its roll (every other service already recreates on
its new $TAG image), so a bind-mounted Caddyfile change always applies. Costs a
~1-2s caddy blip per deploy, acceptable for the infrequent manual prod rollout.
Drop the literal version default from the build files (backend Dockerfile both
stages, loadtest Dockerfile, the compose build-arg) so the release tag is not
duplicated as a stale-prone default a newcomer can't tell from the real source.
DICT_VERSION is now required: compose uses ${DICT_VERSION:?…} and the Dockerfiles
have no ARG default, so a missing value fails loudly instead of baking a stale tag.
The tag lives only in its genuine sources — ci.yaml env (CI tests), the Gitea
TEST_/PROD_DICT_VERSION variables (deploy seed) and deploy/.env.example (local).
Adds a "Bumping the dictionary version" section to deploy/README and fixes the bare
docker-build examples (CLAUDE.md, README.md, loadtest/README) to pass --build-arg.
Bump DICT_VERSION v1.2.1 -> v1.3.0 across the seed surface: .env.example, the
compose build-arg default, both backend Dockerfile stages, the loadtest
Dockerfile, the CI dawg-download version, and the deploy/backend docs.
v1.3.0 drops the abbreviation class from the Russian word list (scrabble-dictionary
#6). This only seeds a FRESH volume; a live contour/prod volume is unaffected (the
.seed_version marker wins — seed-drift guard) and moves to v1.3.0 through the admin
console (ARCHITECTURE §5). Per-contour deploy still overrides via TEST_/PROD_DICT_VERSION.
- prod-deploy.yaml is now four visible sequential jobs (build -> deploy-main ->
deploy-bot -> verify) so the rollout stages show in the Actions UI; the
per-service rolling stays in the deploy-main log.
- prod-rollback.yaml: a separate manual workflow_dispatch. Leave target_version
blank to roll back to the previous deployed version (the host now tracks
DEPLOYED_TAG + PREVIOUS_TAG), or pick a release tag. Re-deploys an already
published image rolling + health-gated, image-only (no rebuild, no DB migration).
- prod-deploy.sh tracks the previous tag (commit_tag) for the blank-input rollback.
- Docs: ARCHITECTURE §13 + deploy/README runbook cover versioning + rollback.
- ARCHITECTURE §13 prod bullet -> the realized mechanism: registry transport,
two-host, rolling + auto-rollback, migration maintenance window, node_exporter,
the undersized launch; the contour paragraph notes node_exporter + the
telegram-local profile.
- deploy/README gains a prod rollout runbook (how to run, migrations/restore, cert
rotation, sizing/monitoring, the full PROD_ set) + node_exporter row, the
telegram-local profile note, and the soft AWG_CONF note.
- PLAN Stage 18 records the resolved open details and the remaining live cutover
(pending erudit-game.ru DNS); the tracker reads 'machinery built; cutover pending DNS'.
- PRERELEASE TX/AG note the prod wiring is built.
Move all Telegram egress off the main host. The single connector held the
bot token, long-polled Telegram and answered the gateway/backend over the
trusted internal network, so the whole component (including login validation)
shared fate with its VPN sidecar. Split it into two binaries that share the
token:
- cmd/validator (home, no VPN): Mini App initData + Login Widget HMAC only,
never calls the Bot API. The gateway dials it for Telegram auth, so game
login is now independent of Telegram reachability.
- cmd/bot (remote): Bot API long-poll + sendMessage, the only component
reaching Telegram. It holds no inbound port — it dials the gateway over a
new reverse mTLS bot-link (pkg/proto/botlink/v1) and executes the send
commands the gateway pushes.
The gateway funnels sends to the bot-link: out-of-app push is fire-and-forget
(at-most-once, dropped if no bot is connected); the backend admin broadcasts
reach a gateway-served relay that forwards them and awaits the bot's ack
(SendToUser/SendToGameChannel contract preserved). mTLS (pkg/mtls) is the one
inter-service link that leaves the trusted segment; validator<->gateway and
the relay stay plaintext internal. The bot is Telegram-rate-limited.
One bot now; the gateway bot registry, an owns_updates flag and per-command
ids leave seams for N later. Webhook rejected (one URL per token, adds inbound
+ a static address).
The unified test contour runs the split (the bot keeps its VPN sidecar and
dials the gateway by its internal name; bot-link certs from deploy/gen-certs.sh,
generated in CI). The prod wiring — the bot on a separate host (no VPN), the
gateway bot-link port published, PROD_ certs with scheduled rotation, an SSH
deploy of both hosts together — is the deferred final stage (PRERELEASE.md TX,
Stage 18).
Docs: ARCHITECTURE, PRERELEASE (phase TX), platform/telegram + gateway +
backend + deploy READMEs, FUNCTIONAL(+ru), CLAUDE.md, .env.example.
The seed-drift guard shipped as refuse-boot: the backend exited when
BACKEND_DICT_VERSION disagreed with the flat dir's recorded .seed_version. On
the test contour that turned a harmless-in-intent action — bumping the
TEST_DICT_VERSION variable to the active release (v1.2.1) on a volume seeded as
v1.0.0 — into a crash loop, because DICT_VERSION is the *seed* of a fresh
volume, not the active version (which the admin console drives).
Make the marker authoritative instead: OpenWithVersions resolves the flat dir's
version from .seed_version when present and ignores bootVersion on an
already-seeded volume; bootVersion only seeds a fresh volume's marker. So a
bumped build seed on a live volume is a no-op (it can't relabel live bytes and
can't void games pinned to the prior label), and it correctly seeds the next
fresh volume. The subdirectory scan now skips the resolved seed, so a version
also present as a subdir (e.g. v1.2.1 uploaded via the console while the build
seed is bumped to v1.2.1) is still loaded rather than shadowed by the flat bytes.
Tests: marker-wins over a bumped boot version; a bumped boot keeps the matching
subdir resident (the live-contour case). Docs updated (ARCHITECTURE §5, READMEs,
compose/.env, PRERELEASE DV) from "refuses to boot" to "marker wins / ignored".
Verified locally against v1.2.1: gofmt, build, vet, unit, integration green.
The dictionary release moved to v1.2.1 while DICT_VERSION stayed pinned at
v1.0.0 in CI and the image/compose seed defaults. Two problems:
1. CI validated against a stale dictionary.
2. The contour seed could be bumped on a live volume, which silently relabels
the already-seeded bytes — voiding games pinned to the prior label and
serving the wrong dictionary for new ones. The flat DAWGs carry no embedded
version, so this drift was undetectable.
Changes:
- Seed-drift guard: OpenWithVersions records the flat dir's version in a
.seed_version marker on first boot and refuses to start when a later
BACKEND_DICT_VERSION disagrees. DICT_VERSION is now the seed for a *fresh*
volume only; a live contour migrates through the admin console (old versions
stay resident, in-progress games keep replaying).
- Track the current release: CI's DICT_VERSION centralised to one workflow-level
env (v1.2.1); image/compose/.env seed defaults bumped to v1.2.1. The deploy
job keeps reading the per-contour vars.TEST_DICT_VERSION.
- Docs: ARCHITECTURE §5 (decision record), backend/deploy READMEs, PRERELEASE
tracker (DV row).
Verified locally against the v1.2.1 artifact: gofmt, build, vet, unit and
integration (-tags=integration) all green.
Collapse the two per-language Telegram bots into one unified bot and
replace language-based variant gating with explicit per-user variant
preferences.
- Telegram: one bot; drop service_language and the supported_languages
set everywhere (DB, account, auth, FlatBuffers Session wire, gateway,
connector proto). The single bot renders chat and out-of-app push in
the recipient's preferred_language; remove the game-language push
routing override (notify Intent.Language / push Event.language).
- Preferences: new accounts.variant_preferences (text[], DB default
{erudit_ru}, CHECK non-empty + subset of the three variants). Gates
the New Game picker, vs-AI and the friend invitation the player
creates, enforced server-side (HTTP 400 otherwise); an invited friend
may still accept any variant. Edited on the Settings screen; variants
are Erudit-first everywhere.
- Admin: drop the per-bot language selectors (broadcast / send-to-user)
and the feedback channel_lang column/field.
- Env/CI: collapse TELEGRAM_BOT_TOKEN_{EN,RU}, TELEGRAM_GAME_CHANNEL_ID_{EN,RU},
VITE_TELEGRAM_LINK{,_EN,_RU} and VITE_TELEGRAM_GAME_CHANNEL_NAME_{EN,RU}
to single unsuffixed names; drop GATEWAY_DEFAULT_SUPPORTED_LANGUAGES.
- Docs updated (ARCHITECTURE, FUNCTIONAL + _ru, platform/telegram, gateway,
backend, ui, UI_DESIGN, PRERELEASE).
The migration squash is deferred to a follow-up PR.
The friend-code 'share' was an <a> that just opened the bot. Make it a real share:
Telegram's native share-to-chat picker inside the Mini App (openTelegramLink +
t.me/share/url), the system share sheet (navigator.share) on the web, else copy the
link. The shared deep link points at the same bot the player is in — it picks
VITE_TELEGRAM_LINK_EN/_RU by the session's service language, falling back to the
single VITE_TELEGRAM_LINK. Adds the per-bot build args across Dockerfile / compose /
ci.yaml / .env / docs; PLAN TODO-5 updated.
Owner review-pass rework of the landing page:
- Rename the per-language Telegram link build var
VITE_TELEGRAM_LINK_EN/_RU -> VITE_TELEGRAM_GAME_CHANNEL_NAME_EN/_RU
(it carries a channel username; the landing builds https://t.me/<name> --
the same channels the connector posts to via TELEGRAM_GAME_CHANNEL_ID_*).
- Language switcher -> a globe icon dropdown (flags + names), saved + synced
to the app prefs.
- Theme switcher -> a sun/moon icon toggle, ephemeral (follows the system
scheme, no auto, never persisted) -- galaxy-game style.
- Drop the "Play in browser" CTA (no standalone-web onboarding yet).
Docs: FUNCTIONAL(+ru), PLAN, deploy + ui READMEs.
Close out Stage 17 round 6:
- Landing page at / — one Vite build with two entries (index.html = game
SPA, landing.html = a lightweight landing reusing the theme/i18n/
aboutContent leaf modules, not the app store).
- Move the web game SPA to /app/; the Telegram Mini App stays at /telegram/
(gateway webui.Handler(stripPrefix, indexName): landing at /, SPA at /app/
+ /telegram/). Per-language "Play in Telegram" link via new
VITE_TELEGRAM_LINK_EN/_RU build vars (button hides when unset).
- Cache headers: hash-named /assets/* immutable, HTML shells no-cache (the
go:embed zero modtime emitted no validators, so the client re-downloaded
the whole bundle every launch).
- Live-stream 15s abort fix: an immediate heartbeat on open + a 10s default
interval (the first tick at 15s raced the edge idle timeout -> reconnect
storm).
PLAN/ARCHITECTURE(§13)/FUNCTIONAL(+ru)/gateway+ui+deploy READMEs updated;
round 6 closed. Tests: gateway webui/connectsrv units, ui landing unit + e2e,
full e2e (60) green.
- #10 a `changes` job path-filters unit/integration/ui; an always-running `gate` job aggregates them (success-or-skipped) and becomes the only required check
- #9 deploy adds a Telegram-connector liveness probe (docker inspect: running, not restarting, stable restart count) with a VPN-handshake grace period
- #1a Game-domain dashboard gains a 'Move think-time by phase (p50/p95)' panel
- deploy README: branch protection now requires only CI / gate
Root cause of the Grafana "readdirent /etc/grafana/dashboards: no such file or
directory": the CI runner checks out into an ephemeral act workspace that is
removed after the job, so binding the compose config files straight from it
dangles the mounts in the long-lived containers (verified the act source dir is
emptied after the job). caddy/otelcol/prometheus/tempo read their config once at
startup so they survive, but would break on a restart — same latent bug.
Fix (mirrors ../galaxy-game's $HOME/.galaxy-dev/monitoring): the deploy job seeds
the config dirs to a stable $HOME/.scrabble-deploy and the compose binds them via
${SCRABBLE_CONFIG_DIR:-.} (local runs keep "."). Documented in the compose header,
deploy/README.md and the ci.yaml step.
- deploy/docker-compose.yml: mount the provisioned dashboards at
/etc/grafana/dashboards, not /var/lib/grafana/dashboards — the grafana-data
volume mounts over the latter and shadows the nested bind, so the provider
logged "readdirent /var/lib/grafana/dashboards: no such file or directory".
dashboards.yaml provider path updated to match.
- Connector telemetry stays OTLP. The VPN sidecar's netns reaches the collector's
internal IP fine (connected route, off-tunnel), but the sidecar's DNS hijacks
name resolution: AWG_CONF must NOT carry a DNS= directive, else otelcol won't
resolve ("produced zero addresses"). Without DNS= the netns uses Docker's
resolver (resolves both otelcol and api.telegram.org). Documented in
deploy/README.md (AWG_CONF row + wiring note), ARCHITECTURE §13, compose comment.
- PLAN.md: new Stage 17 "Test-contour verification & defect fixes" (exercise the
deployed contour end-to-end and fix what it surfaces — connector liveness check,
path-conditional CI); the former prod-deploy stage becomes Stage 18.
- Renumber every "Stage 17" prod-deploy reference to "Stage 18" across docs,
compose, Caddyfile, ci.yaml and CLAUDE.md; the post-Stage-14 split range is now
"Stages 15–18".
- bot.New now selects Telegram's test environment with the library's native
tgbot.UseTestEnvironment() instead of a token += "/test" hack (functionally
identical URL /bot<token>/test/METHOD, but idiomatic) + a bot test asserting
the getMe path for both test and prod.
- ci.yaml pins TELEGRAM_TEST_ENV=true for the test contour (it IS the test
environment) instead of a TEST_TELEGRAM_TEST_ENV variable: removes the
confusing double-TEST, telegram-specific, prefixed operator knob and the
secret-vs-variable footgun. Prod (Stage 17) leaves it false.
- deploy/README.md + PLAN.md updated.
- deploy/README.md documents the services, how to run it locally and in CI, and
every variable: required (the four :? ones + ≥1 bot token) and optional with
defaults, marked secret-vs-variable and with the TEST_/PROD_ Gitea mapping;
plus the fixed internal wiring and the host-side setup.
- ci.yaml maps the remaining POSTGRES_DB/USER, DICT_VERSION and LOG_LEVEL (unset
renders empty -> the compose ":-" defaults apply), so every documented var is
per-contour overridable.
- .env.example points at the README for the full reference.
Ilia Denisov