docs(deploy): bake Stage 18 prod-deploy decisions into the live docs
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 16s
CI / ui (pull_request) Successful in 57s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m8s
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 16s
CI / ui (pull_request) Successful in 57s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m8s
- ARCHITECTURE §13 prod bullet -> the realized mechanism: registry transport, two-host, rolling + auto-rollback, migration maintenance window, node_exporter, the undersized launch; the contour paragraph notes node_exporter + the telegram-local profile. - deploy/README gains a prod rollout runbook (how to run, migrations/restore, cert rotation, sizing/monitoring, the full PROD_ set) + node_exporter row, the telegram-local profile note, and the soft AWG_CONF note. - PLAN Stage 18 records the resolved open details and the remaining live cutover (pending erudit-game.ru DNS); the tracker reads 'machinery built; cutover pending DNS'. - PRERELEASE TX/AG note the prod wiring is built.
This commit is contained in:
+28
-12
@@ -1056,8 +1056,10 @@ plaintext relay (`GATEWAY_BOTLINK_RELAY_ADDR`) the backend admin console calls.
|
||||
|
||||
The full contour (`deploy/docker-compose.yml`) runs one `gateway`, one `backend`,
|
||||
one Postgres, the static `landing`, the Telegram `validator` and `bot` (+ the bot's VPN
|
||||
sidecar) and the **observability stack** —
|
||||
OTel Collector (OTLP/gRPC ingest → Prometheus metrics + Tempo traces) and Grafana
|
||||
sidecar — the `bot`+`vpn` pair is gated to a `telegram-local` compose profile so the prod
|
||||
main host can omit them) and the **observability stack** —
|
||||
OTel Collector (OTLP/gRPC ingest → Prometheus metrics + Tempo traces), a `node_exporter`
|
||||
for host CPU/memory (the prod main host's OOM signal), and Grafana
|
||||
with provisioned datasources and dashboards. All services export OTLP to the
|
||||
collector; the bot shares the VPN sidecar's netns, so its `AWG_CONF` must not
|
||||
carry a `DNS=` directive (that would hijack resolv.conf and stop it resolving
|
||||
@@ -1081,16 +1083,30 @@ Two contours, two secret/variable prefixes (`TEST_` / `PROD_`):
|
||||
generated by `deploy/gen-certs.sh` before `compose up`; the bot keeps its VPN sidecar
|
||||
for Telegram egress and dials the gateway by its internal name, so the bot-link stays
|
||||
on the internal network.
|
||||
- **Prod**: a manual SSH deploy after `development → master`. There is no
|
||||
host caddy, so the contour ships its own caddy terminating TLS — set
|
||||
`CADDY_SITE_ADDRESS` to the domain and the caddy does its own ACME. The **bot runs
|
||||
on a separate host** with native Telegram access (no VPN), deployed by SSH alongside
|
||||
the main app (rolled together so the bot-link protocol versions never skew); the
|
||||
gateway **publishes** the bot-link port and the certificates come from `PROD_`
|
||||
secrets — a long-lived CA with leaves rotated by a scheduled job. The bot dials the
|
||||
gateway's public bot-link endpoint and holds no inbound port; login is unaffected if
|
||||
that host or the link is down. *(This prod wiring is the deferred final stage; the
|
||||
code and the unified test contour land first — see `PRERELEASE.md`.)*
|
||||
- **Prod**: a **manual** rollout — `.gitea/workflows/prod-deploy.yaml`, `workflow_dispatch`
|
||||
only (from `master`, `confirm=deploy`), run after `development → master` is merged green.
|
||||
It builds and pushes the images to the registry (`docker.iliadenisov.ru`), then deploys
|
||||
over SSH onto **two hosts** provisioned by `deploy/ansible/` (docker, a non-sudo `deploy`
|
||||
service account holding a dedicated CI key, key-only sshd, default-deny ufw, fail2ban):
|
||||
the **main host** runs the full stack (`docker-compose.yml` + `docker-compose.prod.yml`),
|
||||
the **bot host** runs only the bot (`docker-compose.bot.yml`, no VPN — native Bot API
|
||||
egress, telemetry off). There is no host caddy, so the contour caddy terminates TLS —
|
||||
`CADDY_SITE_ADDRESS` is the domain and caddy does its own ACME. The gateway **publishes**
|
||||
the bot-link `:9443`; the remote bot dials it over mTLS (certs from `PROD_BOTLINK_*`,
|
||||
ServerName `gateway`, so TLS validation is independent of the public dial address), holds
|
||||
no inbound port, and login is unaffected if that host or the link is down.
|
||||
`deploy/prod-deploy.sh` rolls the main stack **one service at a time in dependency order**
|
||||
(postgres → backend → gateway → landing → validator → caddy), health-checking after each;
|
||||
any failure **rolls the whole stack back to the previous image tag**. A **schema migration**
|
||||
adds a maintenance window: the backend (the sole writer) is stopped for a consistent
|
||||
`pg_dump` before the new backend migrates forward — image rollback stays DB-safe under the
|
||||
expand-contract migration rule, and the dump is kept for a manual restore. The main host is
|
||||
intentionally **launch-sized** (2 vCPU / 1.9 GiB): the prod overlay trims the R7 limits
|
||||
(`GOMAXPROCS=2`, smaller caps, 7d Prometheus retention) and a **node_exporter** feeds
|
||||
host-memory metrics to Grafana so it can be resized reactively as players arrive.
|
||||
`GATEWAY_ABUSE_BAN_ENABLED=true` in prod (the per-IP ban is meaningful only with real
|
||||
client IPs). The `vpn`+`bot` pair is gated to a `telegram-local` compose profile the test
|
||||
contour activates; the prod main host omits it.
|
||||
|
||||
## 14. CI & branches
|
||||
|
||||
|
||||
Reference in New Issue
Block a user