The bot runs on its own host and exports no telemetry (otelcol is unreachable
from there), so it was a monitoring blind spot. Observe its Bot API health
centrally by wrapping the HTTP client — one place, no per-call-site
instrumentation — and relay it up the existing bot-link as a periodic Health
message the gateway turns into its own metrics.
- proto: add Health (delta connect / api / 429 counters + a last-ok stamp) to
the FromBot oneof (additive, backward-compatible).
- bot: platform/telegram/internal/health wraps the Bot API HTTP client — it
stamps liveness on any 2xx (so the getUpdates long-poll keeps it fresh even
when idle), classifies transport/5xx failures (getUpdates vs other) and 429s,
and honours a 429's Retry-After (bounded) so the bot backs off; a 4xx other
than 429 is a normal per-request outcome and is not counted.
- bot-link client: flush the reporter as a Health message every 30s over a
single-sender loop (a gRPC stream forbids concurrent Send).
- gateway: fold each report into bot_tg_errors_total{kind} and the
bot_tg_last_ok_unix liveness gauge.
- grafana: alerts (bot disconnected, bot not reaching the Bot API, sustained
429s) routed to the operator email — which does not go through the bot — plus
a Telegram-bot dashboard.
- docs (ARCHITECTURE, compose comments); unit tests (observer classification,
Retry-After, snapshot/commit, hub last-ok monotonicity).
Accept real money via Telegram Stars (XTR) — the third intake rail
alongside Robokassa (direct) and VK Votes.
Only the bot reaches Telegram, so the rail funnels through the reverse
mTLS bot-link:
- the gateway mints the invoice on a CreateInvoice command (the bot
calls createInvoiceLink, XTR; the link goes to WebApp.openInvoice);
- the bot gates each pre_checkout_query via a ValidatePreCheckout unary
(the order must exist, be still creditable and not already paid — the
reusable-invoice double-pay guard; the decline reason is localised to
the order account's language);
- a completed successful_payment is queued in a durable pure-Go SQLite
outbox and forwarded via a ForwardPayment unary, credited once
(idempotent on telegram_payment_charge_id, honours an expired order),
re-driven on restart and every 30s.
The rail is wired by TELEGRAM_STARS_OUTBOX_DIR (default /data) but stays
inert until a chip pack carries an XTR price, so seeding a Stars price in
the admin is the go-live.
Tests: backend integration (order->forward->credit once, duplicate,
pre_checkout gate) + bot outbox unit (idempotent, restart re-drive) +
executor createInvoice. Docs: PAYMENTS(+ru) §9, ARCHITECTURE, the
platform/telegram README, PLAN.
The promo bot button carries a configurable variant-seed start-param (default verudit_ru-scrabble_en). The gateway parses start_param from the validated initData and forwards it; the backend, on first contact only, seeds the new account variant_preferences from it (English Scrabble alongside the default Erudit).
No schema change (the scrabble_en CHECK is already in the baseline) and the gateway<->backend REST field is additive, so the rolling deploy is safe in either order. TELEGRAM_PROMO_START_PARAM configures the payload (empty forwards the user own /start payload). Covered by account unit tests, a gateway transcode test, and an integration test asserting new-only seeding.
Users who DM the bot anything but /start are relayed into a private
forum supergroup, one topic per user. Operators (the chat's admins)
reply in the topic and the bot copies it back to the user; an info card
opening each topic carries a Block/Unblock toggle and a Clear button.
State is a small JSON file on a new /data volume — the bot host has no
database. Off by default (TELEGRAM_SUPPORT_CHAT_ID=0): the prod bot is
unchanged until the operator sets the chat id and adds the bot as a
forum admin.
- internal/support: concurrency-safe JSON store (topic map, block list,
relayed message ids) with field-targeted mutators and atomic save
- bot/support.go: relay both ways via copyMessage, short-TTL admin
cache, callback buttons, per-user topic-create lock, loop guard
(skip the bot's own posts), reopen a deleted topic on the next message
- config + compose + CI/prod-deploy: TELEGRAM_SUPPORT_CHAT_ID per
contour + TELEGRAM_SUPPORT_STATE_DIR; bot-state named volume; /data
pre-owned by UID 65532 so a fresh volume is writable under distroless
- docs: ARCHITECTURE §15 + decision record, FUNCTIONAL (+ru), README
Split the contour across the two prod hosts and retune for the small main host:
- Gate vpn+bot to the telegram-local profile. The CI test deploy now passes
--profile telegram-local so the test contour still brings them; the prod main
host omits both, and the prod bot runs standalone from docker-compose.bot.yml.
- docker-compose.prod.yml (main-host overlay): publish caddy 80/443 (no host
caddy in prod; caddy owns ACME) and gateway 9443 (the remote bot dials in over
mTLS); GOMAXPROCS=2, smaller memory caps and 7d Prometheus retention for the
2 vCPU / 1.9 GiB host. It launches deliberately undersized; resize reactively.
- docker-compose.bot.yml: standalone bot for the tg host (no VPN, OTLP off since
otelcol is unreachable from there, dials the main host's bot-link).
- Add node_exporter + a Prometheus scrape so host memory pressure (the OOM
signal on the tight main host), not just per-container docker_stats, is visible.
- Soften AWG_CONF to a default: only the profiled vpn sidecar consumes it, and
compose interpolates profiled-out services too, so prod must not require it.
Ilia Denisov