diplomail (Stage E): LibreTranslate client + async translation worker
Tests · Go / test (push) Successful in 1m59s
Tests · Go / test (pull_request) Successful in 2m1s
Tests · Integration / integration (pull_request) Successful in 1m37s

Synchronous translation on read (Stage D) blocks the HTTP handler on
translator I/O. Stage E switches to "send moments-fast, deliver
when translated": recipients whose preferred_language differs from
the detected body_lang are inserted with available_at=NULL, and an
async worker turns them on once a LibreTranslate call materialises
the cache row (or fails terminally after 5 retries).

Schema delta on diplomail_recipients: available_at,
translation_attempts, next_translation_attempt_at, plus a snapshot
recipient_preferred_language so the worker queries do not need a
join. Read paths (ListInbox, GetMessage, UnreadCount) filter on
available_at IS NOT NULL. Push fan-out is moved from Service to the
worker so the recipient only sees the toast when the inbox row is
actually visible.

Translator backend is now a configurable choice: empty
BACKEND_DIPLOMAIL_TRANSLATOR_URL → noop (deliver original);
populated → LibreTranslate HTTP client. Per-attempt timeout, max
attempts, and worker interval all live in DiplomailConfig. The HTTP
client itself is unit-tested via httptest (happy path, BCP47
normalisation, unsupported pair, 5xx, identical src/dst, missing
URL); worker delivery + fallback paths are covered by the
testcontainers-backed e2e tests in diplomail_e2e_test.go.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilia Denisov
2026-05-15 20:15:28 +02:00
parent e22f4b7800
commit 9f7c9099bc
16 changed files with 1222 additions and 155 deletions
+42
View File
@@ -18,6 +18,7 @@ purge, and the language-detection / translation cache.
| B | Owner / admin sends + lifecycle hooks (paused, cancelled, kick); strict soft-access for kicked players | shipped |
| C | Paid-tier personal broadcast + admin multi-game broadcast + bulk purge + admin observability | shipped |
| D | Body-language detection (whatlanggo) + translation cache + lazy per-read translator dispatch | shipped |
| E | LibreTranslate HTTP client + async translation worker with exponential backoff + delivery gating on translation completion | shipped |
## Tables
@@ -94,6 +95,47 @@ Future work plugs a real `translator.Translator` (LibreTranslate
HTTP client is the documented next step) without touching the rest
of the system.
## Async translation (Stage E)
Stage E switches the translation pipeline from "lazy at read" to
"async at send". The send path stays synchronous from the
caller's perspective: the message and recipient rows are inserted
in one transaction. What changes is delivery semantics:
- Recipients whose `preferred_language` matches the detected
`body_lang` (or whose body language is `und`) get
`available_at = now()` straight away and the push event fires
during the request.
- Recipients whose `preferred_language` differs are inserted with
`available_at IS NULL`. They are **not** visible in inbox, unread
count, or push events until the worker translates the message.
The worker (`internal/diplomail.Worker`, started as an
`app.Component` in `cmd/backend/main`) ticks once every
`BACKEND_DIPLOMAIL_WORKER_INTERVAL` (default `2s`). Each tick:
1. Picks one distinct `(message_id, recipient_preferred_language)`
pair from `diplomail_recipients` where `available_at IS NULL`
and `next_translation_attempt_at` is unset or due.
2. Loads the source message, checks the translation cache.
3. On cache hit → marks every pending recipient of the pair
delivered and emits push.
4. On cache miss → asks the configured `Translator`:
- success → caches the translation, marks delivered, push;
- HTTP 400 (unsupported pair) → marks delivered without a
translation (fallback to original);
- other failure → bumps `translation_attempts`, schedules the
retry via `next_translation_attempt_at`, leaves pending.
5. After `BACKEND_DIPLOMAIL_TRANSLATOR_MAX_ATTEMPTS` (default `5`)
the worker falls back to delivering the original body so a
prolonged LibreTranslate outage does not strand messages.
Retry backoff is exponential `1s → 2s → 4s → 8s → 16s` (capped at
60s) per pair. Operators monitor the LibreTranslate dependency
through standard OpenTelemetry export — translation outcomes
surface in `diplomail.worker` logs at Info / Warn levels;
Grafana / Prometheus dashboards live outside this package.
## Push integration
Every successful send emits a `diplomail.message.received` push