diplomail (Stage E): LibreTranslate client + async translation worker
Synchronous translation on read (Stage D) blocks the HTTP handler on translator I/O. Stage E switches to "send moments-fast, deliver when translated": recipients whose preferred_language differs from the detected body_lang are inserted with available_at=NULL, and an async worker turns them on once a LibreTranslate call materialises the cache row (or fails terminally after 5 retries). Schema delta on diplomail_recipients: available_at, translation_attempts, next_translation_attempt_at, plus a snapshot recipient_preferred_language so the worker queries do not need a join. Read paths (ListInbox, GetMessage, UnreadCount) filter on available_at IS NOT NULL. Push fan-out is moved from Service to the worker so the recipient only sees the toast when the inbox row is actually visible. Translator backend is now a configurable choice: empty BACKEND_DIPLOMAIL_TRANSLATOR_URL → noop (deliver original); populated → LibreTranslate HTTP client. Per-attempt timeout, max attempts, and worker interval all live in DiplomailConfig. The HTTP client itself is unit-tested via httptest (happy path, BCP47 normalisation, unsupported pair, 5xx, identical src/dst, missing URL); worker delivery + fallback paths are covered by the testcontainers-backed e2e tests in diplomail_e2e_test.go. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -18,6 +18,7 @@ purge, and the language-detection / translation cache.
|
||||
| B | Owner / admin sends + lifecycle hooks (paused, cancelled, kick); strict soft-access for kicked players | shipped |
|
||||
| C | Paid-tier personal broadcast + admin multi-game broadcast + bulk purge + admin observability | shipped |
|
||||
| D | Body-language detection (whatlanggo) + translation cache + lazy per-read translator dispatch | shipped |
|
||||
| E | LibreTranslate HTTP client + async translation worker with exponential backoff + delivery gating on translation completion | shipped |
|
||||
|
||||
## Tables
|
||||
|
||||
@@ -94,6 +95,47 @@ Future work plugs a real `translator.Translator` (LibreTranslate
|
||||
HTTP client is the documented next step) without touching the rest
|
||||
of the system.
|
||||
|
||||
## Async translation (Stage E)
|
||||
|
||||
Stage E switches the translation pipeline from "lazy at read" to
|
||||
"async at send". The send path stays synchronous from the
|
||||
caller's perspective: the message and recipient rows are inserted
|
||||
in one transaction. What changes is delivery semantics:
|
||||
|
||||
- Recipients whose `preferred_language` matches the detected
|
||||
`body_lang` (or whose body language is `und`) get
|
||||
`available_at = now()` straight away and the push event fires
|
||||
during the request.
|
||||
- Recipients whose `preferred_language` differs are inserted with
|
||||
`available_at IS NULL`. They are **not** visible in inbox, unread
|
||||
count, or push events until the worker translates the message.
|
||||
|
||||
The worker (`internal/diplomail.Worker`, started as an
|
||||
`app.Component` in `cmd/backend/main`) ticks once every
|
||||
`BACKEND_DIPLOMAIL_WORKER_INTERVAL` (default `2s`). Each tick:
|
||||
|
||||
1. Picks one distinct `(message_id, recipient_preferred_language)`
|
||||
pair from `diplomail_recipients` where `available_at IS NULL`
|
||||
and `next_translation_attempt_at` is unset or due.
|
||||
2. Loads the source message, checks the translation cache.
|
||||
3. On cache hit → marks every pending recipient of the pair
|
||||
delivered and emits push.
|
||||
4. On cache miss → asks the configured `Translator`:
|
||||
- success → caches the translation, marks delivered, push;
|
||||
- HTTP 400 (unsupported pair) → marks delivered without a
|
||||
translation (fallback to original);
|
||||
- other failure → bumps `translation_attempts`, schedules the
|
||||
retry via `next_translation_attempt_at`, leaves pending.
|
||||
5. After `BACKEND_DIPLOMAIL_TRANSLATOR_MAX_ATTEMPTS` (default `5`)
|
||||
the worker falls back to delivering the original body so a
|
||||
prolonged LibreTranslate outage does not strand messages.
|
||||
|
||||
Retry backoff is exponential `1s → 2s → 4s → 8s → 16s` (capped at
|
||||
60s) per pair. Operators monitor the LibreTranslate dependency
|
||||
through standard OpenTelemetry export — translation outcomes
|
||||
surface in `diplomail.worker` logs at Info / Warn levels;
|
||||
Grafana / Prometheus dashboards live outside this package.
|
||||
|
||||
## Push integration
|
||||
|
||||
Every successful send emits a `diplomail.message.received` push
|
||||
|
||||
Reference in New Issue
Block a user