feat(deploy): Grafana infra alerts + blackbox cert probe + admin-alert wiring

Grafana: GF_SMTP from the shared relay (STARTTLS host:port) + alerting provisioning
(contact point → SERVICE_EMAIL, route-all policy, and rules for scrape-target down,
gateway internal-error rate + p99 latency, host mem/disk/cpu, postgres connections, and
TLS cert < 20 days). All rules noDataState=OK so an absent metric never false-alerts.
blackbox_exporter probes the edge caddy's TLS (probe_ssl_earliest_cert_expiry) — effective
on prod (caddy terminates TLS; contour caddy is HTTP-only so the metric is absent).
Wires the new env through compose (backend admin From/To, Grafana SMTP), ci.yaml (TEST_),
prod-deploy (PROD_ + env.sh), .env.example and the README var table.
This commit is contained in:
Ilia Denisov
2026-07-03 15:08:09 +02:00
parent 8d2cd97e17
commit 55f6176538
10 changed files with 346 additions and 1 deletions
@@ -0,0 +1,10 @@
# Notification policy: every alert routes to the operator email, grouped so a burst is one
# message, with a 4-hour re-notify while still firing.
apiVersion: 1
policies:
- orgId: 1
receiver: ops-email
group_by: ['alertname']
group_wait: 30s
group_interval: 5m
repeat_interval: 4h