developer/scrabble-game
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

feat(deploy): Grafana infra alerts + blackbox cert probe + admin-alert wiring

Browse Source 
Grafana: GF_SMTP from the shared relay (STARTTLS host:port) + alerting provisioning
(contact point → SERVICE_EMAIL, route-all policy, and rules for scrape-target down,
gateway internal-error rate + p99 latency, host mem/disk/cpu, postgres connections, and
TLS cert < 20 days). All rules noDataState=OK so an absent metric never false-alerts.
blackbox_exporter probes the edge caddy's TLS (probe_ssl_earliest_cert_expiry) — effective
on prod (caddy terminates TLS; contour caddy is HTTP-only so the metric is absent).
Wires the new env through compose (backend admin From/To, Grafana SMTP), ci.yaml (TEST_),
prod-deploy (PROD_ + env.sh), .env.example and the README var table.
This commit is contained in:
Ilia Denisov
2026-07-03 15:08:09 +02:00
parent 8d2cd97e17
commit 55f6176538
10 changed files with 346 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitea/workflows/ci.yaml
+9
View File
@@ -340,6 +340,15 @@ jobs:
SMTP_RELAY_PORT: ${{ vars.TEST_SMTP_RELAY_PORT }}
SMTP_RELAY_TLS: ${{ vars.TEST_SMTP_RELAY_TLS }}
SMTP_RELAY_FROM: ${{ vars.TEST_SMTP_RELAY_FROM }}
# Operator alerts: backend admin emails (new feedback / complaints) + Grafana
# infra alerts. Distinct senders + recipients; Grafana uses the relay's STARTTLS
# host:port. Empty leaves the alert worker off and Grafana SMTP disabled.
SMTP_RELAY_ADMIN_FROM: ${{ vars.TEST_SMTP_RELAY_ADMIN_FROM }}
ADMIN_EMAIL: ${{ vars.TEST_ADMIN_EMAIL }}
SMTP_RELAY_SERVICE_FROM: ${{ vars.TEST_SMTP_RELAY_SERVICE_FROM }}
SERVICE_EMAIL: ${{ vars.TEST_SERVICE_EMAIL }}
GRAFANA_SMTP_HOST: ${{ vars.TEST_GRAFANA_SMTP_HOST }}
GF_SMTP_ENABLED: ${{ vars.TEST_GF_SMTP_ENABLED }}
# Canonical public origin for links in the email (this contour's URL);
# required by the backend whenever SMTP_RELAY_HOST is set.
PUBLIC_BASE_URL: ${{ vars.TEST_PUBLIC_BASE_URL }}
.gitea/workflows/prod-deploy.yaml
+14
View File
@@ -93,6 +93,14 @@ jobs:
SMTP_RELAY_PORT: ${{ vars.PROD_SMTP_RELAY_PORT }}
SMTP_RELAY_TLS: ${{ vars.PROD_SMTP_RELAY_TLS }}
SMTP_RELAY_FROM: ${{ vars.PROD_SMTP_RELAY_FROM }}
# Operator alerts: backend admin emails + Grafana infra alerts (distinct senders +
# recipients; Grafana uses the relay's STARTTLS host:port).
SMTP_RELAY_ADMIN_FROM: ${{ vars.PROD_SMTP_RELAY_ADMIN_FROM }}
ADMIN_EMAIL: ${{ vars.PROD_ADMIN_EMAIL }}
SMTP_RELAY_SERVICE_FROM: ${{ vars.PROD_SMTP_RELAY_SERVICE_FROM }}
SERVICE_EMAIL: ${{ vars.PROD_SERVICE_EMAIL }}
GRAFANA_SMTP_HOST: ${{ vars.PROD_GRAFANA_SMTP_HOST }}
GF_SMTP_ENABLED: ${{ vars.PROD_GF_SMTP_ENABLED }}
PUBLIC_BASE_URL: ${{ vars.PROD_PUBLIC_BASE_URL }}
PROD_BOTLINK_CA: ${{ secrets.PROD_BOTLINK_CA }}
PROD_BOTLINK_GATEWAY_CERT: ${{ secrets.PROD_BOTLINK_GATEWAY_CERT }}
@@ -156,6 +164,12 @@ jobs:
export SMTP_RELAY_USER='$SMTP_RELAY_USER'
export SMTP_RELAY_PASS='$SMTP_RELAY_PASS'
export SMTP_RELAY_FROM='$SMTP_RELAY_FROM'
export SMTP_RELAY_ADMIN_FROM='$SMTP_RELAY_ADMIN_FROM'
export ADMIN_EMAIL='$ADMIN_EMAIL'
export SMTP_RELAY_SERVICE_FROM='$SMTP_RELAY_SERVICE_FROM'
export SERVICE_EMAIL='$SERVICE_EMAIL'
export GRAFANA_SMTP_HOST='$GRAFANA_SMTP_HOST'
export GF_SMTP_ENABLED='$GF_SMTP_ENABLED'
export PUBLIC_BASE_URL='$PUBLIC_BASE_URL'
export GATEWAY_ABUSE_BAN_ENABLED='true'
EOF