feat(edge): community IP blocklist (Spamhaus DROP) at the gateway
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 24s
CI / ui (pull_request) Successful in 1m11s
CI / conformance (pull_request) Successful in 10s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m49s
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 24s
CI / ui (pull_request) Successful in 1m11s
CI / conformance (pull_request) Successful in 10s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m49s
Refuse a client whose IP is in a curated CIDR feed with 403 in the same
abuseGuard, before the fail2ban ban. Prod-only (keys by real client IP), off by
default.
- ratelimit.Blocklist: a sorted-range IPv4 matcher (binary search) with an
allowlist checked first; ParseDROP reads the feed; ApplyRefresh keeps the
last-good feed on a transient fetch failure and drops it fail-open once stale
(better to under-block than block a legitimate client on a frozen feed). A
separate static CIDR set, not the per-IP fail2ban store.
- gateway: a refresher goroutine re-fetches every few hours (bounded fetch + size
cap); config GATEWAY_BLOCKLIST_{ENABLED,URL,ALLOW,REFRESH,MAX_STALENESS}.
IPv6 is not matched (a v6 client is still covered by fail2ban / the honeypot).
- observability: gateway_blocklist_blocked_total + entries/age gauges; a Grafana
alert warns before the feed is dropped; service-overview panels.
- deploy: compose env + write-prod-env.sh + prod-deploy/rollback wiring (opt-in
via PROD_ vars).
- docs (ARCHITECTURE, deploy/README); unit tests (match, allowlist, IPv6 skip,
parser, fault-tolerance) + a 403 abuse-guard integration test.
This commit is contained in:
@@ -115,6 +115,11 @@ jobs:
|
|||||||
# Planted honeytoken bearer: presenting it earns a 24h IP ban + a high-severity alarm.
|
# Planted honeytoken bearer: presenting it earns a 24h IP ban + a high-severity alarm.
|
||||||
# Per-contour secret; empty = trap off. Rendered by deploy/write-prod-env.sh.
|
# Per-contour secret; empty = trap off. Rendered by deploy/write-prod-env.sh.
|
||||||
GATEWAY_HONEYTOKEN: ${{ secrets.PROD_GATEWAY_HONEYTOKEN }}
|
GATEWAY_HONEYTOKEN: ${{ secrets.PROD_GATEWAY_HONEYTOKEN }}
|
||||||
|
# Community IP blocklist (Spamhaus DROP): opt-in. Set _ENABLED=true + _URL (the feed) once
|
||||||
|
# verified; _ALLOW is a comma-separated never-block set (own infra). Empty ⇒ off.
|
||||||
|
GATEWAY_BLOCKLIST_ENABLED: ${{ vars.PROD_GATEWAY_BLOCKLIST_ENABLED }}
|
||||||
|
GATEWAY_BLOCKLIST_URL: ${{ vars.PROD_GATEWAY_BLOCKLIST_URL }}
|
||||||
|
GATEWAY_BLOCKLIST_ALLOW: ${{ vars.PROD_GATEWAY_BLOCKLIST_ALLOW }}
|
||||||
# Signs the finished-game export download URLs (backend BACKEND_EXPORT_SIGN_KEY).
|
# Signs the finished-game export download URLs (backend BACKEND_EXPORT_SIGN_KEY).
|
||||||
EXPORT_SIGN_KEY: ${{ secrets.PROD_EXPORT_SIGN_KEY }}
|
EXPORT_SIGN_KEY: ${{ secrets.PROD_EXPORT_SIGN_KEY }}
|
||||||
# Transactional email via the shared Selectel relay (confirm-codes): one account for
|
# Transactional email via the shared Selectel relay (confirm-codes): one account for
|
||||||
|
|||||||
@@ -70,6 +70,10 @@ jobs:
|
|||||||
VITE_VK_APP_ID: ${{ vars.VITE_VK_APP_ID }}
|
VITE_VK_APP_ID: ${{ vars.VITE_VK_APP_ID }}
|
||||||
GATEWAY_VK_ID_CLIENT_SECRET: ${{ secrets.GATEWAY_VK_ID_CLIENT_SECRET }}
|
GATEWAY_VK_ID_CLIENT_SECRET: ${{ secrets.GATEWAY_VK_ID_CLIENT_SECRET }}
|
||||||
GATEWAY_HONEYTOKEN: ${{ secrets.PROD_GATEWAY_HONEYTOKEN }}
|
GATEWAY_HONEYTOKEN: ${{ secrets.PROD_GATEWAY_HONEYTOKEN }}
|
||||||
|
# Community IP blocklist — rendered on rollback too so a rollback keeps the same edge policy.
|
||||||
|
GATEWAY_BLOCKLIST_ENABLED: ${{ vars.PROD_GATEWAY_BLOCKLIST_ENABLED }}
|
||||||
|
GATEWAY_BLOCKLIST_URL: ${{ vars.PROD_GATEWAY_BLOCKLIST_URL }}
|
||||||
|
GATEWAY_BLOCKLIST_ALLOW: ${{ vars.PROD_GATEWAY_BLOCKLIST_ALLOW }}
|
||||||
EXPORT_SIGN_KEY: ${{ secrets.PROD_EXPORT_SIGN_KEY }}
|
EXPORT_SIGN_KEY: ${{ secrets.PROD_EXPORT_SIGN_KEY }}
|
||||||
SMTP_RELAY_USER: ${{ secrets.SMTP_RELAY_USER }}
|
SMTP_RELAY_USER: ${{ secrets.SMTP_RELAY_USER }}
|
||||||
SMTP_RELAY_PASS: ${{ secrets.SMTP_RELAY_PASS }}
|
SMTP_RELAY_PASS: ${{ secrets.SMTP_RELAY_PASS }}
|
||||||
|
|||||||
@@ -115,6 +115,9 @@ without it Docker's resolver handles `otelcol`, `gateway` and `api.telegram.org`
|
|||||||
| `GATEWAY_VK_APP_SECRET` | secret (shared) | _(empty)_ | The VK Mini App's protected key (`client_secret`): the gateway verifies the launch-parameter signature in-process under it (offline HMAC, no VK API call). Empty disables the VK auth path (`auth.vk`). One VK Mini App for all contours. |
|
| `GATEWAY_VK_APP_SECRET` | secret (shared) | _(empty)_ | The VK Mini App's protected key (`client_secret`): the gateway verifies the launch-parameter signature in-process under it (offline HMAC, no VK API call). Empty disables the VK auth path (`auth.vk`). One VK Mini App for all contours. |
|
||||||
| `GATEWAY_VK_ID_CLIENT_SECRET` | secret (shared) | _(empty)_ | The VK ID "Web" app's protected key (`client_secret`) for the gateway's server-side confidential code exchange. A SEPARATE VK app from `GATEWAY_VK_APP_SECRET` (the Mini App). One VK ID "Web" app for all contours. |
|
| `GATEWAY_VK_ID_CLIENT_SECRET` | secret (shared) | _(empty)_ | The VK ID "Web" app's protected key (`client_secret`) for the gateway's server-side confidential code exchange. A SEPARATE VK app from `GATEWAY_VK_APP_SECRET` (the Mini App). One VK ID "Web" app for all contours. |
|
||||||
| `GATEWAY_HONEYTOKEN` | secret | _(empty)_ | Planted honeytoken bearer value: presenting it earns a 24h IP ban + a high-severity alarm where the IP ban is on (prod), or logs + a `gateway_abuse_banned_total{reason="honeytoken"}` metric (test, ban off). Plant the value somewhere an attacker would find it; empty disables the trap. Per-contour `TEST_`/`PROD_GATEWAY_HONEYTOKEN`. |
|
| `GATEWAY_HONEYTOKEN` | secret | _(empty)_ | Planted honeytoken bearer value: presenting it earns a 24h IP ban + a high-severity alarm where the IP ban is on (prod), or logs + a `gateway_abuse_banned_total{reason="honeytoken"}` metric (test, ban off). Plant the value somewhere an attacker would find it; empty disables the trap. Per-contour `TEST_`/`PROD_GATEWAY_HONEYTOKEN`. |
|
||||||
|
| `GATEWAY_BLOCKLIST_ENABLED` | variable | `false` | Enable the community IP blocklist at the edge (prod-only, same real-client-IP reason as the ban). Requires `GATEWAY_BLOCKLIST_URL`. Opt-in via `PROD_GATEWAY_BLOCKLIST_ENABLED` after verifying the feed. |
|
||||||
|
| `GATEWAY_BLOCKLIST_URL` | variable | _(empty)_ | The curated CIDR feed to fetch (Spamhaus DROP). Required when enabled; refreshed every few hours and dropped fail-open once stale (48h). `PROD_GATEWAY_BLOCKLIST_URL`. |
|
||||||
|
| `GATEWAY_BLOCKLIST_ALLOW` | variable | _(empty)_ | Comma-separated never-block set (CIDRs / bare IPs — own infra, monitoring) the feed can never block. `PROD_GATEWAY_BLOCKLIST_ALLOW`. |
|
||||||
| `VITE_GATEWAY_URL` | variable | _(empty)_ | UI build-arg: gateway origin; empty = same-origin (the usual single-origin deploy). |
|
| `VITE_GATEWAY_URL` | variable | _(empty)_ | UI build-arg: gateway origin; empty = same-origin (the usual single-origin deploy). |
|
||||||
| `SMTP_RELAY_HOST` | variable (shared) | _(empty)_ | Selectel SMTP relay host for confirm-code email. Empty leaves the backend on the log mailer (email disabled) — the contour still boots. One relay for every contour (limit 100 msgs / 5 min). |
|
| `SMTP_RELAY_HOST` | variable (shared) | _(empty)_ | Selectel SMTP relay host for confirm-code email. Empty leaves the backend on the log mailer (email disabled) — the contour still boots. One relay for every contour (limit 100 msgs / 5 min). |
|
||||||
| `SMTP_RELAY_PORT` | variable (shared) | `465` | Relay port. No client certificate is needed (the server cert is validated against the system roots). |
|
| `SMTP_RELAY_PORT` | variable (shared) | `465` | Relay port. No client certificate is needed (the server cert is validated against the system roots). |
|
||||||
|
|||||||
@@ -254,6 +254,12 @@ services:
|
|||||||
# secret; empty (unset secret) leaves the trap off.
|
# secret; empty (unset secret) leaves the trap off.
|
||||||
GATEWAY_ABUSE_BAN_ENABLED: ${GATEWAY_ABUSE_BAN_ENABLED:-false}
|
GATEWAY_ABUSE_BAN_ENABLED: ${GATEWAY_ABUSE_BAN_ENABLED:-false}
|
||||||
GATEWAY_HONEYTOKEN: ${GATEWAY_HONEYTOKEN:-}
|
GATEWAY_HONEYTOKEN: ${GATEWAY_HONEYTOKEN:-}
|
||||||
|
# Community IP blocklist (Spamhaus DROP): prod-only, off unless the real client IP is visible
|
||||||
|
# (the shared-NAT test contour would self-block). Enabled + fed the feed URL + allowlist by the
|
||||||
|
# prod deploy (write-prod-env.sh); the refresh/staleness windows use the built-in defaults.
|
||||||
|
GATEWAY_BLOCKLIST_ENABLED: ${GATEWAY_BLOCKLIST_ENABLED:-false}
|
||||||
|
GATEWAY_BLOCKLIST_URL: ${GATEWAY_BLOCKLIST_URL:-}
|
||||||
|
GATEWAY_BLOCKLIST_ALLOW: ${GATEWAY_BLOCKLIST_ALLOW:-}
|
||||||
GATEWAY_LOG_LEVEL: ${LOG_LEVEL:-info}
|
GATEWAY_LOG_LEVEL: ${LOG_LEVEL:-info}
|
||||||
GATEWAY_SERVICE_NAME: scrabble-gateway
|
GATEWAY_SERVICE_NAME: scrabble-gateway
|
||||||
GATEWAY_OTEL_TRACES_EXPORTER: otlp
|
GATEWAY_OTEL_TRACES_EXPORTER: otlp
|
||||||
|
|||||||
@@ -53,6 +53,31 @@
|
|||||||
"fieldConfig": { "defaults": { "unit": "bytes" }, "overrides": [] },
|
"fieldConfig": { "defaults": { "unit": "bytes" }, "overrides": [] },
|
||||||
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||||
"targets": [{ "refId": "A", "expr": "sum(go_memory_used) by (service_name)", "legendFormat": "{{service_name}}" }]
|
"targets": [{ "refId": "A", "expr": "sum(go_memory_used) by (service_name)", "legendFormat": "{{service_name}}" }]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "stat",
|
||||||
|
"title": "Blocklist entries",
|
||||||
|
"description": "CIDR ranges in the active community IP blocklist feed (0 when disabled or not yet fetched).",
|
||||||
|
"gridPos": { "h": 5, "w": 6, "x": 0, "y": 13 },
|
||||||
|
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||||
|
"targets": [{ "refId": "A", "expr": "max(gateway_blocklist_entries)" }]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "stat",
|
||||||
|
"title": "Blocklist feed age",
|
||||||
|
"description": "Seconds since the community IP blocklist feed was last successfully fetched. Grows if the fetch is failing; the feed is dropped fail-open once stale.",
|
||||||
|
"gridPos": { "h": 5, "w": 6, "x": 6, "y": 13 },
|
||||||
|
"fieldConfig": { "defaults": { "unit": "s" }, "overrides": [] },
|
||||||
|
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||||
|
"targets": [{ "refId": "A", "expr": "max(gateway_blocklist_age_seconds)" }]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "timeseries",
|
||||||
|
"title": "Blocklist blocks/s",
|
||||||
|
"description": "Requests refused at the edge by the community IP blocklist (gateway_blocklist_blocked_total).",
|
||||||
|
"gridPos": { "h": 8, "w": 12, "x": 12, "y": 13 },
|
||||||
|
"datasource": { "type": "prometheus", "uid": "prometheus" },
|
||||||
|
"targets": [{ "refId": "A", "expr": "sum(rate(gateway_blocklist_blocked_total[5m]))" }]
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -108,6 +108,32 @@ groups:
|
|||||||
labels: { severity: critical }
|
labels: { severity: critical }
|
||||||
annotations: { summary: 'Edge TLS cert has under 20 days left — Caddy ACME renewal may have failed.' }
|
annotations: { summary: 'Edge TLS cert has under 20 days left — Caddy ACME renewal may have failed.' }
|
||||||
|
|
||||||
|
# The community IP blocklist feed has not refreshed in over a day (the gateway re-fetches every
|
||||||
|
# few hours). Absent/NaN-safe: the age gauge appears only once a feed has loaded, so a disabled
|
||||||
|
# or never-fetched blocklist stays quiet. The feed itself is dropped (fail-open) at its
|
||||||
|
# max-staleness window; this warns well before that so the operator can fix the fetch.
|
||||||
|
- uid: blocklist_stale
|
||||||
|
title: IP blocklist feed not refreshing
|
||||||
|
condition: C
|
||||||
|
for: 15m
|
||||||
|
noDataState: OK
|
||||||
|
execErrState: OK
|
||||||
|
data:
|
||||||
|
- refId: A
|
||||||
|
relativeTimeRange: { from: 300, to: 0 }
|
||||||
|
datasourceUid: prometheus
|
||||||
|
model: { refId: A, expr: gateway_blocklist_age_seconds, instant: true }
|
||||||
|
- refId: C
|
||||||
|
datasourceUid: __expr__
|
||||||
|
model:
|
||||||
|
refId: C
|
||||||
|
type: threshold
|
||||||
|
expression: A
|
||||||
|
conditions:
|
||||||
|
- evaluator: { type: gt, params: [86400] }
|
||||||
|
labels: { severity: warning }
|
||||||
|
annotations: { summary: 'The community IP blocklist feed has not refreshed in over 24h — the fetch is failing; it will be dropped (fail-open) once stale. Check the gateway blocklist refresher logs and the feed URL.' }
|
||||||
|
|
||||||
- orgId: 1
|
- orgId: 1
|
||||||
name: scrabble-host
|
name: scrabble-host
|
||||||
folder: Alerts
|
folder: Alerts
|
||||||
|
|||||||
@@ -77,6 +77,11 @@ export GF_SMTP_ENABLED='$GF_SMTP_ENABLED'
|
|||||||
export PUBLIC_BASE_URL='$PUBLIC_BASE_URL'
|
export PUBLIC_BASE_URL='$PUBLIC_BASE_URL'
|
||||||
export GATEWAY_HONEYTOKEN='$GATEWAY_HONEYTOKEN'
|
export GATEWAY_HONEYTOKEN='$GATEWAY_HONEYTOKEN'
|
||||||
export GATEWAY_ABUSE_BAN_ENABLED='true'
|
export GATEWAY_ABUSE_BAN_ENABLED='true'
|
||||||
|
# Community IP blocklist (Spamhaus DROP): opt-in — the operator enables it and sets the feed URL +
|
||||||
|
# allowlist via PROD_GATEWAY_BLOCKLIST_* vars once the feed is verified. Unset ⇒ off (safe).
|
||||||
|
export GATEWAY_BLOCKLIST_ENABLED='${GATEWAY_BLOCKLIST_ENABLED:-false}'
|
||||||
|
export GATEWAY_BLOCKLIST_URL='$GATEWAY_BLOCKLIST_URL'
|
||||||
|
export GATEWAY_BLOCKLIST_ALLOW='$GATEWAY_BLOCKLIST_ALLOW'
|
||||||
# Continuous WAL archiving (pgBackRest -> S3) for point-in-time recovery. The artifact
|
# Continuous WAL archiving (pgBackRest -> S3) for point-in-time recovery. The artifact
|
||||||
# ships disarmed: PGBACKREST_ARCHIVE_MODE defaults off, so archiving stays inert until the
|
# ships disarmed: PGBACKREST_ARCHIVE_MODE defaults off, so archiving stays inert until the
|
||||||
# operator sets the S3 repository values + secrets, creates the stanza and flips the switch
|
# operator sets the S3 repository values + secrets, creates the stanza and flips the switch
|
||||||
|
|||||||
@@ -1206,6 +1206,18 @@ link — misses the event; while an add-email confirmation is pending the client
|
|||||||
(`POST /api/v1/internal/bans/sync`, network-trusted like the rejection report) and applies
|
(`POST /api/v1/internal/bans/sync`, network-trusted like the rejection report) and applies
|
||||||
the operator unbans the response returns, so a manual unban takes effect within the sync
|
the operator unbans the response returns, so a manual unban takes effect within the sync
|
||||||
interval.
|
interval.
|
||||||
|
- **Community IP blocklist (prod-only):** with `GATEWAY_BLOCKLIST_ENABLED` set, the gateway also
|
||||||
|
refuses a client whose IP is in a curated CIDR feed (Spamhaus DROP, `GATEWAY_BLOCKLIST_URL`) with
|
||||||
|
**403** in the same `abuseGuard`, before the fail2ban list. A background refresher re-fetches the
|
||||||
|
feed every few hours (bounded fetch + size cap) into a sorted-range matcher (`ratelimit.Blocklist`,
|
||||||
|
binary search, IPv4 only — an IPv6 client is never blocked here). It is **fault-tolerant and
|
||||||
|
fail-open**: a failed fetch keeps the last-good feed, and once the feed is older than
|
||||||
|
`GATEWAY_BLOCKLIST_MAX_STALENESS` it is **dropped** rather than block a legitimate client on a
|
||||||
|
frozen list; a `GATEWAY_BLOCKLIST_ALLOW` allowlist (own infra, monitoring) is never blocked. Off by
|
||||||
|
default and prod-only for the same real-client-IP reason as the ban. It is a **separate** static
|
||||||
|
CIDR set, not the per-IP fail2ban store (a bulk CIDR feed cannot expand into per-IP entries).
|
||||||
|
Observability: `gateway_blocklist_blocked_total`, the `gateway_blocklist_entries` size gauge and the
|
||||||
|
`gateway_blocklist_age_seconds` staleness gauge (a Grafana alert warns before the feed is dropped).
|
||||||
- Unauthenticated `GET /healthz` (liveness) and `GET /readyz` (readiness — the
|
- Unauthenticated `GET /healthz` (liveness) and `GET /readyz` (readiness — the
|
||||||
database answers a bounded ping and the session cache is warmed).
|
database answers a bounded ping and the session cache is warmed).
|
||||||
- The backend serves a **second listener** — a gRPC server
|
- The backend serves a **second listener** — a gRPC server
|
||||||
|
|||||||
@@ -0,0 +1,86 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net"
|
||||||
|
"net/http"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"go.uber.org/zap"
|
||||||
|
|
||||||
|
"scrabble/gateway/internal/config"
|
||||||
|
"scrabble/gateway/internal/ratelimit"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
// blocklistFetchTimeout bounds one feed fetch.
|
||||||
|
blocklistFetchTimeout = 30 * time.Second
|
||||||
|
// maxBlocklistBytes caps the fetched feed (Spamhaus DROP is well under 1 MiB; the cap stops a
|
||||||
|
// hostile or misconfigured URL from streaming unbounded data into memory).
|
||||||
|
maxBlocklistBytes = 8 << 20
|
||||||
|
)
|
||||||
|
|
||||||
|
// parseAllowlist parses the never-block entries (CIDRs or bare IPs, a bare IP read as a /32) into
|
||||||
|
// networks. A malformed entry is a fatal config error — the operator must fix it, not silently lose
|
||||||
|
// a protected range.
|
||||||
|
func parseAllowlist(entries []string) ([]*net.IPNet, error) {
|
||||||
|
out := make([]*net.IPNet, 0, len(entries))
|
||||||
|
for _, e := range entries {
|
||||||
|
s := strings.TrimSpace(e)
|
||||||
|
if s == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !strings.Contains(s, "/") {
|
||||||
|
s += "/32"
|
||||||
|
}
|
||||||
|
_, n, err := net.ParseCIDR(s)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("gateway: GATEWAY_BLOCKLIST_ALLOW: %q: %w", e, err)
|
||||||
|
}
|
||||||
|
out = append(out, n)
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// runBlocklistRefresher keeps the community IP blocklist feed current: it fetches immediately, then
|
||||||
|
// every cfg.Refresh, and applies each outcome through ratelimit.ApplyRefresh (keep-last-good on a
|
||||||
|
// transient failure; drop the feed once it goes stale, fail-open). It returns when ctx is cancelled.
|
||||||
|
func runBlocklistRefresher(ctx context.Context, bl *ratelimit.Blocklist, cfg config.BlocklistConfig, log *zap.Logger) {
|
||||||
|
client := &http.Client{Timeout: blocklistFetchTimeout}
|
||||||
|
for {
|
||||||
|
cidrs, err := fetchBlocklist(ctx, client, cfg.URL)
|
||||||
|
switch ratelimit.ApplyRefresh(bl, cidrs, err, time.Now(), cfg.MaxStaleness) {
|
||||||
|
case ratelimit.RefreshUpdated:
|
||||||
|
log.Info("blocklist refreshed", zap.String("url", cfg.URL), zap.Int("entries", bl.Len()))
|
||||||
|
case ratelimit.RefreshKept:
|
||||||
|
log.Warn("blocklist refresh failed; keeping the last-good feed", zap.Error(err))
|
||||||
|
case ratelimit.RefreshDropped:
|
||||||
|
log.Warn("blocklist refresh failed and the feed is stale; dropped it (fail-open)", zap.Error(err))
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
case <-time.After(cfg.Refresh):
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// fetchBlocklist GETs the feed and parses it, bounded by the fetch timeout and the size cap.
|
||||||
|
func fetchBlocklist(ctx context.Context, client *http.Client, url string) ([]*net.IPNet, error) {
|
||||||
|
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
resp, err := client.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer func() { _ = resp.Body.Close() }()
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
return nil, fmt.Errorf("gateway: blocklist fetch %s: status %d", url, resp.StatusCode)
|
||||||
|
}
|
||||||
|
return ratelimit.ParseDROP(io.LimitReader(resp.Body, maxBlocklistBytes))
|
||||||
|
}
|
||||||
@@ -129,6 +129,11 @@ func run(ctx context.Context, cfg config.Config, logger *zap.Logger) error {
|
|||||||
Window: cfg.Abuse.BanWindow,
|
Window: cfg.Abuse.BanWindow,
|
||||||
Duration: cfg.Abuse.BanDuration,
|
Duration: cfg.Abuse.BanDuration,
|
||||||
})
|
})
|
||||||
|
allow, err := parseAllowlist(cfg.Blocklist.Allow)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
blocklist := ratelimit.NewBlocklist(cfg.Blocklist.Enabled, allow)
|
||||||
hub := push.NewHub(0)
|
hub := push.NewHub(0)
|
||||||
|
|
||||||
var validator transcode.TelegramValidator
|
var validator transcode.TelegramValidator
|
||||||
@@ -222,6 +227,7 @@ func run(ctx context.Context, cfg config.Config, logger *zap.Logger) error {
|
|||||||
Limiter: limiter,
|
Limiter: limiter,
|
||||||
Tracker: tracker,
|
Tracker: tracker,
|
||||||
Banlist: banlist,
|
Banlist: banlist,
|
||||||
|
Blocklist: blocklist,
|
||||||
Honeytoken: cfg.Abuse.Honeytoken,
|
Honeytoken: cfg.Abuse.Honeytoken,
|
||||||
VKAppSecret: cfg.VKAppSecret,
|
VKAppSecret: cfg.VKAppSecret,
|
||||||
Hub: hub,
|
Hub: hub,
|
||||||
@@ -243,6 +249,10 @@ func run(ctx context.Context, cfg config.Config, logger *zap.Logger) error {
|
|||||||
if cfg.Abuse.BanEnabled {
|
if cfg.Abuse.BanEnabled {
|
||||||
go runBanSync(ctx, banlist, backend, logger)
|
go runBanSync(ctx, banlist, backend, logger)
|
||||||
}
|
}
|
||||||
|
// When the edge blocklist is enabled (prod), keep the community feed refreshed.
|
||||||
|
if cfg.Blocklist.Enabled {
|
||||||
|
go runBlocklistRefresher(ctx, blocklist, cfg.Blocklist, logger)
|
||||||
|
}
|
||||||
|
|
||||||
public := &http.Server{Addr: cfg.HTTPAddr, Handler: edge.HTTPHandler(), ReadHeaderTimeout: readHeaderTimeout}
|
public := &http.Server{Addr: cfg.HTTPAddr, Handler: edge.HTTPHandler(), ReadHeaderTimeout: readHeaderTimeout}
|
||||||
servers := []*namedServer{{name: "public", srv: public}}
|
servers := []*namedServer{{name: "public", srv: public}}
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ import (
|
|||||||
"fmt"
|
"fmt"
|
||||||
"os"
|
"os"
|
||||||
"strconv"
|
"strconv"
|
||||||
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
pkgtel "scrabble/pkg/telemetry"
|
pkgtel "scrabble/pkg/telemetry"
|
||||||
@@ -57,6 +58,8 @@ type Config struct {
|
|||||||
RateLimit RateLimitConfig
|
RateLimit RateLimitConfig
|
||||||
// Abuse configures the temporary IP ban and the honeytoken (prod-only).
|
// Abuse configures the temporary IP ban and the honeytoken (prod-only).
|
||||||
Abuse AbuseConfig
|
Abuse AbuseConfig
|
||||||
|
// Blocklist configures the community IP blocklist (Spamhaus DROP) enforced at the edge (prod-only).
|
||||||
|
Blocklist BlocklistConfig
|
||||||
// Telemetry configures the OpenTelemetry providers (shared bootstrap).
|
// Telemetry configures the OpenTelemetry providers (shared bootstrap).
|
||||||
Telemetry pkgtel.Config
|
Telemetry pkgtel.Config
|
||||||
}
|
}
|
||||||
@@ -166,6 +169,42 @@ func DefaultAbuse() AbuseConfig {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// BlocklistConfig configures the community IP blocklist (a curated CIDR feed such as Spamhaus DROP)
|
||||||
|
// enforced at the edge alongside the fail2ban ban. Disabled by default and prod-only, for the same
|
||||||
|
// reason as the ban: it is only safe where the real client IP is visible (not behind the shared-NAT
|
||||||
|
// test contour). Only IPv4 is matched.
|
||||||
|
type BlocklistConfig struct {
|
||||||
|
// Enabled turns edge blocklisting on. Off by default (prod-only).
|
||||||
|
Enabled bool
|
||||||
|
// URL is the CIDR feed to fetch (e.g. the Spamhaus DROP list). Required when Enabled; the
|
||||||
|
// operator sets it explicitly so no feed URL is assumed.
|
||||||
|
URL string
|
||||||
|
// Refresh is how often the feed is re-fetched.
|
||||||
|
Refresh time.Duration
|
||||||
|
// MaxStaleness is how long a stale feed (no successful refresh) is tolerated before it is dropped
|
||||||
|
// (fail-open — a legitimate client is never blocked on a frozen feed).
|
||||||
|
MaxStaleness time.Duration
|
||||||
|
// Allow is the never-block set: CIDRs or bare IPs (own infrastructure, monitoring, known-good) that
|
||||||
|
// the feed can never block. Parsed at startup.
|
||||||
|
Allow []string
|
||||||
|
}
|
||||||
|
|
||||||
|
// Blocklist defaults; the feed URL has no default (the operator sets it).
|
||||||
|
const (
|
||||||
|
defaultBlocklistRefresh = 6 * time.Hour
|
||||||
|
defaultBlocklistMaxStaleness = 48 * time.Hour
|
||||||
|
)
|
||||||
|
|
||||||
|
// DefaultBlocklist returns the built-in blocklist settings: disabled (prod-only), no feed URL, and
|
||||||
|
// the agreed refresh / staleness windows.
|
||||||
|
func DefaultBlocklist() BlocklistConfig {
|
||||||
|
return BlocklistConfig{
|
||||||
|
Enabled: false,
|
||||||
|
Refresh: defaultBlocklistRefresh,
|
||||||
|
MaxStaleness: defaultBlocklistMaxStaleness,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// VKIDConfig holds the VK ID web-login credentials for the confidential
|
// VKIDConfig holds the VK ID web-login credentials for the confidential
|
||||||
// authorization-code exchange. AppID is the VK "Web" app's client id; ClientSecret is
|
// authorization-code exchange. AppID is the VK "Web" app's client id; ClientSecret is
|
||||||
// its protected key; RedirectURI must exactly match the trusted redirect URL registered
|
// its protected key; RedirectURI must exactly match the trusted redirect URL registered
|
||||||
@@ -203,6 +242,7 @@ func Load() (Config, error) {
|
|||||||
SessionCacheMax: defaultSessionCacheMax,
|
SessionCacheMax: defaultSessionCacheMax,
|
||||||
RateLimit: DefaultRateLimit(),
|
RateLimit: DefaultRateLimit(),
|
||||||
Abuse: DefaultAbuse(),
|
Abuse: DefaultAbuse(),
|
||||||
|
Blocklist: DefaultBlocklist(),
|
||||||
BotLink: BotLinkConfig{
|
BotLink: BotLinkConfig{
|
||||||
Addr: os.Getenv("GATEWAY_BOTLINK_ADDR"),
|
Addr: os.Getenv("GATEWAY_BOTLINK_ADDR"),
|
||||||
RelayAddr: os.Getenv("GATEWAY_BOTLINK_RELAY_ADDR"),
|
RelayAddr: os.Getenv("GATEWAY_BOTLINK_RELAY_ADDR"),
|
||||||
@@ -244,6 +284,17 @@ func Load() (Config, error) {
|
|||||||
if c.Abuse.BanDuration, err = envDuration("GATEWAY_ABUSE_BAN_DURATION", c.Abuse.BanDuration); err != nil {
|
if c.Abuse.BanDuration, err = envDuration("GATEWAY_ABUSE_BAN_DURATION", c.Abuse.BanDuration); err != nil {
|
||||||
return Config{}, err
|
return Config{}, err
|
||||||
}
|
}
|
||||||
|
if c.Blocklist.Enabled, err = envBool("GATEWAY_BLOCKLIST_ENABLED", c.Blocklist.Enabled); err != nil {
|
||||||
|
return Config{}, err
|
||||||
|
}
|
||||||
|
c.Blocklist.URL = os.Getenv("GATEWAY_BLOCKLIST_URL")
|
||||||
|
if c.Blocklist.Refresh, err = envDuration("GATEWAY_BLOCKLIST_REFRESH", c.Blocklist.Refresh); err != nil {
|
||||||
|
return Config{}, err
|
||||||
|
}
|
||||||
|
if c.Blocklist.MaxStaleness, err = envDuration("GATEWAY_BLOCKLIST_MAX_STALENESS", c.Blocklist.MaxStaleness); err != nil {
|
||||||
|
return Config{}, err
|
||||||
|
}
|
||||||
|
c.Blocklist.Allow = splitList(os.Getenv("GATEWAY_BLOCKLIST_ALLOW"))
|
||||||
if c.BotLink.SendTimeout, err = envDuration("GATEWAY_BOTLINK_SEND_TIMEOUT", defaultBotLinkSendTimeout); err != nil {
|
if c.BotLink.SendTimeout, err = envDuration("GATEWAY_BOTLINK_SEND_TIMEOUT", defaultBotLinkSendTimeout); err != nil {
|
||||||
return Config{}, err
|
return Config{}, err
|
||||||
}
|
}
|
||||||
@@ -284,6 +335,9 @@ func (c Config) validate() error {
|
|||||||
if c.Abuse.BanEnabled && (c.Abuse.BanThreshold <= 0 || c.Abuse.BanWindow <= 0 || c.Abuse.BanDuration <= 0) {
|
if c.Abuse.BanEnabled && (c.Abuse.BanThreshold <= 0 || c.Abuse.BanWindow <= 0 || c.Abuse.BanDuration <= 0) {
|
||||||
return fmt.Errorf("config: GATEWAY_ABUSE_BAN_THRESHOLD/_WINDOW/_DURATION must be positive when GATEWAY_ABUSE_BAN_ENABLED")
|
return fmt.Errorf("config: GATEWAY_ABUSE_BAN_THRESHOLD/_WINDOW/_DURATION must be positive when GATEWAY_ABUSE_BAN_ENABLED")
|
||||||
}
|
}
|
||||||
|
if c.Blocklist.Enabled && (c.Blocklist.URL == "" || c.Blocklist.Refresh <= 0 || c.Blocklist.MaxStaleness <= 0) {
|
||||||
|
return fmt.Errorf("config: GATEWAY_BLOCKLIST_URL must be set and _REFRESH/_MAX_STALENESS positive when GATEWAY_BLOCKLIST_ENABLED")
|
||||||
|
}
|
||||||
if c.BotLink.Addr != "" {
|
if c.BotLink.Addr != "" {
|
||||||
if c.BotLink.CertFile == "" || c.BotLink.KeyFile == "" || c.BotLink.CAFile == "" {
|
if c.BotLink.CertFile == "" || c.BotLink.KeyFile == "" || c.BotLink.CAFile == "" {
|
||||||
return fmt.Errorf("config: GATEWAY_BOTLINK_ADDR requires GATEWAY_BOTLINK_TLS_CERT, _KEY and _CA")
|
return fmt.Errorf("config: GATEWAY_BOTLINK_ADDR requires GATEWAY_BOTLINK_TLS_CERT, _KEY and _CA")
|
||||||
@@ -304,6 +358,21 @@ func envOr(key, fallback string) string {
|
|||||||
return fallback
|
return fallback
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// splitList splits a comma-separated environment value into trimmed, non-empty items.
|
||||||
|
func splitList(v string) []string {
|
||||||
|
if v == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
parts := strings.Split(v, ",")
|
||||||
|
out := make([]string, 0, len(parts))
|
||||||
|
for _, p := range parts {
|
||||||
|
if p = strings.TrimSpace(p); p != "" {
|
||||||
|
out = append(out, p)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
// envBool parses the environment variable named key as a bool, returning fallback
|
// envBool parses the environment variable named key as a bool, returning fallback
|
||||||
// when it is unset and an error when it is set but malformed.
|
// when it is unset and an error when it is set but malformed.
|
||||||
func envBool(key string, fallback bool) (bool, error) {
|
func envBool(key string, fallback bool) (bool, error) {
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ package connectsrv_test
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
"net"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"testing"
|
"testing"
|
||||||
@@ -24,7 +25,7 @@ const honeypotHeader = "X-Scrabble-Honeypot"
|
|||||||
|
|
||||||
// guardedEdge wires an edge with an explicit banlist and honeytoken over a fake
|
// guardedEdge wires an edge with an explicit banlist and honeytoken over a fake
|
||||||
// backend, returning the front URL, a Connect client and a cleanup func.
|
// backend, returning the front URL, a Connect client and a cleanup func.
|
||||||
func guardedEdge(t *testing.T, bl *ratelimit.Banlist, honeytoken string, limits config.RateLimitConfig, backendHandler http.HandlerFunc) (string, edgev1connect.GatewayClient, func()) {
|
func guardedEdge(t *testing.T, bl *ratelimit.Banlist, block *ratelimit.Blocklist, honeytoken string, limits config.RateLimitConfig, backendHandler http.HandlerFunc) (string, edgev1connect.GatewayClient, func()) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
backendSrv := httptest.NewServer(backendHandler)
|
backendSrv := httptest.NewServer(backendHandler)
|
||||||
backend, err := backendclient.New(backendSrv.URL, "localhost:9090", 2*time.Second)
|
backend, err := backendclient.New(backendSrv.URL, "localhost:9090", 2*time.Second)
|
||||||
@@ -36,6 +37,7 @@ func guardedEdge(t *testing.T, bl *ratelimit.Banlist, honeytoken string, limits
|
|||||||
Sessions: session.NewCache(backend, time.Minute, 100),
|
Sessions: session.NewCache(backend, time.Minute, 100),
|
||||||
Limiter: ratelimit.New(),
|
Limiter: ratelimit.New(),
|
||||||
Banlist: bl,
|
Banlist: bl,
|
||||||
|
Blocklist: block,
|
||||||
Honeytoken: honeytoken,
|
Honeytoken: honeytoken,
|
||||||
Hub: push.NewHub(0),
|
Hub: push.NewHub(0),
|
||||||
RateLimit: limits,
|
RateLimit: limits,
|
||||||
@@ -69,7 +71,7 @@ func enabledBanlist(threshold int) *ratelimit.Banlist {
|
|||||||
func TestAbuseGuardBlocksBannedIP(t *testing.T) {
|
func TestAbuseGuardBlocksBannedIP(t *testing.T) {
|
||||||
bl := enabledBanlist(100)
|
bl := enabledBanlist(100)
|
||||||
bl.BanNow("127.0.0.1", ratelimit.ReasonTripwire)
|
bl.BanNow("127.0.0.1", ratelimit.ReasonTripwire)
|
||||||
url, _, cleanup := guardedEdge(t, bl, "", config.DefaultRateLimit(), func(w http.ResponseWriter, r *http.Request) {})
|
url, _, cleanup := guardedEdge(t, bl, nil, "", config.DefaultRateLimit(), func(w http.ResponseWriter, r *http.Request) {})
|
||||||
defer cleanup()
|
defer cleanup()
|
||||||
|
|
||||||
resp, err := noRedirect().Get(url + "/")
|
resp, err := noRedirect().Get(url + "/")
|
||||||
@@ -86,7 +88,7 @@ func TestAbuseGuardBlocksBannedIP(t *testing.T) {
|
|||||||
// and bans the client IP, so the next request is blocked.
|
// and bans the client IP, so the next request is blocked.
|
||||||
func TestHoneypotHeaderTrips(t *testing.T) {
|
func TestHoneypotHeaderTrips(t *testing.T) {
|
||||||
bl := enabledBanlist(100)
|
bl := enabledBanlist(100)
|
||||||
url, _, cleanup := guardedEdge(t, bl, "", config.DefaultRateLimit(), func(w http.ResponseWriter, r *http.Request) {
|
url, _, cleanup := guardedEdge(t, bl, nil, "", config.DefaultRateLimit(), func(w http.ResponseWriter, r *http.Request) {
|
||||||
t.Error("backend must not be called for a honeypot hit")
|
t.Error("backend must not be called for a honeypot hit")
|
||||||
})
|
})
|
||||||
defer cleanup()
|
defer cleanup()
|
||||||
@@ -118,7 +120,7 @@ func TestHoneypotHeaderTrips(t *testing.T) {
|
|||||||
// banlist still 404s the decoy (detection/logging) but bans nothing.
|
// banlist still 404s the decoy (detection/logging) but bans nothing.
|
||||||
func TestHoneypotDetectsWithoutBanWhenDisabled(t *testing.T) {
|
func TestHoneypotDetectsWithoutBanWhenDisabled(t *testing.T) {
|
||||||
bl := ratelimit.NewBanlist(ratelimit.BanConfig{}) // disabled
|
bl := ratelimit.NewBanlist(ratelimit.BanConfig{}) // disabled
|
||||||
url, _, cleanup := guardedEdge(t, bl, "", config.DefaultRateLimit(), func(w http.ResponseWriter, r *http.Request) {})
|
url, _, cleanup := guardedEdge(t, bl, nil, "", config.DefaultRateLimit(), func(w http.ResponseWriter, r *http.Request) {})
|
||||||
defer cleanup()
|
defer cleanup()
|
||||||
|
|
||||||
req, _ := http.NewRequest(http.MethodGet, url+"/.env", nil)
|
req, _ := http.NewRequest(http.MethodGet, url+"/.env", nil)
|
||||||
@@ -150,7 +152,7 @@ func TestPublicRejectionStrikesBan(t *testing.T) {
|
|||||||
bl := enabledBanlist(1)
|
bl := enabledBanlist(1)
|
||||||
limits := config.DefaultRateLimit()
|
limits := config.DefaultRateLimit()
|
||||||
limits.PublicPerMinute, limits.PublicBurst = 1, 1
|
limits.PublicPerMinute, limits.PublicBurst = 1, 1
|
||||||
_, client, cleanup := guardedEdge(t, bl, "", limits, func(w http.ResponseWriter, r *http.Request) {
|
_, client, cleanup := guardedEdge(t, bl, nil, "", limits, func(w http.ResponseWriter, r *http.Request) {
|
||||||
_, _ = w.Write([]byte(`{"token":"tok","user_id":"u-1","is_guest":true,"display_name":"Guest"}`))
|
_, _ = w.Write([]byte(`{"token":"tok","user_id":"u-1","is_guest":true,"display_name":"Guest"}`))
|
||||||
})
|
})
|
||||||
defer cleanup()
|
defer cleanup()
|
||||||
@@ -173,7 +175,7 @@ func TestUserRejectionDoesNotBan(t *testing.T) {
|
|||||||
bl := enabledBanlist(1)
|
bl := enabledBanlist(1)
|
||||||
limits := config.DefaultRateLimit()
|
limits := config.DefaultRateLimit()
|
||||||
limits.UserPerMinute, limits.UserBurst = 1, 1
|
limits.UserPerMinute, limits.UserBurst = 1, 1
|
||||||
_, client, cleanup := guardedEdge(t, bl, "", limits, func(w http.ResponseWriter, r *http.Request) {
|
_, client, cleanup := guardedEdge(t, bl, nil, "", limits, func(w http.ResponseWriter, r *http.Request) {
|
||||||
switch r.URL.Path {
|
switch r.URL.Path {
|
||||||
case "/api/v1/internal/sessions/resolve":
|
case "/api/v1/internal/sessions/resolve":
|
||||||
_, _ = w.Write([]byte(`{"user_id":"u-1","is_guest":false}`))
|
_, _ = w.Write([]byte(`{"user_id":"u-1","is_guest":false}`))
|
||||||
@@ -202,7 +204,7 @@ func TestUserRejectionDoesNotBan(t *testing.T) {
|
|||||||
// caller and returns the ordinary invalid-session error without a backend call.
|
// caller and returns the ordinary invalid-session error without a backend call.
|
||||||
func TestHoneytokenBansAndRejects(t *testing.T) {
|
func TestHoneytokenBansAndRejects(t *testing.T) {
|
||||||
bl := enabledBanlist(100)
|
bl := enabledBanlist(100)
|
||||||
_, client, cleanup := guardedEdge(t, bl, "s3cr3t-trap", config.DefaultRateLimit(), func(w http.ResponseWriter, r *http.Request) {
|
_, client, cleanup := guardedEdge(t, bl, nil, "s3cr3t-trap", config.DefaultRateLimit(), func(w http.ResponseWriter, r *http.Request) {
|
||||||
t.Error("backend must not be called for the honeytoken")
|
t.Error("backend must not be called for the honeytoken")
|
||||||
})
|
})
|
||||||
defer cleanup()
|
defer cleanup()
|
||||||
@@ -217,3 +219,25 @@ func TestHoneytokenBansAndRejects(t *testing.T) {
|
|||||||
t.Fatal("the honeytoken must ban the caller")
|
t.Fatal("the honeytoken must ban the caller")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// TestAbuseGuardBlocksBlocklistedIP verifies a client IP in the community blocklist feed is refused
|
||||||
|
// with 403 at the HTTP layer, before any handler runs.
|
||||||
|
func TestAbuseGuardBlocksBlocklistedIP(t *testing.T) {
|
||||||
|
block := ratelimit.NewBlocklist(true, nil)
|
||||||
|
_, feed, err := net.ParseCIDR("127.0.0.0/8")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
block.SetCIDRs([]*net.IPNet{feed}, time.Now())
|
||||||
|
url, _, cleanup := guardedEdge(t, nil, block, "", config.DefaultRateLimit(), func(w http.ResponseWriter, r *http.Request) {})
|
||||||
|
defer cleanup()
|
||||||
|
|
||||||
|
resp, err := noRedirect().Get(url + "/")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("get: %v", err)
|
||||||
|
}
|
||||||
|
_ = resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusForbidden {
|
||||||
|
t.Fatalf("blocklisted GET / = %d, want 403", resp.StatusCode)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -7,6 +7,8 @@ import (
|
|||||||
"go.opentelemetry.io/otel/attribute"
|
"go.opentelemetry.io/otel/attribute"
|
||||||
"go.opentelemetry.io/otel/metric"
|
"go.opentelemetry.io/otel/metric"
|
||||||
"go.opentelemetry.io/otel/metric/noop"
|
"go.opentelemetry.io/otel/metric/noop"
|
||||||
|
|
||||||
|
"scrabble/gateway/internal/ratelimit"
|
||||||
)
|
)
|
||||||
|
|
||||||
// meterName scopes the gateway edge's OpenTelemetry instruments.
|
// meterName scopes the gateway edge's OpenTelemetry instruments.
|
||||||
@@ -27,6 +29,7 @@ type serverMetrics struct {
|
|||||||
edge metric.Float64Histogram
|
edge metric.Float64Histogram
|
||||||
rateLimited metric.Int64Counter
|
rateLimited metric.Int64Counter
|
||||||
banned metric.Int64Counter
|
banned metric.Int64Counter
|
||||||
|
blocklisted metric.Int64Counter
|
||||||
active *activeUsers
|
active *activeUsers
|
||||||
// Client-reported local move-preview adoption (see localEvalMetricsHandler).
|
// Client-reported local move-preview adoption (see localEvalMetricsHandler).
|
||||||
localColdStart metric.Int64Counter
|
localColdStart metric.Int64Counter
|
||||||
@@ -40,7 +43,7 @@ type serverMetrics struct {
|
|||||||
// falling back to a no-op histogram on the (rare) construction error. The
|
// falling back to a no-op histogram on the (rare) construction error. The
|
||||||
// active_users gauge is registered as an observable callback over the in-memory
|
// active_users gauge is registered as an observable callback over the in-memory
|
||||||
// tracker.
|
// tracker.
|
||||||
func newServerMetrics(meter metric.Meter) *serverMetrics {
|
func newServerMetrics(meter metric.Meter, bl *ratelimit.Blocklist) *serverMetrics {
|
||||||
if meter == nil {
|
if meter == nil {
|
||||||
meter = noop.NewMeterProvider().Meter(meterName)
|
meter = noop.NewMeterProvider().Meter(meterName)
|
||||||
}
|
}
|
||||||
@@ -68,6 +71,8 @@ func newServerMetrics(meter metric.Meter) *serverMetrics {
|
|||||||
}
|
}
|
||||||
m := &serverMetrics{
|
m := &serverMetrics{
|
||||||
edge: h, rateLimited: c, banned: b, active: newActiveUsers(),
|
edge: h, rateLimited: c, banned: b, active: newActiveUsers(),
|
||||||
|
blocklisted: counterOf(meter, "gateway_blocklist_blocked_total",
|
||||||
|
"Requests refused at the edge by the community IP blocklist (Spamhaus DROP)."),
|
||||||
localColdStart: counterOf(meter, "local_eval_cold_start_total", "App cold starts reported by clients — the denominator for local-move-preview adoption."),
|
localColdStart: counterOf(meter, "local_eval_cold_start_total", "App cold starts reported by clients — the denominator for local-move-preview adoption."),
|
||||||
localDictLoad: counterOf(meter, "local_eval_dict_load_total", "Client dictionary loads for the local move preview, by result (fetched, cache_hit or miss)."),
|
localDictLoad: counterOf(meter, "local_eval_dict_load_total", "Client dictionary loads for the local move preview, by result (fetched, cache_hit or miss)."),
|
||||||
localPreview: counterOf(meter, "local_eval_preview_total", "Client move previews, by path (local on-device, or network fallback)."),
|
localPreview: counterOf(meter, "local_eval_preview_total", "Client move previews, by path (local on-device, or network fallback)."),
|
||||||
@@ -90,6 +95,25 @@ func newServerMetrics(meter metric.Meter) *serverMetrics {
|
|||||||
return nil
|
return nil
|
||||||
}, gauge)
|
}, gauge)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Community blocklist status: the feed size and how stale it is, for the dashboard and the
|
||||||
|
// "feed not refreshing" alert. Age is observed only once a feed has loaded, so a disabled or
|
||||||
|
// never-fetched blocklist reports no age (and entries 0).
|
||||||
|
if bl != nil {
|
||||||
|
entries, e1 := meter.Int64ObservableGauge("gateway_blocklist_entries",
|
||||||
|
metric.WithDescription("CIDR ranges in the active community IP blocklist feed."))
|
||||||
|
age, e2 := meter.Float64ObservableGauge("gateway_blocklist_age_seconds",
|
||||||
|
metric.WithDescription("Seconds since the community IP blocklist feed was last successfully fetched (absent until the first fetch)."))
|
||||||
|
if e1 == nil && e2 == nil {
|
||||||
|
_, _ = meter.RegisterCallback(func(_ context.Context, o metric.Observer) error {
|
||||||
|
o.ObserveInt64(entries, int64(bl.Len()))
|
||||||
|
if last := bl.LastFetch(); !last.IsZero() {
|
||||||
|
o.ObserveFloat64(age, time.Since(last).Seconds())
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}, entries, age)
|
||||||
|
}
|
||||||
|
}
|
||||||
return m
|
return m
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -118,6 +142,11 @@ func (m *serverMetrics) recordBan(ctx context.Context, reason string) {
|
|||||||
m.banned.Add(ctx, 1, metric.WithAttributes(attribute.String("reason", reason)))
|
m.banned.Add(ctx, 1, metric.WithAttributes(attribute.String("reason", reason)))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// recordBlocklistBlock counts one request refused by the community IP blocklist.
|
||||||
|
func (m *serverMetrics) recordBlocklistBlock(ctx context.Context) {
|
||||||
|
m.blocklisted.Add(ctx, 1)
|
||||||
|
}
|
||||||
|
|
||||||
// recordUnsupportedEngine counts one client turned away by the unsupported-engine boot screen,
|
// recordUnsupportedEngine counts one client turned away by the unsupported-engine boot screen,
|
||||||
// labelled by reason and Chromium major. The caller passes both already reduced to bounded label
|
// labelled by reason and Chromium major. The caller passes both already reduced to bounded label
|
||||||
// sets (see normalizeUnsupported) so a spoofed beacon cannot explode the metric cardinality.
|
// sets (see normalizeUnsupported) so a spoofed beacon cannot explode the metric cardinality.
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ func TestEdgeMetric(t *testing.T) {
|
|||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
reader := sdkmetric.NewManualReader()
|
reader := sdkmetric.NewManualReader()
|
||||||
meter := sdkmetric.NewMeterProvider(sdkmetric.WithReader(reader)).Meter("test")
|
meter := sdkmetric.NewMeterProvider(sdkmetric.WithReader(reader)).Meter("test")
|
||||||
m := newServerMetrics(meter)
|
m := newServerMetrics(meter, nil)
|
||||||
|
|
||||||
m.recordEdge(ctx, "game.submit_play", "ok", time.Now().Add(-time.Millisecond))
|
m.recordEdge(ctx, "game.submit_play", "ok", time.Now().Add(-time.Millisecond))
|
||||||
m.recordEdge(ctx, "game.submit_play", "ok", time.Now().Add(-time.Millisecond))
|
m.recordEdge(ctx, "game.submit_play", "ok", time.Now().Add(-time.Millisecond))
|
||||||
@@ -74,7 +74,7 @@ func TestRateLimitedMetric(t *testing.T) {
|
|||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
reader := sdkmetric.NewManualReader()
|
reader := sdkmetric.NewManualReader()
|
||||||
meter := sdkmetric.NewMeterProvider(sdkmetric.WithReader(reader)).Meter("test")
|
meter := sdkmetric.NewMeterProvider(sdkmetric.WithReader(reader)).Meter("test")
|
||||||
m := newServerMetrics(meter)
|
m := newServerMetrics(meter, nil)
|
||||||
|
|
||||||
m.recordRateLimited(ctx, "user")
|
m.recordRateLimited(ctx, "user")
|
||||||
m.recordRateLimited(ctx, "user")
|
m.recordRateLimited(ctx, "user")
|
||||||
@@ -112,7 +112,7 @@ func TestBannedMetric(t *testing.T) {
|
|||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
reader := sdkmetric.NewManualReader()
|
reader := sdkmetric.NewManualReader()
|
||||||
meter := sdkmetric.NewMeterProvider(sdkmetric.WithReader(reader)).Meter("test")
|
meter := sdkmetric.NewMeterProvider(sdkmetric.WithReader(reader)).Meter("test")
|
||||||
m := newServerMetrics(meter)
|
m := newServerMetrics(meter, nil)
|
||||||
|
|
||||||
m.recordBan(ctx, "tripwire")
|
m.recordBan(ctx, "tripwire")
|
||||||
m.recordBan(ctx, "tripwire")
|
m.recordBan(ctx, "tripwire")
|
||||||
@@ -150,7 +150,7 @@ func TestUnsupportedEngineMetric(t *testing.T) {
|
|||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
reader := sdkmetric.NewManualReader()
|
reader := sdkmetric.NewManualReader()
|
||||||
meter := sdkmetric.NewMeterProvider(sdkmetric.WithReader(reader)).Meter("test")
|
meter := sdkmetric.NewMeterProvider(sdkmetric.WithReader(reader)).Meter("test")
|
||||||
m := newServerMetrics(meter)
|
m := newServerMetrics(meter, nil)
|
||||||
|
|
||||||
m.recordUnsupportedEngine(ctx, "no_bigint", "66")
|
m.recordUnsupportedEngine(ctx, "no_bigint", "66")
|
||||||
m.recordUnsupportedEngine(ctx, "no_bigint", "66")
|
m.recordUnsupportedEngine(ctx, "no_bigint", "66")
|
||||||
|
|||||||
@@ -77,6 +77,7 @@ type Server struct {
|
|||||||
limiter *ratelimit.Limiter
|
limiter *ratelimit.Limiter
|
||||||
tracker *ratelimit.Tracker
|
tracker *ratelimit.Tracker
|
||||||
banlist *ratelimit.Banlist
|
banlist *ratelimit.Banlist
|
||||||
|
blocklist *ratelimit.Blocklist
|
||||||
honeytoken string
|
honeytoken string
|
||||||
vkAppSecret string
|
vkAppSecret string
|
||||||
hub *push.Hub
|
hub *push.Hub
|
||||||
@@ -109,6 +110,9 @@ type Deps struct {
|
|||||||
// Banlist enforces temporary IP bans on the hot path; nil selects a disabled
|
// Banlist enforces temporary IP bans on the hot path; nil selects a disabled
|
||||||
// (inert) banlist.
|
// (inert) banlist.
|
||||||
Banlist *ratelimit.Banlist
|
Banlist *ratelimit.Banlist
|
||||||
|
// Blocklist enforces the community IP blocklist on the hot path; nil selects a
|
||||||
|
// disabled (inert) blocklist.
|
||||||
|
Blocklist *ratelimit.Blocklist
|
||||||
// Honeytoken, when non-empty, is the planted bearer value whose presentation
|
// Honeytoken, when non-empty, is the planted bearer value whose presentation
|
||||||
// bans the caller and raises a high-severity alarm.
|
// bans the caller and raises a high-severity alarm.
|
||||||
Honeytoken string
|
Honeytoken string
|
||||||
@@ -148,6 +152,10 @@ func NewServer(d Deps) *Server {
|
|||||||
if banlist == nil {
|
if banlist == nil {
|
||||||
banlist = ratelimit.NewBanlist(ratelimit.BanConfig{})
|
banlist = ratelimit.NewBanlist(ratelimit.BanConfig{})
|
||||||
}
|
}
|
||||||
|
blocklist := d.Blocklist
|
||||||
|
if blocklist == nil {
|
||||||
|
blocklist = ratelimit.NewBlocklist(false, nil)
|
||||||
|
}
|
||||||
rl := d.RateLimit
|
rl := d.RateLimit
|
||||||
if rl == (config.RateLimitConfig{}) {
|
if rl == (config.RateLimitConfig{}) {
|
||||||
rl = config.DefaultRateLimit()
|
rl = config.DefaultRateLimit()
|
||||||
@@ -160,12 +168,13 @@ func NewServer(d Deps) *Server {
|
|||||||
limiter: limiter,
|
limiter: limiter,
|
||||||
tracker: tracker,
|
tracker: tracker,
|
||||||
banlist: banlist,
|
banlist: banlist,
|
||||||
|
blocklist: blocklist,
|
||||||
honeytoken: d.Honeytoken,
|
honeytoken: d.Honeytoken,
|
||||||
hub: d.Hub,
|
hub: d.Hub,
|
||||||
heartbeat: d.Heartbeat,
|
heartbeat: d.Heartbeat,
|
||||||
log: log,
|
log: log,
|
||||||
adminProxy: d.AdminProxy,
|
adminProxy: d.AdminProxy,
|
||||||
metrics: newServerMetrics(d.Meter),
|
metrics: newServerMetrics(d.Meter, blocklist),
|
||||||
maxBodyBytes: maxBody,
|
maxBodyBytes: maxBody,
|
||||||
publicPolicy: ratelimit.PerMinute(rl.PublicPerMinute, rl.PublicBurst),
|
publicPolicy: ratelimit.PerMinute(rl.PublicPerMinute, rl.PublicBurst),
|
||||||
userPolicy: ratelimit.PerMinute(rl.UserPerMinute, rl.UserBurst),
|
userPolicy: ratelimit.PerMinute(rl.UserPerMinute, rl.UserBurst),
|
||||||
@@ -255,6 +264,13 @@ func (s *Server) abuseGuard(next http.Handler) http.Handler {
|
|||||||
http.Error(w, "banned", http.StatusTooManyRequests)
|
http.Error(w, "banned", http.StatusTooManyRequests)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
// The community IP blocklist (Spamhaus DROP): a known-bad source is refused before any work.
|
||||||
|
// Counted, not logged per request (a blocked scanner hammers). Inert on a disabled blocklist.
|
||||||
|
if s.blocklist.Blocked(ip) {
|
||||||
|
s.metrics.recordBlocklistBlock(r.Context())
|
||||||
|
http.Error(w, "forbidden", http.StatusForbidden)
|
||||||
|
return
|
||||||
|
}
|
||||||
if r.Header.Get(honeypotHeader) != "" {
|
if r.Header.Get(honeypotHeader) != "" {
|
||||||
s.log.Warn("honeypot tripwire",
|
s.log.Warn("honeypot tripwire",
|
||||||
zap.String("path", r.URL.Path),
|
zap.String("path", r.URL.Path),
|
||||||
|
|||||||
@@ -0,0 +1,188 @@
|
|||||||
|
package ratelimit
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bufio"
|
||||||
|
"encoding/binary"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ipRange is an inclusive IPv4 range [lo, hi] as uint32 — the form the blocklist matches against.
|
||||||
|
type ipRange struct{ lo, hi uint32 }
|
||||||
|
|
||||||
|
// Blocklist is a static IPv4 CIDR blocklist (a curated community feed such as Spamhaus DROP) enforced
|
||||||
|
// on the hot path alongside the fail2ban [Banlist]. It is refreshed periodically ([ApplyRefresh]); an
|
||||||
|
// allowlist (never blocked) protects known-good infrastructure and is checked first. Only IPv4 is
|
||||||
|
// matched — an IPv6 client is never blocked here (the fail2ban list and the honeypot still cover it).
|
||||||
|
// Disabled by default (prod-only, like the ban): while disabled, Blocked is always false.
|
||||||
|
type Blocklist struct {
|
||||||
|
enabled bool
|
||||||
|
allow []ipRange // sorted, non-overlapping; from config, immutable after construction
|
||||||
|
|
||||||
|
mu sync.RWMutex
|
||||||
|
ranges []ipRange // sorted, non-overlapping; the current feed
|
||||||
|
fetchedAt time.Time // last successful SetCIDRs; zero = never loaded
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewBlocklist builds a Blocklist. enabled gates the whole mechanism; allow is the never-block set
|
||||||
|
// (CIDRs / bare IPs already parsed) — a client in it is never blocked even if the feed lists it.
|
||||||
|
func NewBlocklist(enabled bool, allow []*net.IPNet) *Blocklist {
|
||||||
|
return &Blocklist{enabled: enabled, allow: toRanges(allow)}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Blocked reports whether ip (a textual address) is in the current feed and not allowlisted. It is
|
||||||
|
// false on a disabled or empty blocklist, and false for any non-IPv4 address.
|
||||||
|
func (b *Blocklist) Blocked(ip string) bool {
|
||||||
|
if !b.enabled {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
v, ok := ipv4ToUint32(ip)
|
||||||
|
if !ok {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
b.mu.RLock()
|
||||||
|
defer b.mu.RUnlock()
|
||||||
|
if len(b.ranges) == 0 || rangesContain(b.allow, v) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return rangesContain(b.ranges, v)
|
||||||
|
}
|
||||||
|
|
||||||
|
// SetCIDRs swaps in a freshly fetched feed, recording the fetch time. Non-IPv4 CIDRs are ignored.
|
||||||
|
func (b *Blocklist) SetCIDRs(cidrs []*net.IPNet, at time.Time) {
|
||||||
|
r := toRanges(cidrs)
|
||||||
|
b.mu.Lock()
|
||||||
|
b.ranges = r
|
||||||
|
b.fetchedAt = at
|
||||||
|
b.mu.Unlock()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Clear drops the current feed (fail-open) but keeps the last-fetch time, so a staleness gauge keeps
|
||||||
|
// climbing. Blocked then returns false until a fresh feed loads.
|
||||||
|
func (b *Blocklist) Clear() {
|
||||||
|
b.mu.Lock()
|
||||||
|
b.ranges = nil
|
||||||
|
b.mu.Unlock()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Len returns the number of ranges currently enforced.
|
||||||
|
func (b *Blocklist) Len() int {
|
||||||
|
b.mu.RLock()
|
||||||
|
defer b.mu.RUnlock()
|
||||||
|
return len(b.ranges)
|
||||||
|
}
|
||||||
|
|
||||||
|
// LastFetch returns the time of the last successful feed load (zero if never).
|
||||||
|
func (b *Blocklist) LastFetch() time.Time {
|
||||||
|
b.mu.RLock()
|
||||||
|
defer b.mu.RUnlock()
|
||||||
|
return b.fetchedAt
|
||||||
|
}
|
||||||
|
|
||||||
|
// RefreshOutcome is the result of one refresh attempt, for the caller's logging and metrics.
|
||||||
|
type RefreshOutcome int
|
||||||
|
|
||||||
|
const (
|
||||||
|
// RefreshUpdated: a new feed was fetched and applied.
|
||||||
|
RefreshUpdated RefreshOutcome = iota
|
||||||
|
// RefreshKept: the fetch failed but the last-good feed is still fresh, so it was kept.
|
||||||
|
RefreshKept
|
||||||
|
// RefreshDropped: the fetch failed and the feed went stale, so it was dropped (fail-open).
|
||||||
|
RefreshDropped
|
||||||
|
)
|
||||||
|
|
||||||
|
// ApplyRefresh updates bl from one fetch outcome: on success it applies the new feed; on failure it
|
||||||
|
// keeps the last-good feed unless it is older than maxStaleness, in which case it drops it (fail-open
|
||||||
|
// — better to under-block than to block a legitimate client on a frozen feed). now is the wall clock.
|
||||||
|
func ApplyRefresh(bl *Blocklist, cidrs []*net.IPNet, fetchErr error, now time.Time, maxStaleness time.Duration) RefreshOutcome {
|
||||||
|
if fetchErr == nil {
|
||||||
|
bl.SetCIDRs(cidrs, now)
|
||||||
|
return RefreshUpdated
|
||||||
|
}
|
||||||
|
last := bl.LastFetch()
|
||||||
|
if last.IsZero() || now.Sub(last) > maxStaleness {
|
||||||
|
bl.Clear()
|
||||||
|
return RefreshDropped
|
||||||
|
}
|
||||||
|
return RefreshKept
|
||||||
|
}
|
||||||
|
|
||||||
|
// ParseDROP parses a Spamhaus DROP-style feed: one CIDR per line with an optional "; comment" tail,
|
||||||
|
// plus blank and comment lines. It returns the IPv4 networks; a bare IP is read as a /32, and
|
||||||
|
// non-IPv4 or malformed entries are skipped so one bad line never fails the whole feed.
|
||||||
|
func ParseDROP(r io.Reader) ([]*net.IPNet, error) {
|
||||||
|
var out []*net.IPNet
|
||||||
|
sc := bufio.NewScanner(r)
|
||||||
|
sc.Buffer(make([]byte, 0, 64*1024), 1<<20)
|
||||||
|
for sc.Scan() {
|
||||||
|
line := sc.Text()
|
||||||
|
if i := strings.IndexByte(line, ';'); i >= 0 {
|
||||||
|
line = line[:i]
|
||||||
|
}
|
||||||
|
line = strings.TrimSpace(line)
|
||||||
|
if line == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !strings.Contains(line, "/") {
|
||||||
|
line += "/32"
|
||||||
|
}
|
||||||
|
_, ipnet, err := net.ParseCIDR(line)
|
||||||
|
if err != nil || ipnet.IP.To4() == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
out = append(out, ipnet)
|
||||||
|
}
|
||||||
|
if err := sc.Err(); err != nil {
|
||||||
|
return nil, fmt.Errorf("ratelimit: read blocklist: %w", err)
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// toRanges converts IPv4 CIDRs to sorted, uint32 inclusive ranges (the match form); non-IPv4 CIDRs
|
||||||
|
// are dropped. Sorting by lo lets [rangesContain] binary-search.
|
||||||
|
func toRanges(cidrs []*net.IPNet) []ipRange {
|
||||||
|
out := make([]ipRange, 0, len(cidrs))
|
||||||
|
for _, n := range cidrs {
|
||||||
|
if n == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
ip4 := n.IP.To4()
|
||||||
|
if ip4 == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
ones, bits := n.Mask.Size()
|
||||||
|
if bits != 32 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
lo := binary.BigEndian.Uint32(ip4)
|
||||||
|
hi := lo | (uint32(0xffffffff) >> uint(ones))
|
||||||
|
out = append(out, ipRange{lo: lo, hi: hi})
|
||||||
|
}
|
||||||
|
sort.Slice(out, func(i, j int) bool { return out[i].lo < out[j].lo })
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// rangesContain reports whether v falls in any range of the sorted, non-overlapping slice, via a
|
||||||
|
// binary search for the last range whose lo is at most v.
|
||||||
|
func rangesContain(ranges []ipRange, v uint32) bool {
|
||||||
|
i := sort.Search(len(ranges), func(i int) bool { return ranges[i].lo > v })
|
||||||
|
return i > 0 && ranges[i-1].hi >= v
|
||||||
|
}
|
||||||
|
|
||||||
|
// ipv4ToUint32 parses a textual address to a big-endian uint32, reporting whether it was IPv4.
|
||||||
|
func ipv4ToUint32(s string) (uint32, bool) {
|
||||||
|
ip := net.ParseIP(s)
|
||||||
|
if ip == nil {
|
||||||
|
return 0, false
|
||||||
|
}
|
||||||
|
ip4 := ip.To4()
|
||||||
|
if ip4 == nil {
|
||||||
|
return 0, false
|
||||||
|
}
|
||||||
|
return binary.BigEndian.Uint32(ip4), true
|
||||||
|
}
|
||||||
@@ -0,0 +1,106 @@
|
|||||||
|
package ratelimit
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"net"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// cidrs parses CIDR strings into networks for a test fixture.
|
||||||
|
func cidrs(t *testing.T, ss ...string) []*net.IPNet {
|
||||||
|
t.Helper()
|
||||||
|
out := make([]*net.IPNet, 0, len(ss))
|
||||||
|
for _, s := range ss {
|
||||||
|
_, n, err := net.ParseCIDR(s)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("bad cidr %q: %v", s, err)
|
||||||
|
}
|
||||||
|
out = append(out, n)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBlocklistBlocked(t *testing.T) {
|
||||||
|
bl := NewBlocklist(true, cidrs(t, "10.20.30.0/24")) // allowlist
|
||||||
|
bl.SetCIDRs(cidrs(t, "1.2.3.0/24", "203.0.113.5/32", "198.51.100.0/28", "10.20.30.0/24"), time.Now())
|
||||||
|
cases := map[string]bool{
|
||||||
|
"1.2.3.0": true, // range start
|
||||||
|
"1.2.3.255": true, // range end
|
||||||
|
"1.2.4.0": false, // just past the /24
|
||||||
|
"203.0.113.5": true, // /32 exact
|
||||||
|
"203.0.113.6": false,
|
||||||
|
"198.51.100.15": true, // within /28
|
||||||
|
"198.51.100.16": false, // past the /28
|
||||||
|
"9.9.9.9": false, // not listed
|
||||||
|
"10.20.30.99": false, // listed BUT allowlisted — never blocked
|
||||||
|
"::1": false, // IPv6 — never matched here
|
||||||
|
"not-an-ip": false,
|
||||||
|
}
|
||||||
|
for ip, want := range cases {
|
||||||
|
if got := bl.Blocked(ip); got != want {
|
||||||
|
t.Errorf("Blocked(%q) = %v, want %v", ip, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBlocklistDisabledOrEmpty(t *testing.T) {
|
||||||
|
off := NewBlocklist(false, nil)
|
||||||
|
off.SetCIDRs(cidrs(t, "1.2.3.0/24"), time.Now())
|
||||||
|
if off.Blocked("1.2.3.4") {
|
||||||
|
t.Error("a disabled blocklist must not block")
|
||||||
|
}
|
||||||
|
empty := NewBlocklist(true, nil)
|
||||||
|
if empty.Blocked("1.2.3.4") {
|
||||||
|
t.Error("an empty (never-loaded) blocklist must not block")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseDROP(t *testing.T) {
|
||||||
|
in := "; Spamhaus DROP\n" +
|
||||||
|
"1.2.3.0/24 ; SBL1\n" +
|
||||||
|
" 198.51.100.0/28 ; SBL2\n" +
|
||||||
|
"\n" +
|
||||||
|
"203.0.113.5 ; a bare IP -> /32\n" +
|
||||||
|
"2001:db8::/32 ; IPv6 skipped\n" +
|
||||||
|
"garbage line\n"
|
||||||
|
nets, err := ParseDROP(strings.NewReader(in))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(nets) != 3 {
|
||||||
|
t.Fatalf("got %d networks, want 3: %v", len(nets), nets)
|
||||||
|
}
|
||||||
|
bl := NewBlocklist(true, nil)
|
||||||
|
bl.SetCIDRs(nets, time.Now())
|
||||||
|
for ip, want := range map[string]bool{"1.2.3.9": true, "198.51.100.1": true, "203.0.113.5": true, "203.0.113.6": false, "2001:db8::1": false} {
|
||||||
|
if got := bl.Blocked(ip); got != want {
|
||||||
|
t.Errorf("Blocked(%s) = %v, want %v", ip, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestApplyRefresh(t *testing.T) {
|
||||||
|
bl := NewBlocklist(true, nil)
|
||||||
|
now := time.Now()
|
||||||
|
|
||||||
|
if o := ApplyRefresh(bl, cidrs(t, "1.2.3.0/24"), nil, now, time.Hour); o != RefreshUpdated {
|
||||||
|
t.Fatalf("success: want RefreshUpdated, got %v", o)
|
||||||
|
}
|
||||||
|
if !bl.Blocked("1.2.3.4") {
|
||||||
|
t.Fatal("a successful refresh must apply the feed")
|
||||||
|
}
|
||||||
|
if o := ApplyRefresh(bl, nil, errors.New("net"), now.Add(30*time.Minute), time.Hour); o != RefreshKept {
|
||||||
|
t.Fatalf("fresh failure: want RefreshKept, got %v", o)
|
||||||
|
}
|
||||||
|
if !bl.Blocked("1.2.3.4") {
|
||||||
|
t.Error("a kept feed must still block")
|
||||||
|
}
|
||||||
|
if o := ApplyRefresh(bl, nil, errors.New("net"), now.Add(2*time.Hour), time.Hour); o != RefreshDropped {
|
||||||
|
t.Fatalf("stale failure: want RefreshDropped, got %v", o)
|
||||||
|
}
|
||||||
|
if bl.Blocked("1.2.3.4") {
|
||||||
|
t.Error("a stale feed must be dropped (fail-open)")
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user