fix(grafana): real byte thresholds for the Database size stat #120

Merged
developer merged 1 commits from feature/grafana-db-size-thresholds into development 2026-06-22 19:41:42 +00:00
Owner

What

Give the Database size stat on the Scrabble - Resources dashboard real byte
thresholds, so a healthy small database no longer renders permanently red.

Why

The panel had no thresholds, so Grafana applied its stat default (green base, red at
>=80
). The query is max(pg_database_size_bytes{datname="scrabble"}) (unit: bytes), so
a ~9 MiB database (9.4M >> 80) showed permanently RED on both test and prod —
reading as a false alert. (The neighbouring percentunit cache-hit stat stayed green only
because its 0..1 values fall under 80.) There are no Grafana alert rules provisioned;
this was purely panel coloring.

loadtest/ is clean — the small DB is expected (the harness wipes + --cleanups, and the
contour is DROP SCHEMA'd on schema changes), not an error.

Thresholds

Sized to the prod disk (40 GiB; 4.6 GiB used; observability bounded — Tempo <=1 GiB,
Prometheus 7d — so the DB is the only unbounded grower):

step size ~% of 40 GiB
green < 8 GiB
yellow 8 GiB ~20%
red 16 GiB ~40%

An early warning with ample runway before the disk tightens, not a panic line. Easy to
retune if the disk budget changes.

Verification

json.load parses; the panel resolves to green/yellow(8 GiB)/red(16 GiB), and the
current ~9 MiB maps to green.

## What Give the **Database size** stat on the *Scrabble - Resources* dashboard real byte thresholds, so a healthy small database no longer renders permanently red. ## Why The panel had no `thresholds`, so Grafana applied its stat default (green base, **red at `>=80`**). The query is `max(pg_database_size_bytes{datname="scrabble"})` (unit: bytes), so a ~9 MiB database (9.4M `>>` 80) showed **permanently RED** on both test and prod — reading as a false alert. (The neighbouring `percentunit` cache-hit stat stayed green only because its 0..1 values fall under 80.) There are **no Grafana alert rules** provisioned; this was purely panel coloring. `loadtest/` is clean — the small DB is expected (the harness wipes + `--cleanup`s, and the contour is `DROP SCHEMA`'d on schema changes), not an error. ## Thresholds Sized to the prod disk (40 GiB; 4.6 GiB used; observability bounded — Tempo `<=`1 GiB, Prometheus 7d — so the DB is the only unbounded grower): | step | size | ~% of 40 GiB | |------|------|--------------| | green | < 8 GiB | — | | yellow | 8 GiB | ~20% | | red | 16 GiB | ~40% | An early warning with ample runway before the disk tightens, not a panic line. Easy to retune if the disk budget changes. ## Verification `json.load` parses; the panel resolves to `green/yellow(8 GiB)/red(16 GiB)`, and the current ~9 MiB maps to **green**.
developer added 1 commit 2026-06-22 19:36:02 +00:00
fix(grafana): give "Database size" stat real byte thresholds
CI / changes (pull_request) Successful in 2s
CI / unit (pull_request) Successful in 10s
CI / integration (pull_request) Successful in 17s
CI / ui (pull_request) Successful in 54s
CI / gate (pull_request) Successful in 0s
CI / deploy (pull_request) Successful in 1m18s
46d569720c
The "Database size" stat had no thresholds, so Grafana applied its stat
default (green base, red at >=80). The query is pg_database_size_bytes, so a
healthy ~9 MiB database (9.4M >> 80) rendered permanently RED on the
Scrabble - Resources dashboard (test + prod), reading as a false alert; the
neighbouring percentunit cache-hit stat stayed green only because its 0..1
values fall under 80.

Add absolute byte thresholds sized to the 40 GiB prod disk (4.6 GiB used,
observability bounded -- Tempo <=1 GiB, Prometheus 7d -- so the DB is the
only unbounded grower): green up to 8 GiB, yellow at 8 GiB (~20% of disk),
red at 16 GiB (~40%), an early warning with ample runway before the disk
tightens, not a panic line. Cosmetic panel coloring only; there are no
Grafana alert rules provisioned.
owner approved these changes 2026-06-22 19:38:38 +00:00
developer merged commit ba57687430 into development 2026-06-22 19:41:42 +00:00
developer deleted branch feature/grafana-db-size-thresholds 2026-06-22 19:41:42 +00:00
Sign in to join this conversation.
No Reviewers
No Label
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: developer/scrabble-game#120