Ilia Denisov
84a0ccb23f
Stand up a production-mirror monitoring stack in the long-lived dev contour, all on galaxy-dev-internal with no host ports (reached only via the in-repo galaxy-dev-caddy): - Prometheus scrapes backend:9100, gateway:9191, node-exporter and cadvisor (30s interval, 15d retention); Loki (7d) + promtail (Docker service discovery by the galaxy.stack=dev-deploy label) for logs; Tempo (3d) for traces. - Backend and gateway now export OTLP traces to Tempo over plaintext gRPC on the internal network (OTEL_EXPORTER_OTLP_INSECURE). - Grafana provisioned as code (Prometheus/Loki/Tempo datasources plus a starter dashboard), served under /grafana/ via Caddy sub-path mode; admin password from the GALAXY_DEV_GRAFANA_ADMIN_PASSWORD secret. - Expose the Mailpit capture UI under /mailpit/ (Caddy basic-auth + MP_WEBROOT) so every captured message is readable regardless of relay. - dev-deploy.yaml seeds the monitoring config to a stable, reboot- surviving host path and injects the Grafana admin secret. Per-service memory limits keep the footprint within budget. All collector config lives under tools/dev-deploy/monitoring/ for dev/prod parity.
31 lines
932 B
YAML
31 lines
932 B
YAML
# Promtail tails the dev stack's container logs via the Docker API
|
|
# (service discovery filtered to the galaxy.stack=dev-deploy label) and
|
|
# ships them to Loki. Requires the Docker socket mounted read-only.
|
|
server:
|
|
http_listen_port: 9080
|
|
grpc_listen_port: 0
|
|
log_level: warn
|
|
|
|
positions:
|
|
filename: /tmp/positions.yaml
|
|
|
|
clients:
|
|
- url: http://galaxy-loki:3100/loki/api/v1/push
|
|
|
|
scrape_configs:
|
|
- job_name: docker
|
|
docker_sd_configs:
|
|
- host: unix:///var/run/docker.sock
|
|
refresh_interval: 15s
|
|
filters:
|
|
- name: label
|
|
values: ["galaxy.stack=dev-deploy"]
|
|
relabel_configs:
|
|
- source_labels: ["__meta_docker_container_name"]
|
|
regex: "/?(.*)"
|
|
target_label: container
|
|
- source_labels: ["__meta_docker_container_label_galaxy_game_id"]
|
|
target_label: game_id
|
|
- source_labels: ["__meta_docker_container_log_stream"]
|
|
target_label: stream
|