Commit graph

1 commit

Author SHA1 Message Date
b5cc970a86
feat: bench/load_monitor.py — stdlib request-load monitor for single-slot endpoints
Answers fox's 2026-05-21 question — 'are we being swamped because we're
open to internet?' — before the multi-day GPU bench, where uncontrolled
internet traffic on the single-slot hermes (3090/vLLM) and qwen
(4090/llama.cpp) endpoints would contaminate wattage + throughput.

Three stdlib subcommands (urllib + sqlite3 + hand-rolled SVG, nothing to
install):
  poll   — scrape each endpoint's Prometheus /metrics on an interval into
           SQLite; queue depth (num_requests_waiting / requests_deferred)
           is the swamp signal a single GPU slot exposes. Prunes past
           --retention-days each cycle (bounded store, no cancer growth).
  graph  — multi-panel SVG: queue depth, running, req/s, tok/s, e2e latency.
  access — parse Caddy/nginx access log for the real client IPs the backend
           can't see behind the proxy hop; top talkers + per-IP rate SVG.

Backend /metrics = HOW MUCH; proxy log = WHO. make monitor-poll /
monitor-graph / monitor-access.
2026-05-21 09:20:53 -04:00