docs(#000057): add 3090/Hermes-8B rig, dollars, quant/precision confound

- §2 now a two-rig table: 4090/Qwen-27B-Q4_K_XL/llama.cpp (isolated) vs
  3090/Hermes-8B-FP8-Dynamic/vLLM (live/public). Spells out that the
  cross-model rate gap confounds FOUR variables (params, quant/precision,
  engine, GPU) — not '8B vs 27B' alone.
- §5.4 hermes results under live traffic. Method finding: the watt_bench
  window integral is unusable on a contended card, but the slope
  calibration survives clean (cancels the variable shared-load baseline)
  — so on shared/public cards use watt_calibrate, not the window integral.
  Rates: hermes 0.109/4.40 J/tok vs qwen 0.175/6.16; decode 35-40x prefill.
- Per-query DOLLARS both rigs @$0.33/kWh: a grounded substrate answer is
  <$0.10 per 1000 queries (hermes $0.085, qwen $0.158); hermes ~half qwen.
- §5.2 budget confound corrected to flag Hermes-tuning honestly (was
  rationalized). §9: fixed-budget apples-to-apples re-run + per-model
  budget tuning added as next steps.

docs/energy-cogs-benchmark.md

@@ -1,7 +1,8 @@
 # Energy COGS benchmark — arborist substrate vs bare model
 
-**Status:** living report. First rig: qwen-27B on a dedicated RTX 4090.
-3090 + Hermes-3-8B and the reasoning variants follow on the same harness.
+**Status:** living report. Two rigs measured: qwen-27B-Q4 on a dedicated
+RTX 4090 (isolated) and Hermes-3-8B-FP8 on a 3090 (live/public,
+contended). Reasoning variants follow on the same harness.
 **Ticket:** #000057 (watt bench) / #000060 (same-model substrate delta,
 cost axis). **Audience:** Dav1d de-novo review + maintainers.
 
@@ -15,14 +16,22 @@ answers (energy is correctness-independent).
 
 ## 2. System under test
 
-| component | value |
-|---|---|
-| GPU | NVIDIA RTX 4090 (24 GB), `ai.foxhop.net` |
-| model | `Qwen3.6-27B-UD-Q4_K_XL.gguf`, llama.cpp, `enable_thinking=false` |
-| substrate | arborist `query()` pipeline, **STOCK V.1** frozen config |
-| substrate hash | `claim_lattice` `036a4c79…`, `quote` `5b6ca4c5…` (`bench/stock_v1.py`) |
-| driver | orchestrator drives the LLM endpoint; power sampled on the box |
-| price | $0.33/kWh (configurable `--price-per-kwh`; the only non-measured input) |
+Two rigs:
+
+| | rig A | rig B |
+|---|---|---|
+| GPU | **RTX 4090** (24 GB), `ai.foxhop.net` | **RTX 3090** (24 GB), `3090-ai.foxhop.net` |
+| model | `Qwen3.6-27B-UD-Q4_K_XL.gguf` | `Hermes-3-Llama-3.1-8B-FP8-Dynamic` |
+| params | **27 B** | **8 B** |
+| precision / quant | **~4-bit** (unsloth-dynamic Q4_K_XL GGUF) | **FP8-Dynamic** (8-bit) |
+| inference engine | **llama.cpp** (`llama-server`) | **vLLM** |
+| warm-idle (resident) | ~19 W | ~196–350 W |
+| isolation | **dedicated** (public blocked) | **live/public** (contended) |
+
+Common to both: arborist `query()` pipeline, **STOCK V.1** frozen config
+(`claim_lattice` hash `036a4c79…`, `quote` `5b6ca4c5…`,
+`bench/stock_v1.py`); orchestrator drives the endpoint, power sampled on
+the box; **$0.33/kWh** (the only non-measured input, `--price-per-kwh`).
 
 **Two arms, same model (the substrate-delta design):**
 - **solo** — question only, neutral system prompt, no retrieval, no verifier.
@@ -30,6 +39,13 @@ answers (energy is correctness-independent).
   LLM synthesis → deterministic verifier. Two answer modes
   (`claim_lattice` JSON, `quote` prose) — STOCK V.1 is a two-cell family.
 
+**Cross-rig comparison confounds FOUR variables at once** — params
+(8 B vs 27 B), quantization/precision (FP8-Dynamic vs ~4-bit Q4_K_XL),
+inference engine (vLLM vs llama.cpp), and GPU (3090 vs 4090). The
+per-token rate gap (hermes cheaper) is the **bundle**, not any single
+factor; do not read it as "8 B vs 27 B" alone. Within a rig the
+solo-vs-substrate delta is clean (same model/quant/engine/GPU).
+
 ## 3. Why the card had to be isolated
 
 The 4090 served live internet traffic through a Caddy proxy. A first
@@ -111,6 +127,21 @@ nothing. This — not a higher draw — is why the substrate costs more.
 | substrate · claim_lattice | 6,617 | 91 | **127×** |
 | substrate · quote | 4,424 | 88 | 85× |
 
+The two substrate modes differ in **input** tokens by POLICY, not noise:
+`max_context_chars_by_mode` caps `quote` at 24 KB and `claim_lattice` at
+48 KB (2×), so claim_lattice prefills ~1.5× more context.
+
+**CONFOUND — not apples-to-apples.** Those caps were bench-tuned on
+**Hermes-3-8B** (`docs/qa-modes-bench.md` Sprint 1b), then baked into
+`DEFAULT_QUERY_POLICY` and applied to every model. So (a) the cross-mode
+energy gap is mostly the *budget* difference, not anything intrinsic to
+the mode; and (b) **qwen-27B here runs Hermes-tuned budgets** — its own
+quality-optimal context size is unmeasured. For a clean cross-mode
+energy comparison, hold the context budget constant across modes; for
+cross-model, re-tune the budget per model first. The prefill lever is
+real (≈⅔ of substrate GPU energy is prefill, so the budget directly sets
+prefill COGS) — but the value driving it is currently a Hermes artifact.
+
 ### 5.3 Per-query GPU energy, decomposed (calibration applied)
 `energy ≈ 0.175·input_tok + 6.157·output_tok`
 
@@ -124,6 +155,41 @@ nothing. This — not a higher draw — is why the substrate costs more.
 vs the independently **measured 1,804 J/q** (~5 %) — the input/output
 decomposition reconstructs the directly-measured energy.
 
+### 5.4 Second rig — 3090 / Hermes-8B, under live traffic
+Run on the **public, contended** 3090 (0–4 concurrent web requests
+throughout; idle baseline flagged 69–97 % util). Key method result:
+
+- **The watt_bench per-query window integral is unusable here** — the
+  contaminated idle baseline made marginal clamp toward $0 (`unimodal —
+  no gen state`). Expected: a shared card can't be profiled by a window
+  integral.
+- **The slope calibration survived the traffic clean.** The per-token
+  slope cancels the variable shared-load baseline across reps, so the
+  rates came out clean linear fits (`cached_tokens=0` throughout):
+
+```
+                 prefill (input)         decode (output)
+hermes / 3090    0.109 J/tok $0.010/M    4.40 J/tok $0.403/M   (decode 40× prefill)
+qwen   / 4090    0.175 J/tok $0.016/M    6.16 J/tok $0.564/M   (decode 35× prefill)
+```
+
+  **On a shared/public card, use `watt_calibrate` (slope), not the
+  per-query window integral.**
+
+Per-query energy = calibration rate × **real** token mix (the token
+counts are usage-reported, contamination-independent), sidestepping the
+contaminated window integral:
+
+| arm / mode | hermes/3090 J/q | **hermes $/1k-q** | qwen/4090 J/q | **qwen $/1k-q** |
+|---|---|---|---|---|
+| solo | ~98 | **$0.0090** | ~95 | $0.0087 |
+| substrate · claim_lattice | **927** | **$0.085** | 1,719 | $0.158 |
+| substrate · quote | 765 | $0.070 | 1,317 | $0.121 |
+
+Hermes substrate ≈ **half** qwen's per-query GPU energy — but that is the
+four-way confound (§2), dominated by 8 B vs 27 B. A grounded answer costs
+**< $0.10 / 1,000 queries** of GPU electricity on either rig.
+
 ## 6. Interpretation
 
 - **The substrate's cost is reading the evidence, not generating it.**
@@ -167,20 +233,26 @@ decomposition reconstructs the directly-measured energy.
   split + slope unit-tested in `tests/test_watt_cogs.py` (14 tests).
 - Real token usage: `OpenAICompatibleClient.last_usage`.
 - Frozen substrate: `bench/stock_v1.py` (`assert_not_drifted`).
-- Raw per-cell power samples, window timestamps, and the calibration
-  sweep persisted in `bench/qa_results/watt_4090_*.json` /
-  `watt_calibrate_4090_*.json` for offline re-analysis.
+- Raw per-cell power samples, window timestamps, and calibration sweeps
+  persisted in `bench/qa_results/watt_{4090,3090}_*.json` /
+  `watt_calibrate_{4090,3090}_*.json` (gitignored — local artifacts;
+  numbers reproduced in §5).
 - Isolation: qwen closed to public at the proxy
   (`proxy.unturf.com` `ingress/Caddyfile`, commit `4b1fed1`) — dedicated
-  benchmark rig.
+  4090 rig. The 3090/hermes run was on the **live public** endpoint;
+  contention quantified via vLLM `/metrics` queue depth.
 
 ## 9. Next
 
-- **3090 + Hermes-3-8B** on the same harness + a fresh calibration
-  (`--gpu-label 3090 --models hermes --arborist-ref hermes`; re-run
-  `watt_calibrate --model-key hermes`).
+- **Apples-to-apples re-run with a FIXED context budget** — the per-mode
+  budgets (`max_context_chars_by_mode`) are Hermes-tuned (§5.2 confound),
+  so the cross-mode prefill gap is a budget artifact. A constant-budget
+  pass would isolate intrinsic mode cost.
 - **Reasoning variants** (qwen-think, hermes-reasoning) — phase 3; the
-  decode coefficient will dominate (long reasoning traces = many output
-  tokens at 6 J each).
+  decode coefficient dominates (long reasoning traces = many output
+  tokens at 4–6 J each).
 - **Prefill-cache study** — measure `cached_tokens` lift under repeated
   contexts to size the KV-reuse lever (§6).
+- **Per-model context-budget tuning** — qwen's quality-optimal budget is
+  unmeasured (only Hermes was tuned); needed before any cross-model
+  *quality*-per-joule claim.