docs(#000057): add quality/value side + cross-model comparison to energy report

The report is now cost AND value (quality-per-dollar), not cost-only.

§5.5 quality delta: substrate-vs-solo (code judge, n=30) on numeral +
stale fixtures, hermes-8B vs qwen-27B-nothink. Substrate lifts both
2-13x and they nearly converge (hermes 18/21, qwen 19/26 grounded-
correct) — grounding comes from retrieval+verification, not parametric
size, so the base model matters far less. Bare-model failure styles
diverge (hermes abstains, qwen fabricates). Quality-per-dollar: ~18-26/30
grounded for $0.085-0.158/1k-q.

Honest caveats recorded: CORRECT=grounded-in-2010-corpus (not current);
the stale fixture is no longer post-corpus; the qwen thinking bug (39c040c)
that voided the first run and the judge strengthening (2d31866). §1
reframed; §8 repro + §9 next updated (post-2010 fixture, SOTA judge for
residue).

docs/energy-cogs-benchmark.md

@@ -8,11 +8,12 @@ cost axis). **Audience:** Dav1d de-novo review + maintainers.
 
 ## 1. Question
 
-What does the arborist substrate cost in **GPU energy per token**,
-versus the same base model answering alone (no retrieval, no verifier)?
-Energy is the *cost* axis that pairs with the quality axis
-(`control_sweep` CG%); this report is cost only — it does **not** grade
-answers (energy is correctness-independent).
+What does the arborist substrate **cost** (GPU energy per token) and what
+is it **worth** (grounded-correct answers), versus the same base model
+answering alone (no retrieval, no verifier)? §1–§5.4 are the cost axis
+(energy is correctness-independent, measured separately); §5.5 is the
+value axis (substrate-vs-solo answer quality + a cross-model comparison).
+Together they give **quality-per-dollar**.
 
 ## 2. System under test
 
@@ -193,6 +194,70 @@ Hermes substrate ≈ **half** qwen's per-query GPU energy — but that is the
 four-way confound (§2), dominated by 8 B vs 27 B. A grounded answer costs
 **< $0.10 / 1,000 queries** of GPU electricity on either rig.
 
+### 5.5 Quality delta (value side) — does the substrate earn its cost?
+
+Cost is meaningless without value. `bench/control_ab.py` runs the SAME
+model **solo** (question only, no retrieval/verifier) vs **substrate-ON**
+(full `query()`), grades each with the deterministic **code judge**
+(`bench/judge_code.py`, NLI-assisted, no quota), n=30 per fixture. Two
+fixtures: **numeral** (stable historical facts, in-corpus) and **stale**
+(world-leader questions — also in-corpus now that the corpus is 2010, see
+below). Verdicts: CORRECT_GROUNDED / WRONG / ABSTAINED / JUDGE_ERROR.
+
+**Substrate-ON — grounded-correct (CG / WRONG / JUDGE_ERROR):**
+
+| fixture | hermes-8B | qwen-27B-nothink |
+|---|---|---|
+| numeral | 18 / 5 / 6 | 19 / 6 / 3 |
+| stale   | 21 / 6 / 3 | **26 / 2 / 1** |
+
+**Solo (bare model) — and the failure *style* differs:**
+
+| fixture | hermes-8B | qwen-27B-nothink |
+|---|---|---|
+| numeral | 9 CG / 4 WR / **14 ABSTAIN** | 5 CG / **12 WR** / 3 AB |
+| stale   | 5 CG / 20 WR | 2 CG / 22 WR |
+
+Findings:
+
+1. **The substrate lifts both models 2–13×** (hermes 9→18 / 5→21; qwen
+   5→19 / 2→26) and they **nearly converge** under it (18 vs 19, 21 vs
+   26). An 8 B and a 27 B reach *similar* grounded quality — the
+   substrate's thesis: grounding comes from retrieval+verification, not
+   parametric size, so the base model matters far less.
+2. **qwen-27B is the marginally stronger base** (26 vs 21 on stale, fewer
+   WRONG/JUDGE_ERROR), but the gap is small relative to the lift.
+3. **Bare-model failure styles diverge:** bare hermes **abstains** on
+   obscure entities (14/30 — honest "don't know"); bare qwen **fabricates**
+   (12 WRONG, 3 abstain — confidently wrong). The substrate fixes both.
+
+**Quality-per-dollar (the synthesis):** the substrate delivers ~18–26/30
+grounded-correct answers (vs 2–9 bare) for **$0.085–0.158 per 1,000
+queries** of GPU electricity (§5.3–5.4). The grounding is nearly free; the
+base model is largely interchangeable.
+
+Two honest caveats (a reviewer will press these):
+- **CORRECT_GROUNDED = grounded in the 2010 corpus, not factually current.**
+  On leader questions the substrate grounds the *2010* answer (Sarkozy for
+  France); bare models give the *current* answer (Macron) from parametric
+  memory → ungrounded → WRONG-vs-source. That is the substrate's contract
+  working: answer *from the corpus*, not from memory.
+- **The "stale" fixture stopped being post-corpus** once the 2010 corpus
+  was loaded — those leaders are now *in* corpus, so it measures grounding
+  (like numeral), not abstention. A truly post-2010 fabrication-bait set is
+  needed to re-test the honest-abstention path (§9).
+
+**Validity note (why this is trustworthy now):** an earlier qwen quality
+run showed arborist abstaining 30/30 — a **bug**, not epistemics: Qwen3
+defaults to thinking-on and burned its whole token budget on hidden
+`<think>` over the 20 K context, returning empty `content` → false
+UNGROUNDED. Fixed in commit `39c040c` (client defaults Qwen3
+`enable_thinking=False`). The energy numbers were unaffected (real
+inference happened; bench harnesses already passed the no-think flag via
+MODELS). The code judge was then strengthened (`2d31866`) to resolve
+HYBRID answers with a verified verbatim quote + on-topic subject + no
+unsourced numeric, cutting JUDGE_ERROR from 40 % to ~10 %.
+
 ## 6. Interpretation
 
 - **The substrate's cost is reading the evidence, not generating it.**
@@ -231,9 +296,12 @@ four-way confound (§2), dominated by 8 B vs 27 B. A grounded answer costs
 
 ## 8. Reproducibility
 
-- Instruments: `bench/watt_bench.py`, `bench/watt_probe.py`,
+- Energy instruments: `bench/watt_bench.py`, `bench/watt_probe.py`,
   `bench/watt_calibrate.py` (commits `5b1cbee` → `892d9ed`); math + band
   split + slope unit-tested in `tests/test_watt_cogs.py` (14 tests).
+- Quality (§5.5): `bench/control_ab.py` + `bench/judge_code.py` (code
+  judge, NLI-assisted, no quota; HYBRID-resolution strengthened in
+  `2d31866`, self-test 4/4). Qwen3 no-think fix `39c040c`.
 - Real token usage: `OpenAICompatibleClient.last_usage`.
 - Frozen substrate: `bench/stock_v1.py` (`assert_not_drifted`).
 - Raw per-cell power samples, window timestamps, and calibration sweeps
@@ -259,3 +327,9 @@ four-way confound (§2), dominated by 8 B vs 27 B. A grounded answer costs
 - **Per-model context-budget tuning** — qwen's quality-optimal budget is
   unmeasured (only Hermes was tuned); needed before any cross-model
   *quality*-per-joule claim.
+- **Truly-post-2010 fabrication-bait fixture** — the "stale" set is now
+  in-corpus (§5.5 caveat); a genuinely post-2010 set is needed to re-test
+  the honest-abstention path that the corpus upgrade erased.
+- **Stronger non-code judge for the residue** — ~10 % JUDGE_ERROR remains
+  (mostly bare-solo ungrounded answers); a SOTA blinded judge would grade
+  those (the open #000057 "which judge" decision).