feat(#000057): benchmark matrix doc (for David) + GPU wattage harness

Two deliverables for the cost/energy axis of the constraint optimizer.

docs/benchmark-matrix.md — shareable spec of the control experiment:
the question, fixture (386 office-holder Qs with corpus-vintage gold),
the 3-model × 3-framing × 2-arm matrix (18 cells), the verdict
vocabulary + two reads (accuracy vs grounding-fidelity), the
deterministic code judge + its Opus calibration, the results-so-far
table, and the NEW cost dimension (tokens / latency / GPU watts /
joules-per-answer measured per GPU tier). Self-contained — readable
cold by David.

bench/watt_bench.py — GPU wattage harness. Samples nvidia-smi
power.draw on the inference GPU while driving a small representative
subset, reports mean/peak watts, trapezoid-integrated joules,
joules-per-question, and joules-per-token. Tags the GPU
(--gpu-label 3090|4090) so the optimizer can compare hardware tiers.
Idle-baseline sampling separates load draw from idle. Does NOT grade
(energy is independent of correctness); saves answers + per-question
timing to JSONL for a later quality-per-joule pass via
score_with_code_judge.

Designed to run ON the GPU box (the orchestrator has no GPU; the
3090/4090 live on the inference boxes). Degrades gracefully when
nvidia-smi is absent (energy fields null) so it is testable anywhere.
Verified: PowerSampler graceful degradation + trapezoid integration
(synthetic 100->200->200W over 2s = 350 J, exact).

The headline cost finding the optimizer must weight: qwen-think
reasoning = 1300-3300 tokens/answer vs qwen-nothink ~50-100 (20-50x),
for a workload where arborist+qwen-nothink already lands 82% CG. The
energy numbers will quantify whether reasoning's premium is ever
justified — grounding-fidelity per joule, not per answer.
This commit is contained in:
russell@unturf.com 2026-05-20 12:33:58 -04:00
parent 14cc9c891a
commit 5260161e6f
No known key found for this signature in database
2 changed files with 443 additions and 0 deletions

139
docs/benchmark-matrix.md Normal file
View file

@ -0,0 +1,139 @@
# #000057 benchmark test matrix
Shareable spec of the control-experiment benchmark — what we measure,
how, and the cells in the matrix. Companion to the rolling results in
`docs/qa-modes-bench.md` (Addendum 8).
## The question
A small language model, asked "who is the president of France?", answers
with the *present-day* office-holder ("Emmanuel Macron") even when the
grounding corpus is a fixed ~2010 vintage (where the answer is Sarkozy).
The model isn't lying — it's reciting its training-data present — but it
fails to recognise it should defer to the supplied source, or abstain
when it has none. The matrix isolates **what fixes that**: model scale,
question framing, retrieval, or reasoning.
## Fixture
- **386 office-holder questions** (`bench/qa_questions_stale_map.json`),
each paired with a **fixed corpus-vintage gold article** (~2010-2011,
verified from the artefact). The gold is the single source of truth a
grounded answer must match.
- Adversarial by construction: the corpus answer (Sarkozy) differs from
the model's training-data present (Macron), so a model reciting its
prior is measurably *wrong against the source*.
## Axes
| axis | values | what it isolates |
|------|--------|------------------|
| **model** | `hermes` (Hermes-3-8B, vLLM) · `qwen-nothink` (Qwen3.6-27B, llama.cpp, reasoning off) · `qwen-think` (same, reasoning on) | scale (8B vs 27B) and reasoning |
| **framing** | `plain` ("who is the president of France?") · `source_relative` ("According to the reference knowledge base, who is …") · `as_of_corpus` ("As of 2010, who was …") | whether prompt framing alone fixes the drift |
| **arm** | `solo` (model alone — measures the parametric prior) · `arborist` (model + retrieval over the 2010 corpus, `answer_mode=claim_lattice`) | whether retrieval fixes it |
Full cross product = **3 models × 3 framings × 2 arms = 18 cells**, each
over up to 386 items. (The `arborist` arm currently runs `hermes` and
`qwen-nothink`; `qwen-think` is measurable but cost-prohibitive — see
"Cost dimension".)
## Metrics
Every (question, answer, gold) triple gets one verdict from a closed
vocabulary:
| verdict | meaning |
|---------|---------|
| **CORRECT_GROUNDED** (CG) | answer matches the gold source |
| **WRONG** (W) | gold states something the answer contradicts |
| **FABRICATED** (F) | answer asserts specifics absent from gold |
| **ABSTAINED** (A) | answer honestly declines ("I don't have that") |
| **JUDGE_ERROR** (JE) | judge could not classify deterministically |
Two reads of the same verdicts:
- **accuracy** — "did it say the corpus answer?" Penalises a model for
knowing the *current* office-holder. Reported but flagged as
misleading.
- **grounding-fidelity** (the defensible read) — "did the model do the
right conversational thing: answer when grounded, abstain when not?"
W + F are failures (confident ungrounded assertion); A is success on
the questions the model can't ground. **CG% and abstain% are the
headline numbers.**
## The judge
Deterministic, no LLM, no quota: `bench/judge_code.py`. Pipeline (first
hit decides): empty/no-gold guard → explicit-abstention regex →
short-answer entity-grounding fast path → NLI contradiction (θ=0.85) →
lexical verifier (quote/span/entity/paraphrase) → WRONG-vs-FABRICATED
tie-break on subject-in-gold. Claim-lattice JSON envelopes are unwrapped
to prose before grading so the `arborist` arm grades on equal terms with
`solo`.
The judge was calibrated against an Opus (SOTA) reference judge on the
records Opus could grade: agreement CG 13→47 %, WRONG 56→89 %, ABSTAINED
80→95 %. The deterministic verdict is a *proxy* for grounding (lexical +
NLI), not Opus-grade reading; JUDGE_ERROR residue is the natural input
to a later LLM-batch pass. NLI runs on a GPU (currently the 4090 `ai`
box); the rest is pure Python.
## Cost dimension (new — for the constraint optimizer)
The matrix has been scored on **quality** (CG%) but not yet on **cost**.
Adding the energy axis so the constraint optimizer can trade grounding
fidelity against power:
| cost metric | unit | source |
|-------------|------|--------|
| tokens / answer | completion tokens | API usage field |
| latency / answer | seconds | wall-clock per cell |
| **GPU power** | **watts (mean / peak)** | **`nvidia-smi power.draw`, sampled on the inference GPU during the cell** |
| **energy / answer** | **joules** | **∫ power dt / questions** |
Measured per GPU (**3090** and **4090**) so the optimizer knows the
energy cost of each config on each hardware tier. The wattage harness
(`bench/watt_bench.py`) samples `nvidia-smi` on the GPU box while driving
a small representative subset of the matrix.
**Why it matters:** reasoning chains are the headline cost finding —
`qwen-think` spends 1300-3300 completion tokens/answer vs `qwen-nothink`'s
~50-100 (**20-50× the token cost**), for a workload where
`arborist + qwen-nothink` already lands **82 % CG**. The optimizer should
weight that: grounding-fidelity *per joule*, not per answer.
## Results so far (CG%, all arms on the identical calibrated judge)
| arm / model | plain | source_relative | as_of_corpus |
|-------------|-------|-----------------|--------------|
| solo / hermes | 9 % | 5 % | 18 % |
| solo / qwen-nothink | 7 % | 0 % | 50 % |
| solo / qwen-think | 6 % | 5 % | 44 % |
| arborist / hermes (n=40) | 60 % | 62 % | 25 % |
| **arborist / qwen-nothink** | **82 %** | **65 %** | 50 % |
Headline findings: (1) **retrieval dominates** — no solo config
approaches the arborist arms; (2) **reasoning doesn't improve raw
correctness** (qwen-think/as_of 44 % vs nothink 50 %) and **breaks
honest-abstention** under source_relative framing (nothink abstains
98 %, think only 62 %); (3) **production answer: arborist + qwen-nothink,
plain framing, reasoning off — 82 % CG.** The cost axis will confirm
whether reasoning's energy premium is ever justified.
## Reproduce
```
# full matrix cell (one model, all framings, both arms)
python -m bench.control_sweep --judge code --models qwen-nothink \
--arborist-ref qwen-nothink --n 386 --arborist-n 386 --max-workers 1
# arborist-only (solo data already collected)
python -m bench.control_sweep ... --skip-solo
# wattage subset (run ON the GPU box)
python -m bench.watt_bench --models qwen-nothink --n 20 --gpu-label 4090
```
Artifacts land in `bench/qa_results/`; scorecards via
`bench/score_with_code_judge.py`; judge-vs-judge reconciliation via
`bench/analyze_judge_disagreement.py`.