$0.10/1k-q overstated the qwen-27B case. Honest range: ~$0.07-0.16 per
1,000 queries of GPU electricity — hermes-8B $0.07-0.09 (under a dime),
qwen-27B $0.12-0.16 (over a dime; claim_lattice dearer than quote from
more prefilled context). Fixes the §5.4 'either rig' claim.
The report is now cost AND value (quality-per-dollar), not cost-only.
§5.5 quality delta: substrate-vs-solo (code judge, n=30) on numeral +
stale fixtures, hermes-8B vs qwen-27B-nothink. Substrate lifts both
2-13x and they nearly converge (hermes 18/21, qwen 19/26 grounded-
correct) — grounding comes from retrieval+verification, not parametric
size, so the base model matters far less. Bare-model failure styles
diverge (hermes abstains, qwen fabricates). Quality-per-dollar: ~18-26/30
grounded for $0.085-0.158/1k-q.
Honest caveats recorded: CORRECT=grounded-in-2010-corpus (not current);
the stale fixture is no longer post-corpus; the qwen thinking bug (39c040c)
that voided the first run and the judge strengthening (2d31866). §1
reframed; §8 repro + §9 next updated (post-2010 fixture, SOTA judge for
residue).
fox 2026-05-21: two corpora exist (2003-05-16 + 2010); the 2010 one is
loaded in ~/.arborist/shards (verified: 866K docs/shard, has Barack
Obama / YouTube articles a 2003 dump can't). CLAUDE.md Live-endpoints and
the energy-COGS report now state 2010; fabrication-bait fixtures target
post-2010 events. Historical closed-ticket prose left as point-in-time
records.
- §2 now a two-rig table: 4090/Qwen-27B-Q4_K_XL/llama.cpp (isolated) vs
3090/Hermes-8B-FP8-Dynamic/vLLM (live/public). Spells out that the
cross-model rate gap confounds FOUR variables (params, quant/precision,
engine, GPU) — not '8B vs 27B' alone.
- §5.4 hermes results under live traffic. Method finding: the watt_bench
window integral is unusable on a contended card, but the slope
calibration survives clean (cancels the variable shared-load baseline)
— so on shared/public cards use watt_calibrate, not the window integral.
Rates: hermes 0.109/4.40 J/tok vs qwen 0.175/6.16; decode 35-40x prefill.
- Per-query DOLLARS both rigs @$0.33/kWh: a grounded substrate answer is
<$0.10 per 1000 queries (hermes $0.085, qwen $0.158); hermes ~half qwen.
- §5.2 budget confound corrected to flag Hermes-tuning honestly (was
rationalized). §9: fixed-budget apples-to-apples re-run + per-model
budget tuning added as next steps.
Full report for Dav1d: qwen-nothink on the dedicated 4090. Headline —
prefill 0.175 J/input-tok ($0.016/M) vs decode 6.157 J/output-tok
($0.564/M), decode 35x dearer per token. The substrate prefills ~6.6k
input tok/query (vs solo ~52, 127x): ~67% of its GPU energy is reading
the retrieved context, not generating. Per-query 1719 J (substrate-CL)
vs 95 J (solo) = $0.16 vs $0.009 per 1k queries; calibration predicts
measured within ~5%. Real cost is CPU latency, not GPU watts.
Documents the full methodology + the corrections that got here
(contamination -> isolation; blended mean-W -> measured states; gross
-> not-attributable; per-token -> separated input/output; len//4 ->
real usage), threats to validity, reproducibility (commits, config hash,
persisted samples), and next rigs (3090+Hermes, reasoning, prefill-cache
study).
