russell@unturf.com
32aeb37086
fox 2026-05-21: 'we dont touch card during retrieval'. Retrieval + verify are CPU/SQLite on the orchestrator; the GPU is idle (at the shared, always- on model-resident floor) during them. So the substrate does NOT 'hold the card' through its wall-clock — the gross window integral over-counts by charging that always-on floor for the seconds we spend retrieving (energy that exists regardless of the query; the card serves other traffic then). Correct attribution: the GPU cost of a query is its GENERATION energy only (the marginal — burst above the serving floor). Relabel: marginal is the headline GPU COGS; gross is demoted to a reference 'window total, not query-attributable'. Print + energy_cogs docstring + stock-v1-config doc updated. No math change (marginal was already right) — this corrects the narration. Measured qwen-nothink/4090: substrate GPU COGS is LOW (claim_lattice $0.21/M-tok, quote $0.83) — its real overhead is latency (CPU retrieval), not GPU watts.
120 lines
6.2 KiB
Markdown
120 lines
6.2 KiB
Markdown
# STOCK V.1 — the frozen substrate under multi-day test
|
||
|
||
**Pinned 2026-05-21 (fox).** Before the multi-day GPU campaign, exactly
|
||
one substrate-ON configuration is frozen so the treatment arm cannot
|
||
silently drift across a 3-4 day run. Source of truth:
|
||
[`bench/stock_v1.py`](../bench/stock_v1.py).
|
||
|
||
## What V.1 is
|
||
|
||
The full merkle-agi-dag reverse-RAG SQD / Prometheus-σ recursive-
|
||
falsification substrate, **non-reasoning** and **non-distributed**.
|
||
|
||
STOCK V.1 is a **two-cell config family** (fox 2026-05-21): the substrate
|
||
is characterized under both answer shapes, so `answer_mode` is a **swept
|
||
axis**, each cell frozen with its own governance hash —
|
||
|
||
- **`quote`** — prose with inline verbatim quotes; bench leader on raw
|
||
lexical grounding (~0.54 strict).
|
||
- **`claim_lattice`** — structured JSON claim-lattice (the four-rung
|
||
ladder); what `control_ab`/`control_sweep` already drive, and the only
|
||
shape with phase-3 reasoning-variant support built (~0.42 strict).
|
||
|
||
Everything else below is frozen identically across both modes.
|
||
|
||
| knob | value | note |
|
||
|---|---|---|
|
||
| `answer_mode` | **swept**: `quote` \| `claim_lattice` | two-cell family |
|
||
| `temperature` / `top_p` / `max_tokens` | `0.1` / `1.0` / `512` | from `DEFAULT_QUERY_POLICY` |
|
||
| `repair_enabled` | `False` | one-shot discipline, no self-heal reprompts |
|
||
| crosslang guard / translate / entity-mask | **OFF** | English-only; sandwich-MT is a separate capability |
|
||
| `quantifier_guard_apply_caps` | `False` | caps reported, not applied (dry-run) |
|
||
| `quantifier_reject_broad` | `False` | no preflight rejection |
|
||
| `metacognition_*` | label-only | `block_on_contradiction=False`, no verdict gating |
|
||
| `soft_preflight_enabled` | `False` | no extra LLM round-trip |
|
||
| `claim_lattice_max_claims_per_answer` | `12` | runaway guard |
|
||
| `content_token_rules` | `v2-acronym-aware` | verifier token rules |
|
||
| `base_version` | `wikitext-base-v1` | prose normalization |
|
||
| chunker / norm / schema | `tok-512-v1` / `norm-v1` / `v9.8.0` | versioned defaults |
|
||
| **reasoning** (inference layer) | **OFF** | reasoning variants are phase 3 |
|
||
| **mesh / multi-witness** | **OFF** | distributed is the later fork |
|
||
|
||
**substrate-OFF** = bare model, no retrieval, no verifier — `control_ab`
|
||
arm A (question-only, neutral system prompt; gold text supplied
|
||
identically to both arms per the §4b ruling).
|
||
|
||
## The hashes (one per mode)
|
||
|
||
```
|
||
governance_policy_hash[quote] = 5b6ca4c5e754e96b7e2e8af16a8948dec8d8b2a80b9304df21b1f6d494aade4e
|
||
governance_policy_hash[claim_lattice] = 036a4c79fd9d381a9ebf54094091c5f4e892793ae3b6dc56f70b8d4921c47455
|
||
```
|
||
|
||
These identify the campaign. `bench/stock_v1.py` snapshots the live
|
||
`DEFAULT_QUERY_POLICY`, re-asserts the load-bearing pins, sets the mode's
|
||
`answer_mode`, hashes the whole effective dict, and
|
||
`assert_not_drifted(mode)` fails **loudly** if that hash ever changes —
|
||
a mid-campaign edit to `DEFAULT_QUERY_POLICY` (a pinned knob *or any
|
||
other field the hash covers*) stops the harness rather than quietly
|
||
changing what "substrate-ON" means. Re-pinning is a deliberate fox-gated
|
||
bump to V.2, never silent.
|
||
|
||
**Phase 3 (reasoning) note.** Reasoning refs (qwen-think) layer
|
||
documented overrides onto `claim_lattice` inside `control_sweep` (cleared
|
||
JSON stop sequences, 8192-token budget for the reasoning trace,
|
||
empty-output self-heal). Those overrides change the policy, so a
|
||
reasoning run has a *different* governance hash by construction — correct,
|
||
since it is a different phase-3 config. The V.1 drift guard therefore
|
||
covers only the non-reasoning frozen base; reasoning runs skip the assert.
|
||
|
||
Harnesses select the mode via `--answer-mode` (`control_ab`) /
|
||
`--arborist-answer-mode` (`control_sweep`), default `claim_lattice`.
|
||
|
||
## Campaign matrix
|
||
|
||
The substrate (above) is **frozen**; these axes are **swept**:
|
||
|
||
1. **hermes-8B**, 3090 + 4090 — ~few days
|
||
2. **qwen-27B**, 3090 + 4090
|
||
3. **reasoning variants** — hermes-reasoning + qwen-reasoning
|
||
|
||
Models + reasoning flags live in `bench/control_sweep.py:MODELS`.
|
||
Quality (CG%) is scored by `control_sweep` / `control_ab`; non-jaggedness
|
||
by `bench/jaggedness.py` (#000060).
|
||
|
||
## Cost axis — energy COGS (companion, #000057)
|
||
|
||
`bench/watt_bench.py` adds the cost side: GPU energy per completion-token,
|
||
on each card. The card occupies DISTINCT measured states — idle,
|
||
middle-idle (model resident, between requests), generation burst — split
|
||
data-derived by `watt_probe.classify_power_bands` (largest-gap, no
|
||
hardcoded watts). **Crucially: retrieval + verification are CPU/SQLite
|
||
work; the GPU is untouched during them** (fox 2026-05-21). So:
|
||
|
||
- **marginal = THE GPU COGS of a query** — generation energy above the
|
||
serving floor, `÷ tokens`. This is what the query actually causes on
|
||
the GPU. Measured qwen-nothink/4090 (n=10, isolated): claim_lattice
|
||
**$0.21/M-tok**, quote **$0.83/M-tok**, bare solo ~$0.5–0.65/M-tok.
|
||
- **gross = wall-clock window total, NOT query-attributable.** It charges
|
||
the always-on model-resident floor for the seconds spent in CPU-side
|
||
retrieval — energy that exists regardless of the query (the card is
|
||
free for other traffic then). Reference only, never the COGS.
|
||
- **dollars** — `joules / 3.6e6 → kWh × $/kWh ÷ tokens` for `$/1k-tok`
|
||
against API pricing.
|
||
- **duty cycle** exposes the shape: a substrate query generates only
|
||
~9–25 % of its wall-clock; the rest is CPU retrieval. The substrate's
|
||
real overhead is **latency, not GPU watts.**
|
||
|
||
**Every power state is MEASURED per (card, model, inference-server) at
|
||
runtime — never hardcoded.** `P_warm_idle` comes from the no-request idle
|
||
baseline (`--idle-baseline-s`, default 5 s); `P_gen` from the burst
|
||
during actual completions. Each card × model × server has its own
|
||
profile. The only operator input is `--price-per-kwh` (default 0.33
|
||
USD/kWh, a configurable site rate). `bench/watt_bench.py:energy_cogs`
|
||
emits gross + marginal `$/1k-tok` per cell and records the measurement
|
||
window (`window_start_unix`/`window_end_unix`) for the cross-ref below.
|
||
|
||
Because the single-slot endpoints (hermes 3090 / vLLM, qwen 4090 /
|
||
llama.cpp) serve **real internet traffic and cannot be isolated**,
|
||
energy attribution cross-references `bench/load_monitor.py` queue-depth
|
||
+ req/s so organic-traffic bursts don't get counted as bench cost.
|