arborist/docs/stock-v1-config.md
russell@unturf.com 32aeb37086
fix(#000057): GPU COGS = generation only — retrieval/verify don't touch the card
fox 2026-05-21: 'we dont touch card during retrieval'. Retrieval + verify
are CPU/SQLite on the orchestrator; the GPU is idle (at the shared, always-
on model-resident floor) during them. So the substrate does NOT 'hold the
card' through its wall-clock — the gross window integral over-counts by
charging that always-on floor for the seconds we spend retrieving (energy
that exists regardless of the query; the card serves other traffic then).

Correct attribution: the GPU cost of a query is its GENERATION energy only
(the marginal — burst above the serving floor). Relabel: marginal is the
headline GPU COGS; gross is demoted to a reference 'window total, not
query-attributable'. Print + energy_cogs docstring + stock-v1-config doc
updated. No math change (marginal was already right) — this corrects the
narration. Measured qwen-nothink/4090: substrate GPU COGS is LOW
(claim_lattice $0.21/M-tok, quote $0.83) — its real overhead is latency
(CPU retrieval), not GPU watts.
2026-05-21 11:07:37 -04:00

6.2 KiB
Raw Permalink Blame History

STOCK V.1 — the frozen substrate under multi-day test

Pinned 2026-05-21 (fox). Before the multi-day GPU campaign, exactly one substrate-ON configuration is frozen so the treatment arm cannot silently drift across a 3-4 day run. Source of truth: bench/stock_v1.py.

What V.1 is

The full merkle-agi-dag reverse-RAG SQD / Prometheus-σ recursive- falsification substrate, non-reasoning and non-distributed.

STOCK V.1 is a two-cell config family (fox 2026-05-21): the substrate is characterized under both answer shapes, so answer_mode is a swept axis, each cell frozen with its own governance hash —

  • quote — prose with inline verbatim quotes; bench leader on raw lexical grounding (~0.54 strict).
  • claim_lattice — structured JSON claim-lattice (the four-rung ladder); what control_ab/control_sweep already drive, and the only shape with phase-3 reasoning-variant support built (~0.42 strict).

Everything else below is frozen identically across both modes.

knob value note
answer_mode swept: quote | claim_lattice two-cell family
temperature / top_p / max_tokens 0.1 / 1.0 / 512 from DEFAULT_QUERY_POLICY
repair_enabled False one-shot discipline, no self-heal reprompts
crosslang guard / translate / entity-mask OFF English-only; sandwich-MT is a separate capability
quantifier_guard_apply_caps False caps reported, not applied (dry-run)
quantifier_reject_broad False no preflight rejection
metacognition_* label-only block_on_contradiction=False, no verdict gating
soft_preflight_enabled False no extra LLM round-trip
claim_lattice_max_claims_per_answer 12 runaway guard
content_token_rules v2-acronym-aware verifier token rules
base_version wikitext-base-v1 prose normalization
chunker / norm / schema tok-512-v1 / norm-v1 / v9.8.0 versioned defaults
reasoning (inference layer) OFF reasoning variants are phase 3
mesh / multi-witness OFF distributed is the later fork

substrate-OFF = bare model, no retrieval, no verifier — control_ab arm A (question-only, neutral system prompt; gold text supplied identically to both arms per the §4b ruling).

The hashes (one per mode)

governance_policy_hash[quote]         = 5b6ca4c5e754e96b7e2e8af16a8948dec8d8b2a80b9304df21b1f6d494aade4e
governance_policy_hash[claim_lattice] = 036a4c79fd9d381a9ebf54094091c5f4e892793ae3b6dc56f70b8d4921c47455

These identify the campaign. bench/stock_v1.py snapshots the live DEFAULT_QUERY_POLICY, re-asserts the load-bearing pins, sets the mode's answer_mode, hashes the whole effective dict, and assert_not_drifted(mode) fails loudly if that hash ever changes — a mid-campaign edit to DEFAULT_QUERY_POLICY (a pinned knob or any other field the hash covers) stops the harness rather than quietly changing what "substrate-ON" means. Re-pinning is a deliberate fox-gated bump to V.2, never silent.

Phase 3 (reasoning) note. Reasoning refs (qwen-think) layer documented overrides onto claim_lattice inside control_sweep (cleared JSON stop sequences, 8192-token budget for the reasoning trace, empty-output self-heal). Those overrides change the policy, so a reasoning run has a different governance hash by construction — correct, since it is a different phase-3 config. The V.1 drift guard therefore covers only the non-reasoning frozen base; reasoning runs skip the assert.

Harnesses select the mode via --answer-mode (control_ab) / --arborist-answer-mode (control_sweep), default claim_lattice.

Campaign matrix

The substrate (above) is frozen; these axes are swept:

  1. hermes-8B, 3090 + 4090 — ~few days
  2. qwen-27B, 3090 + 4090
  3. reasoning variants — hermes-reasoning + qwen-reasoning

Models + reasoning flags live in bench/control_sweep.py:MODELS. Quality (CG%) is scored by control_sweep / control_ab; non-jaggedness by bench/jaggedness.py (#000060).

Cost axis — energy COGS (companion, #000057)

bench/watt_bench.py adds the cost side: GPU energy per completion-token, on each card. The card occupies DISTINCT measured states — idle, middle-idle (model resident, between requests), generation burst — split data-derived by watt_probe.classify_power_bands (largest-gap, no hardcoded watts). Crucially: retrieval + verification are CPU/SQLite work; the GPU is untouched during them (fox 2026-05-21). So:

  • marginal = THE GPU COGS of a query — generation energy above the serving floor, ÷ tokens. This is what the query actually causes on the GPU. Measured qwen-nothink/4090 (n=10, isolated): claim_lattice $0.21/M-tok, quote $0.83/M-tok, bare solo ~$0.50.65/M-tok.
  • gross = wall-clock window total, NOT query-attributable. It charges the always-on model-resident floor for the seconds spent in CPU-side retrieval — energy that exists regardless of the query (the card is free for other traffic then). Reference only, never the COGS.
  • dollarsjoules / 3.6e6 → kWh × $/kWh ÷ tokens for $/1k-tok against API pricing.
  • duty cycle exposes the shape: a substrate query generates only ~925 % of its wall-clock; the rest is CPU retrieval. The substrate's real overhead is latency, not GPU watts.

Every power state is MEASURED per (card, model, inference-server) at runtime — never hardcoded. P_warm_idle comes from the no-request idle baseline (--idle-baseline-s, default 5 s); P_gen from the burst during actual completions. Each card × model × server has its own profile. The only operator input is --price-per-kwh (default 0.33 USD/kWh, a configurable site rate). bench/watt_bench.py:energy_cogs emits gross + marginal $/1k-tok per cell and records the measurement window (window_start_unix/window_end_unix) for the cross-ref below.

Because the single-slot endpoints (hermes 3090 / vLLM, qwen 4090 / llama.cpp) serve real internet traffic and cannot be isolated, energy attribution cross-references bench/load_monitor.py queue-depth

  • req/s so organic-traffic bursts don't get counted as bench cost.