From 6d20aeb9107ef508cfa90fac9467c2ec52875baf Mon Sep 17 00:00:00 2001 From: "russell@unturf.com" Date: Sun, 10 May 2026 06:52:40 -0400 Subject: [PATCH] =?UTF-8?q?ticket=20#000028:=20=C2=A78.1=20terminology=20l?= =?UTF-8?q?ine=20+=20=C2=A78.2/=C2=A78.3=20status=20refresh?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit §8.1 — Witness-vs-modality terminology clarification (pure docs). The ticket title and §1 throughout use "modality" — operationally meaning kernel / cache / LLM. The 2026-05-09 review (response_ticket- 000027-canonical-projections-in-providence-cache.txt) flagged that a future reader looking for cross-carrier work (text↔image, audio, world-state) might land here by mistake, since "modality" in that sense means a different thing. Added a one-line note near §1 distinguishing *witness channel* (an independent epistemic source) from *carrier* (text / image / audio / world state), and pointing forward to a later ticket for cross- carrier extensions once π* libraries support non-text carriers. §8.2 + §8.3 — status refresh from "deferred" to "landed in 708aa45". Both follow-ups already shipped in commit 708aa45 ("fan-out: warrant ladder wiring · witness follow-ups · 5F Phase 1d", 2026-05-09 12:42 EDT) — pre-dating the e19aed8 close commit. The ticket file's §8.2 and §8.3 sections still carried the "deferred" labels because they were written from the pre-implementation design notes; refreshing both to point at the actual implementation in arborist/qa/query.py (lines 2140-2155 for sample-rate gating; 2237-2285 for the capital_ledger sidecar) and the corresponding test_witness.py coverage. Map-vs-territory hygiene: stale ticket markers are how future readers waste a session re-implementing something that's already there. No code change in this commit. 33/33 tests in test_witness.py continue to pass, all 7 shards chain-check clean. --- .../ticket-000028-multi-modality-witness.md | 74 ++++++++++++------- 1 file changed, 48 insertions(+), 26 deletions(-) diff --git a/docs/tickets/ticket-000028-multi-modality-witness.md b/docs/tickets/ticket-000028-multi-modality-witness.md index e90cd72..944767d 100644 --- a/docs/tickets/ticket-000028-multi-modality-witness.md +++ b/docs/tickets/ticket-000028-multi-modality-witness.md @@ -19,6 +19,13 @@ or per-policy. ## 1. Problem statement +> **Terminology note** (per §8.1): in this ticket *modality* refers +> to a **witness channel** — an independent epistemic source +> (deterministic kernel, persisted cache, LLM output) — not to a +> *carrier* (text / image / audio / world state). Cross-carrier +> witness extensions are a later ticket once π* libraries support +> non-text carriers. + Canonical projections (post-#000027) provide deterministic ground truth for arithmetic and propositional logic. The LLM, in contrast, is trained to produce plausible text and can hallucinate even on @@ -456,43 +463,58 @@ non-text carriers."* Pure docs change; no code impact. -### 8.2 Capital-ledger integration (#000020) — deferred +### 8.2 Capital-ledger integration (#000020) — landed in 708aa45 Witness mode adds one full LLM call per canonical question -(§5 Risk 1). The MVP records the LLM latency on the witness -result dict but does NOT thread the cost into the capital -ledger. +(§5 Risk 1). The MVP recorded LLM latency on the witness result +dict but did NOT thread the cost into the capital ledger. -Recommend a follow-up commit to wire it: +**Landed**: commit `708aa45` ("fan-out: warrant ladder wiring · +witness follow-ups · 5F Phase 1d", 2026-05-09 12:42 EDT). For +each FIRED witness (sample-rate gating respected), one +``capital_ledger`` row is written under +``op_type='canonical_witness'`` with the cost shape inherited +from the ``qa`` estimator (LLM is LLM). Estimator inputs include +``prompt_chars``, ``answer_chars``, ``llm_seconds``, +``agreement_label``, and ``pi_star_ref`` so retroactive cost +re-estimation can rebuild cleanly. Best-effort: ledger-write +failure never fails the query (sidecar discipline). -```python -if witness is not None: - capital_ledger.record( - kind="canonical_witness", - llm_tokens=witness.modalities["llm"].tokens or 0, - llm_latency_ms=witness.modalities["llm"].elapsed_ms, - agreement_label=witness.agreement_label, - ) -``` +Implementation in ``arborist/qa/query.py`` lines 2237-2285; +test coverage in ``tests/test_witness.py`` includes +``test_query_canonical_witness_records_capital_ledger`` which +verifies a row lands under the right op_type with the full +input blob. ForkScore (#000012) can now compare witness-on vs +witness-off forks honestly via ``capital_ledger.summary()``. -Without this, ForkScore (#000012) cannot compare witness-on vs -witness-off forks honestly, and operators have no mechanism to -cap witness-mode spend per shard / per session. ~15 LOC; queued -as Phase 2 of this ticket or a new sub-ticket. +### 8.3 Sample-rate policy field — landed in 708aa45 -### 8.3 Sample-rate policy field — deferred +§2.4 explicitly said sampling was out of scope for the MVP. MVP +fired witness on every canonical question when the master flag +was on, which made calibration-data collection an all-or-nothing +LLM-cost decision. -§2.4 explicitly says sampling is out of scope. MVP fires witness -on every canonical question when the master flag is on. For -passive calibration data without full witness cost on every -question, a future bolt-on: +**Landed**: commit `708aa45` ("fan-out: warrant ladder wiring · +witness follow-ups · 5F Phase 1d", 2026-05-09 12:42 EDT) added +the policy field: ``` -policy["canonical_witness_sample_rate"] = 0.05 # 0.0 = off +policy["canonical_witness_sample_rate"] = 0.05 # 0.0 = effectively off, 1.0 = always on ``` -Captured here so it doesn't get re-discovered as a new -requirement when calibration-data hunger surfaces. +Defaults to ``1.0`` (current always-on behavior preserved). +Operators wanting passive calibration set ``0.05`` to fire +witness on 5% of canonical questions while paying 5% of LLM +cost; ``0.0`` is effectively off. Gating uses +``random.random()`` for uniform distribution; clamped to +``[0, 1]``. The field folds into ``governance_policy_hash`` +naturally via the existing policy-hash machinery — flipping it +invalidates prior records as expected. + +Implementation in ``arborist/qa/query.py`` lines 2140-2155. +Test coverage in ``tests/test_witness.py`` includes +``test_query_canonical_witness_sample_rate_zero_skips`` and +``test_query_canonical_witness_sample_rate_one_always_fires``. ### 8.4 Dependency relationship vs #000027