ticket #000028: §8.1 terminology line + §8.2/§8.3 status refresh
§8.1 — Witness-vs-modality terminology clarification (pure docs). The ticket title and §1 throughout use "modality" — operationally meaning kernel / cache / LLM. The 2026-05-09 review (response_ticket- 000027-canonical-projections-in-providence-cache.txt) flagged that a future reader looking for cross-carrier work (text↔image, audio, world-state) might land here by mistake, since "modality" in that sense means a different thing. Added a one-line note near §1 distinguishing *witness channel* (an independent epistemic source) from *carrier* (text / image / audio / world state), and pointing forward to a later ticket for cross- carrier extensions once π* libraries support non-text carriers. §8.2 + §8.3 — status refresh from "deferred" to "landed in708aa45". Both follow-ups already shipped in commit708aa45("fan-out: warrant ladder wiring · witness follow-ups · 5F Phase 1d", 2026-05-09 12:42 EDT) — pre-dating thee19aed8close commit. The ticket file's §8.2 and §8.3 sections still carried the "deferred" labels because they were written from the pre-implementation design notes; refreshing both to point at the actual implementation in arborist/qa/query.py (lines 2140-2155 for sample-rate gating; 2237-2285 for the capital_ledger sidecar) and the corresponding test_witness.py coverage. Map-vs-territory hygiene: stale ticket markers are how future readers waste a session re-implementing something that's already there. No code change in this commit. 33/33 tests in test_witness.py continue to pass, all 7 shards chain-check clean.
This commit is contained in:
parent
6ba833878d
commit
6d20aeb910
1 changed files with 48 additions and 26 deletions
|
|
@ -19,6 +19,13 @@ or per-policy.
|
|||
|
||||
## 1. Problem statement
|
||||
|
||||
> **Terminology note** (per §8.1): in this ticket *modality* refers
|
||||
> to a **witness channel** — an independent epistemic source
|
||||
> (deterministic kernel, persisted cache, LLM output) — not to a
|
||||
> *carrier* (text / image / audio / world state). Cross-carrier
|
||||
> witness extensions are a later ticket once π* libraries support
|
||||
> non-text carriers.
|
||||
|
||||
Canonical projections (post-#000027) provide deterministic ground
|
||||
truth for arithmetic and propositional logic. The LLM, in contrast,
|
||||
is trained to produce plausible text and can hallucinate even on
|
||||
|
|
@ -456,43 +463,58 @@ non-text carriers."*
|
|||
|
||||
Pure docs change; no code impact.
|
||||
|
||||
### 8.2 Capital-ledger integration (#000020) — deferred
|
||||
### 8.2 Capital-ledger integration (#000020) — landed in 708aa45
|
||||
|
||||
Witness mode adds one full LLM call per canonical question
|
||||
(§5 Risk 1). The MVP records the LLM latency on the witness
|
||||
result dict but does NOT thread the cost into the capital
|
||||
ledger.
|
||||
(§5 Risk 1). The MVP recorded LLM latency on the witness result
|
||||
dict but did NOT thread the cost into the capital ledger.
|
||||
|
||||
Recommend a follow-up commit to wire it:
|
||||
**Landed**: commit `708aa45` ("fan-out: warrant ladder wiring ·
|
||||
witness follow-ups · 5F Phase 1d", 2026-05-09 12:42 EDT). For
|
||||
each FIRED witness (sample-rate gating respected), one
|
||||
``capital_ledger`` row is written under
|
||||
``op_type='canonical_witness'`` with the cost shape inherited
|
||||
from the ``qa`` estimator (LLM is LLM). Estimator inputs include
|
||||
``prompt_chars``, ``answer_chars``, ``llm_seconds``,
|
||||
``agreement_label``, and ``pi_star_ref`` so retroactive cost
|
||||
re-estimation can rebuild cleanly. Best-effort: ledger-write
|
||||
failure never fails the query (sidecar discipline).
|
||||
|
||||
```python
|
||||
if witness is not None:
|
||||
capital_ledger.record(
|
||||
kind="canonical_witness",
|
||||
llm_tokens=witness.modalities["llm"].tokens or 0,
|
||||
llm_latency_ms=witness.modalities["llm"].elapsed_ms,
|
||||
agreement_label=witness.agreement_label,
|
||||
)
|
||||
```
|
||||
Implementation in ``arborist/qa/query.py`` lines 2237-2285;
|
||||
test coverage in ``tests/test_witness.py`` includes
|
||||
``test_query_canonical_witness_records_capital_ledger`` which
|
||||
verifies a row lands under the right op_type with the full
|
||||
input blob. ForkScore (#000012) can now compare witness-on vs
|
||||
witness-off forks honestly via ``capital_ledger.summary()``.
|
||||
|
||||
Without this, ForkScore (#000012) cannot compare witness-on vs
|
||||
witness-off forks honestly, and operators have no mechanism to
|
||||
cap witness-mode spend per shard / per session. ~15 LOC; queued
|
||||
as Phase 2 of this ticket or a new sub-ticket.
|
||||
### 8.3 Sample-rate policy field — landed in 708aa45
|
||||
|
||||
### 8.3 Sample-rate policy field — deferred
|
||||
§2.4 explicitly said sampling was out of scope for the MVP. MVP
|
||||
fired witness on every canonical question when the master flag
|
||||
was on, which made calibration-data collection an all-or-nothing
|
||||
LLM-cost decision.
|
||||
|
||||
§2.4 explicitly says sampling is out of scope. MVP fires witness
|
||||
on every canonical question when the master flag is on. For
|
||||
passive calibration data without full witness cost on every
|
||||
question, a future bolt-on:
|
||||
**Landed**: commit `708aa45` ("fan-out: warrant ladder wiring ·
|
||||
witness follow-ups · 5F Phase 1d", 2026-05-09 12:42 EDT) added
|
||||
the policy field:
|
||||
|
||||
```
|
||||
policy["canonical_witness_sample_rate"] = 0.05 # 0.0 = off
|
||||
policy["canonical_witness_sample_rate"] = 0.05 # 0.0 = effectively off, 1.0 = always on
|
||||
```
|
||||
|
||||
Captured here so it doesn't get re-discovered as a new
|
||||
requirement when calibration-data hunger surfaces.
|
||||
Defaults to ``1.0`` (current always-on behavior preserved).
|
||||
Operators wanting passive calibration set ``0.05`` to fire
|
||||
witness on 5% of canonical questions while paying 5% of LLM
|
||||
cost; ``0.0`` is effectively off. Gating uses
|
||||
``random.random()`` for uniform distribution; clamped to
|
||||
``[0, 1]``. The field folds into ``governance_policy_hash``
|
||||
naturally via the existing policy-hash machinery — flipping it
|
||||
invalidates prior records as expected.
|
||||
|
||||
Implementation in ``arborist/qa/query.py`` lines 2140-2155.
|
||||
Test coverage in ``tests/test_witness.py`` includes
|
||||
``test_query_canonical_witness_sample_rate_zero_skips`` and
|
||||
``test_query_canonical_witness_sample_rate_one_always_fires``.
|
||||
|
||||
### 8.4 Dependency relationship vs #000027
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue