diff --git a/docs/tickets/ticket-000027-canonical-projections-in-providence-cache.md b/docs/tickets/ticket-000027-canonical-projections-in-providence-cache.md index 0103c83..0be082a 100644 --- a/docs/tickets/ticket-000027-canonical-projections-in-providence-cache.md +++ b/docs/tickets/ticket-000027-canonical-projections-in-providence-cache.md @@ -8,9 +8,20 @@ Cache lookup before kernel execution; kernel result writes a row plus a `providence_canonical` audit event; second call to the same question hits the cache. **Audience:** fox + future blackops shifts. -**Hard constraint:** `providence_cache` 8-dim cache_key invariant -preserved. No schema changes. No new admissibility mode (the existing -`audit_mode = "CANONICAL_PROJECTION"` already round-trips). +**Hard constraint:** +- No new columns. +- No new tables. +- 8-dimensional cache_key invariant preserved. +- Existing CHECK constraints MUST be widened to admit: + - `audit_mode = 'CANONICAL_PROJECTION'` + - `verifier_method = 'canonical_projection'` + +(Original draft said "no schema changes / no new admissibility +mode" — corrected 2026-05-09 per the response review §4.1 and the +implementation appendix §9.1. The CHECK widening is a value-space +extension, handled by the existing `_rebuild_providence_cache_*` +migration pattern — not a column/table add. See §9.1 for the five +locations in `arborist/store.py` that need the new tokens.) --- @@ -183,13 +194,18 @@ kernel from `pi_star_ref + canonical_input`, asserts - `make stats` / `analyze-shards` count canonical rows. **Does NOT change:** -- `providence_cache` schema (no new columns). +- `providence_cache` schema (no new columns; CHECK constraints + are widened to admit the CANONICAL_PROJECTION + canonical_projection + tokens via the existing rebuild-table migration pattern). - 8-dim cache_key invariant. - `audit_events` table schema or chain protocol. - The 4-rung label ladder (CANONICAL_PROJECTION sits outside the STRICT/HYBRID/UNGROUNDED trichotomy and outside the lattice ladder — distinct admissibility class). - RAG path (zero impact on quote / claim_lattice / claim_lattice_pointer). +- `arborist canon` (stays transient — direct one-shot probe; this + ticket's persistence applies ONLY to the `arborist query` + preflight path). --- @@ -412,6 +428,14 @@ through RAG. No conflicts with open tickets. +**Forward link:** #000028 (multi-witness canonical agreement) +proposes a calibration / witness layer that runs ON TOP of this +ticket's persisted cache. #000028 explicitly depends on this +ticket landing first; it is NOT a Phase 2 of #000027 — it has its +own scope, threat model, and acceptance criteria. Keeping the +two separate prevents the persistence primitive from being +contaminated with LLM-fan-out concerns. + --- ## 7. Acceptance criteria diff --git a/docs/tickets/ticket-000028-multi-modality-witness.md b/docs/tickets/ticket-000028-multi-modality-witness.md index 317b1b3..6d63f33 100644 --- a/docs/tickets/ticket-000028-multi-modality-witness.md +++ b/docs/tickets/ticket-000028-multi-modality-witness.md @@ -421,3 +421,135 @@ Pre-existing canonical answers unaffected. - Reuses progress emitter from 2026-05-08 state-machine wiring. - Independent of #000027 (witness can ship before persistence; cache-leg is a no-op closure until #000027 lands). + +--- + +## 8. Post-MVP follow-ups + design notes + +Added 2026-05-09. MVP shipped in commit `656b573` +(`arborist/qa/witness.py` 394 LOC + `tests/test_witness.py` 354 +LOC + query.py / cli.py / Makefile integration; 28/28 tests pass). +Captures items from the 2026-05-09 review +(`response_ticket-000027-canonical-projections-in-providence-cache.txt`) +that are deferred or worth flagging. + +### 8.1 Witness vs modality terminology (review §2) + +The review argued that `kernel / cache / LLM` are not "modalities" +in the carrier-modality sense (text / image / audio / world-state). +They are **epistemic witness channels**: + +``` +carrier modality: text, image, code, table, audio, world state +witness channel: deterministic kernel, persisted cache, + LLM output, external tool, human validator +``` + +The ticket title and body use "modality" throughout, which is +fine as terminology fox chose — but a future reader looking for +cross-carrier (text↔image) work might land here by mistake. +Suggest a single clarifying line near §1: *"In this ticket +'modality' refers to a witness channel — independent epistemic +source — not to a carrier (text/image/audio). Cross-carrier +witness extensions are a later ticket once π* libraries support +non-text carriers."* + +Pure docs change; no code impact. + +### 8.2 Capital-ledger integration (#000020) — deferred + +Witness mode adds one full LLM call per canonical question +(§5 Risk 1). The MVP records the LLM latency on the witness +result dict but does NOT thread the cost into the capital +ledger. + +Recommend a follow-up commit to wire it: + +```python +if witness is not None: + capital_ledger.record( + kind="canonical_witness", + llm_tokens=witness.modalities["llm"].tokens or 0, + llm_latency_ms=witness.modalities["llm"].elapsed_ms, + agreement_label=witness.agreement_label, + ) +``` + +Without this, ForkScore (#000012) cannot compare witness-on vs +witness-off forks honestly, and operators have no mechanism to +cap witness-mode spend per shard / per session. ~15 LOC; queued +as Phase 2 of this ticket or a new sub-ticket. + +### 8.3 Sample-rate policy field — deferred + +§2.4 explicitly says sampling is out of scope. MVP fires witness +on every canonical question when the master flag is on. For +passive calibration data without full witness cost on every +question, a future bolt-on: + +``` +policy["canonical_witness_sample_rate"] = 0.05 # 0.0 = off +``` + +Captured here so it doesn't get re-discovered as a new +requirement when calibration-data hunger surfaces. + +### 8.4 Dependency relationship vs #000027 + +§7 says #000028 is "independent of #000027" — true for shipping +order, but worth disambiguating design coupling: + +- **Shipping**: MVP shipped before #000027 with the cache-leg + stubbed to return None. `STRICT-WITNESSED` is structurally + unreachable today — only `KERNEL-LLM-AGREE` / + `KERNEL-LLM-DIVERGED` / etc. fire. +- **Full witness value**: requires #000027 to land. The strongest + warrant in the matrix (3-of-3 byte-equal) is dormant until + the cache leg has real rows to look up. + +When #000027 lands, swap the no-op cache-lookup closure for the +real `lookup_canonical(qa_conn, ckey)` call — likely a one-line +change in the canonical branch of `arborist/qa/query.py`. + +### 8.5 5F / 5T bench-data integration + +Review §13 frames this as ongoing downstream value: every +witness-on canonical question is supervised calibration data. +Divergence events become natural fixtures for: + +- **5F Falsification**: `LLM-DIVERGED` rows are falsification + samples; metric `llm_canonical_error_rate`. +- **5F Feedback Loop**: divergence → improve prompt / + canonicalizer / extractor; metric + `feedback_efficiency = downstream_effect_count / Δcapital_cost`. +- **5T Truthtables**: LLM truth-table-row agreement with + `logic-kernel@v1`. +- **5T Transfer Learning**: does the LLM improve across canonical + shapes after calibration? +- **5S Semantics**: does free-form answer preserve invariant + meaning after canonicalization? + +Suggest a follow-up `make bench-witness-divergence` Makefile +target that dumps divergence audit events as bench fixtures. +Low-cost; defers naturally when fox decides to invest in +calibration-driven self-improvement. + +### 8.6 Threat model — caveats from review §7 + +The review flags two qualifying claims worth pinning here: + +- The strongest correct claim is **"no clean single-channel + adversarial path"** — NOT "no adversarial path." Coordinated + multi-channel compromise (e.g. shared canonicalizer bug + affecting kernel and LLM canonicalization simultaneously) is + still possible but requires strictly stronger adversary + assumptions. +- Future hardening: **dual-kernel witness** — + `arithmetic@v1_python` (Fraction + Decimal — current) plus + `arithmetic@v1_sympy` (independent implementation). If two + independent deterministic kernels agree alongside cache + LLM, + warrant is even stronger. Out of scope; captured for future + reference. + +These aren't action items; they're calibrations of the warrant +strength claims so future readers don't overstate them.