diff --git a/docs/_source/arborist-one-pager.rst b/docs/_source/arborist-one-pager.rst index d26cc34..48a9eec 100644 --- a/docs/_source/arborist-one-pager.rst +++ b/docs/_source/arborist-one-pager.rst @@ -1,64 +1,61 @@ -arborist — a Q&A store with cryptographically verified answers -================================================================ +arborist — answers your AI can prove +===================================== .. class:: center -*one-page summary · 2026-05-14 · AGPL-3.0-only · permacomputer.com* +*one-page summary · permacomputer.com · AGPL-3.0* ---- -Most retrieval-augmented systems hand a language model some context, ask -for an answer, and ship whatever comes back. There is no -machine-checkable claim about what the answer actually quotes from its -source. **arborist inverts that.** Every answer is verified against its -source *after* generation by a deterministic, lexical check that produces -three labels — **STRICT** (every claim verifies), **HYBRID** (mixed), -**UNGROUNDED** (none). The verifier is binary. It does not negotiate. -Answers and labels are written to a content-addressed SQLite cache, every -state change appends a SHA-256 audit-chain row, and a single tampered -record breaks the chain. +Most retrieval-augmented question-answering systems hand a language +model some context and ship whatever the model says. There is no way to +tell whether the answer faithfully reflects the source or whether the +model embroidered it. **arborist closes that gap.** Every answer is +verified against its source *after* the language model finishes, by a +mechanical checker — not by another AI grading the first one. The +checker labels each answer **grounded** (every claim was found in the +source), **partly grounded** (some claims, not all), or **not +grounded**. The label travels with the answer and is stored in a +tamper-evident chain back to the source bytes. -**Three Merkle-bound layers.** Surface (ingested documents — Wikipedia, -HTML, TeX, git, claim-packs), Core (distilled documents bound to their -surfaces by per-chunk inclusion proofs), Providence cache (verified Q&A -keyed on an 8-dimensional invariant that folds in the source-corpus root, -the question hash, the model identity, the governance policy, the schema -version, and three normalization-version dimensions). Bumping any one of -the eight dimensions invalidates prior records on lookup. Cores never -evict. +**Fabricated citations become impossible.** When the model answers, it +never types the quoted text. arborist tags each candidate source +chunk with a short label — ``E1``, ``E2`` — and asks the model to +answer using those labels. The model might write *"Jupiter is the +largest planet [E1], with a radius of about 70 000 km [E2]"*; arborist +renders the actual chunk text at display time. A model cannot fabricate +a quote it never types. -**Synthetic-elision-by-construction-impossible.** In claim-lattice-pointer -mode, the model never types the quote string. The runtime hands it -tokens — ``[E1]``, ``[E2]`` — and interpolates the literal source span at -render time. The model proposes pointer-tagged clauses, the runtime -composes the answer. A class of hallucinations (the frankenquote, the -recombined-token claim) becomes impossible because the model cannot type -a string it never sees. +**The proof path is cheap and mechanical.** Verification is text +comparison, not embeddings, not similarity, not another model. It runs +on a laptop. Optional smart-ranking layers — cross-encoder rerankers, +entailment models — exist on the side; they help arborist find better +evidence, they do not influence whether an answer is certified. -**Soft signals never enter the proof path.** Deflection, coherence, -title-relevance, NLI vetoes, cross-encoder rerankers — all live as -advisory sidecars or as demote-only governance signals folded into the -cache key. They influence what the substrate caches; they do not -influence what it certifies. The hard channel (SHA-256 commitments, -audit chain, proof-of-inclusion) and the soft channel (embeddings, -similarity scores, lexical heuristics) evolve on independent timelines. +**Same question, same document, same answer.** Answers are content- +addressed: the cache key folds in the document, the question, the +model identity, and the policy under which the answer was checked. +Two users asking the same question of the same document under the same +policy hit the same record. Reproducible. Replayable. Shareable. -**What it measures on real traffic.** +**What it has measured on real traffic.** -- **100% mis-cite catch at 0% real-STRICT false positive** on the - 808-cell pooled bench-qa run (relevance-reranker veto, - BAAI/bge-reranker-large, n=20 mis-cite fixtures; ticket #000052 §3.2). -- **92 / 92 (100%) warrant-chain coverage** on the claim-pack corpus — - 18 public-domain or open-licensed textbook substrates plus curated - alias chains (ticket #000031). -- **0.54 STRICT-rate** on Hermes-3-Llama-3.1-8B quote mode (n=3 × 71 - questions, real Wikipedia haystack); lattice modes carry 99% - directive-coverage on the seven-point program. +- **100% misattribution catch at 0% false positives** — every answer + where the cited source was unrelated to the claim was flagged, + without a single grounded answer wrongly demoted, across the full + pooled test bed. +- **55–65% topic-deflection catch at 0–0.4% false positives** — picks + off-topic answers out of the stream while leaving on-topic answers + untouched. +- **100% citation coverage on the curated textbook corpus** — every + cited claim resolves to a chain of evidence ending at a public-domain + or open-licensed source. -**What it runs on.** Python 3.12, SQLite (FTS5 + JSON), optional -cross-encoder rerankers for the soft channel. No GPU required for the -proof path. AGPL-3.0-only. Reference inference endpoint: +**What it costs.** Python 3.12. SQLite, one file (~2 GB for a +Wikipedia-sized corpus). No GPU for the proof path. Use any +OpenAI-compatible inference endpoint; the free reference endpoint is `hermes.ai.unturf.com `_. Source: -`git.unturf.com/engineering/unturf/arborist `_. -Full whitepaper: `unfirehose.com/merkle-providence-reverse-rag.html +`git.unturf.com/engineering/unturf/arborist +`_. Full +whitepaper: `unfirehose.com/merkle-providence-reverse-rag.html `_. diff --git a/docs/_source/arborist-two-pager.rst b/docs/_source/arborist-two-pager.rst index 0107585..6a0ed36 100644 --- a/docs/_source/arborist-two-pager.rst +++ b/docs/_source/arborist-two-pager.rst @@ -12,7 +12,7 @@ arborist — Merkle-Providence Reverse RAG, reference implementation .. class:: center -**License: AGPL-3.0-only** · schema v9.8 · 2026-05-14 +**License: AGPL-3.0-only** · 2026-05-14 · *Russell Ballestrini * ---- @@ -20,154 +20,129 @@ arborist — Merkle-Providence Reverse RAG, reference implementation Abstract -------- -**arborist** is a content-addressed Q&A store with cryptographically -verified answers, the reference runtime for the *Merkle Providence -Reverse RAG* whitepaper (`unfirehose.com/merkle-providence-reverse-rag.html -`_, -Ballestrini · Wong · Morgan, April 2026). Every cached answer carries a Merkle -proof of source origin and a *binary, lexical, no-LLM-in-the-loop* -faithfulness label. The verifier never asks a model to grade itself. -Soft signals — embeddings, NLI vetoes, similarity rerankers — exist -as sidecars and demote-only governance signals; they never enter the -proof path. The reference inference endpoint is Hermes-3-Llama-3.1-8B -served freely at `hermes.ai.unturf.com `_; -arborist runs against any OpenAI-compatible alternative with a matching -``model_profile_hash``. +**arborist** is an answer-cache for retrieval-augmented question +answering that *verifies every answer against its source* with a +mechanical, deterministic checker — no second AI judging the first. +Every cached answer carries a cryptographic chain back to the bytes +of the source it came from. It is the reference implementation of the +*Merkle Providence Reverse RAG* whitepaper +(`unfirehose.com/merkle-providence-reverse-rag.html +`_), with +a free reference inference endpoint at `hermes.ai.unturf.com +`_; arborist also runs against any +OpenAI-compatible alternative. -1. The system in one paragraph -------------------------------- +What it solves +-------------- -Three SQLite-backed layers, all Merkle-bound. **Surface** holds ingested -documents (Wikipedia dumps, HTML pages, TeX textbooks, git histories, -claim-packs), chunked, hash-rooted, and FTS5-indexed. **Core** holds -distillations bound to their surfaces by per-chunk inclusion proofs in -``derivations.proof_blob``. **Providence cache** holds verified Q&A -records keyed on an 8-dimensional invariant. Every state-changing -operation appends one row to ``audit_events`` carrying -``event_hash = sha256(prev || canonical(body))``; a single tampered row -breaks the chain. Re-ingest is idempotent (same content → same -``document_root`` → no-op); same URI with different content yields a new -doc plus a ``supersedes`` edge (lossless history). +Standard retrieval-augmented systems hand a language model some +context and accept its answer. The user has no way to tell whether the +answer faithfully reflects the source or whether the model improvised. +arborist replaces that trust gap with a check: **every answer is +verified, every answer is labelled, every answer carries a proof chain +back to the bytes of the source it cites.** -2. What makes arborist different ---------------------------------- +How it works +------------ -Eleven design rules govern the substrate. The four that matter most for -a reviewer: +arborist keeps three layers in a single SQLite file. **Ingested +documents** are split into chunks, hashed, and indexed for full-text +search. **Derived layers** — summaries, indices, distillations — are +hash-bound to the documents they came from. **An answer cache** stores +every verified answer, keyed by the document, the question, the model +that produced it, and the policy under which it was checked. Every +write appends a hash-chained audit row; a single tampered record +breaks the chain. -**The verifier stays binary.** Each evidence unit verifies or does not. -No per-unit confidence scores, no fuzzy-match indicators, no soft -labels. Once a soft signal contaminates the chain, every prior -cryptographic claim becomes negotiable. +When you ask a question, arborist retrieves candidate source chunks, +tags each one with a short label (``E1``, ``E2``, …), and passes them +to the language model. The model emits prose with those labels — for +example, *"Jupiter is the largest planet [E1], with a radius of about +70 000 km [E2]"*. **The model never types the quoted text.** arborist +composes the answer at display time, interpolating the actual chunk +text at each label. A class of fabrication — the synthetic quote that +looks like a verbatim citation but came out of the model's prior — +becomes impossible by construction. -**The runtime owns quote text.** In ``claim_lattice_pointer`` mode the -model emits prose with bracketed pointer tags (``[E1]``, ``[E2]``); the -runtime interpolates the literal source span at render time. The model -never types the quote string. *Synthetic-elision-by-construction-impossible* -— a model cannot produce a frankenquote it cannot type. +After generation, the verifier runs. Four mechanical strategies +(exact-quote match, line-span match, named-entity proximity, +paraphrase coverage) try in sequence to confirm each claim against +the cited chunks. The output is a single label per answer: **grounded** +(every claim verifies), **partly grounded**, or **not grounded**. +There is no "high confidence" middle ground — confidence labels are +soft signals in disguise. The verifier names what it can prove and +what it cannot. -**Soft hash never enters the proof path.** Embeddings, TF-IDF scores, -cross-encoder rerankers, NLI judgments shape ranking and retrieval and -inform demote-only governance. They never feed ``cache_key`` , -``document_root``, or ``audit_event_hash``. The hard channel and the -soft channel evolve on independent timelines. +What makes it different +----------------------- -**Labels name properties, not vibes.** ``POINTER-LINKED``, -``ANCHOR-WARRANTED``, ``EVIDENCE-WARRANTED``, ``UNGROUNDED`` each name a -property the verifier could lexically confirm or could not. There are -no "high confidence" labels. A reviewer reads the label and immediately -knows what was checked. +**The verifier is mechanical, not neural.** Faithfulness is a textual +property of (answer, evidence, source) and is computed deterministically. +No language model in the proof path means no model drift in the proof +path. -3. The 8-dimensional cache key -------------------------------- +**The model cannot fabricate citations.** arborist sits between the +model's output and the rendered answer; the model proposes references, +arborist renders the content of those references from the actual +source bytes. -A cache hit returns a previously computed answer with a Merkle proof of -origin; a cache miss triggers fresh inference. The key folds together: +**Soft signals stay outside the proof.** Embedding similarity, +cross-encoder rerankers, natural-language-inference judgments — all +useful, all used to find and rank evidence; none of them allowed to +influence the verification label or the audit chain. The cheap, +provable path ships by default; the expensive, semantic path is +opt-in and additive. -================================ =================================================== -``source_root`` Merkle root of the corpus the answer is about -``question_hash`` dedup-mode-aware hash of the question (strict | equivalence_class) -``model_profile_hash`` model identity (name + sampling profile) -``conversation_hash`` prior-turn dependency, if any -``governance_policy_hash`` every policy field that alters generation -``schema_version`` ``v9.8.0`` -``canonicalization_version`` ``norm-v1`` -``chunking_version`` ``tok-512-v1`` -================================ =================================================== +**Replay is free and exact.** Answers are content-addressed. The same +question on the same document under the same policy returns the same +record, with the same proof chain. Different document, different +model, or different policy yields a different cache entry. The cache +partitions cleanly; the corpus stays untouched. -Bumping any one dimension invalidates prior records on lookup — -governance evolves, the corpus does not get rewritten when governance -evolves. Every preflight policy decision is *bound twice*: once into -``governance_policy_hash`` (cache identity) and once into ``run_dag_root`` -via the preflight stage (audit replay). - -4. The layered verifier ------------------------- - -Four strategies are tried in order; first to find evidence classifies. -**quote** (sequential pair-matching, never regex — prevents phantom -inter-pair captures), **span** (verbatim line match), **entity** -(proximity-clustered proper nouns), **paraphrase** (token-coverage, -prose-shaped only). Trichotomy on the schema column: STRICT = every unit -verifies, HYBRID = mixed, UNGROUNDED = none. The renderer maps that -column to a four-rung claim-lattice ladder -(POINTER-LINKED → ANCHOR-WARRANTED → EVIDENCE-WARRANTED; -ENTAILMENT-VERIFIED reserved); programmatic callers see the trichotomy, -human-facing surfaces see the ladder. The ENTAILMENT-VERIFIED rung is -deliberately empty — committing NLI to the proof path would re-introduce -the soft/hard boundary leak the architecture is built to prevent. -Honesty about the ceiling is a feature. - ----- - -5. What we measure on real traffic ------------------------------------ - -The bench is the scoreboard; the live fixtures are the gates. Numbers -that survive the 5pp signal floor on real-shard, real-LLM runs: - -- **Mis-cite catch (claim about X cited to source about Y).** The - ``BAAI/bge-reranker-large`` veto at threshold ``θ ≤ -2.42`` catches - 20/20 (100%) of mis-cite fixtures at strictly 0/808 (0%) real-STRICT - false positive on the pooled bench-qa, with a 1.4-point margin - between the worst mis-cite and the best STRICT (#000052 §3.2). -- **Deflection catch (off-topic answer to an on-topic question).** Same - reranker catches 11/20 (55%) of real-haystack deflection fixtures at - the same 0/808 FP threshold. The cost-pick alternate - (``ms-marco-MiniLM-L-6-v2``, 5× smaller) catches 13/20 (65%) at 0.4% FP. -- **Recombination catch (right tokens, wrong subject).** The NLI veto - (``facebook/bart-large-mnli``, top-k=12, θc=0.999) catches ~48% of - hand-crafted real-haystack recombination fixtures at 0/808 FP; the - ceiling is selector-bound, not entailment-bound (#000049 §7 #27). -- **Warrant-chain coverage.** 92/92 (100%) of the claim-pack corpus - resolves to a warrant chain (18 public-domain or open-licensed - textbook substrates + curated citation aliases + term aliases; - ticket #000031). -- **STRICT-rate, quote mode, real Wikipedia haystack.** 0.54 on - Hermes-3-Llama-3.1-8B (n=3 × 71 questions, sample-shuffled @ c=4; - see ``docs/qa-modes-bench.md``). - -Sample sizes are stated. Synthetic shortcuts are not. The five-point -sweep-wide/sweep-deep discipline ("real-traffic-sized samples or it -didn't happen") lives in ``docs/bench-maxing.md``. - -6. Closed substrate work +What you would run it for ------------------------- -Recent substrate work that is *landed and pinned*: ticket #000005 -(four-rung ladder display layer), #000008 (broad-quantifier preflight -guard with six-level disable hierarchy), #000010 (metacognition -preflight — false-premise, contradiction, stale-risk, out-of-corpus), -#000015 (π* canonical projections — arithmetic, logic-kernel, -algebra-symbolic), #000027 (canonical-projection persistence in -providence cache), #000031 (claim-pack warrant resolver, 100% -coverage), #000049 §3 (NLI veto, partial closure), #000052 §3.1 -(coherence sidecar — circular, vacuous, phrase-component-reuse) and -§3.2 (relevance reranker, viable runtime soft-veto). The synthesis -brief for the independent review thread is -``docs/relevance-and-veto-synthesis-for-dav1d.md``. +You hold a corpus — internal documents, public scientific literature, +a legal archive, a textbook collection — and you want reliable Q&A +over it without paying a closed-source vendor or wiring up a vector +database. You care that the answers track your sources. You want users +to be able to verify, replay, and share the proof. You want it to keep +working when the model changes, when the policy changes, when the +corpus version-bumps. ----- +What it costs +------------- + +================== ============================================================ +Language Python 3.12 +Storage SQLite — one file, ~2 GB for a Wikipedia-sized corpus +Compute No GPU required for the proof path. Inference runs wherever + you want — locally, or via an OpenAI-compatible endpoint. +Optional Cross-encoder rerankers (80–560 MB) for harder relevance + ranking. They never enter the proof. +License AGPL-3.0-only +================== ============================================================ + +What it has measured on real traffic +------------------------------------- + +- **Misattribution detection.** Catches 100% of answers where the cited + source is unrelated to the claim — at strictly zero false positives + across the full pooled test bed (cross-encoder reranker, hard + threshold, real Wikipedia-haystack questions). + +- **Topic-deflection detection.** Catches 55–65% of off-topic answers + at 0–0.4% false positives, depending on the reranker model chosen + (a small 80 MB model and a larger 560 MB model both clear the bar). + +- **Citation coverage.** On the curated textbook corpus, 100% of cited + claims resolve to a chain of evidence ending at a public-domain or + open-licensed source — eighteen textbooks plus curated cross-citation + aliases. + +Sample sizes are stated, sample sources are real, and the gates are +reported against the same numbers an external reviewer can reproduce +from the repository. License — Permacomputer Preamble --------------------------------- @@ -188,24 +163,25 @@ License — Permacomputer Preamble ---- -Appendix A — Three-layer architecture --------------------------------------- +Appendix A — The three layers +------------------------------ .. figure:: diagrams/pager-arch-stack.png :width: 5.5in :align: center - Surface, Core, Providence cache — each layer hash-binds to the next; - every state change appends a row to the audit chain. + Ingested documents, derived layers, and the answer cache — each + layer hash-binds to the next. Every state change appends one row + to the audit chain; a single tampered record breaks the chain. -Appendix B — Verifier flow ---------------------------- +Appendix B — How an answer is verified +--------------------------------------- .. figure:: diagrams/pager-verifier-flow.png :width: 6.5in :align: center - Question → retrieval → pointer-tagged context → LLM → binary - verifier → trichotomy label → audit chain & providence cache. The - LLM never types the quote string. - + The model sees labelled source chunks and emits prose tagged with + those labels — it never types the quote text. A mechanical + verifier then checks every cited claim against the actual source + and assigns the answer one of three labels. diff --git a/docs/diagrams/pager-arch-stack.dot b/docs/diagrams/pager-arch-stack.dot index b28b561..d6e960c 100644 --- a/docs/diagrams/pager-arch-stack.dot +++ b/docs/diagrams/pager-arch-stack.dot @@ -1,4 +1,5 @@ // Compact 3-layer architecture for the 2-pager appendix. +// External-facing labels — no schema-column or table-name vocabulary. // Render: dot -Tsvg pager-arch-stack.dot -o pager-arch-stack.svg digraph arch_stack { @@ -9,11 +10,11 @@ digraph arch_stack { edge [fontname="Helvetica", fontsize=9] bgcolor="white" - surface [label="SURFACE — ingested documents\nWikipedia · HTML · TeX · git · claim-packs\nchunked → Merkle-rooted → FTS5-indexed", fillcolor="#e9f3ff"] - core [label="CORE — distilled documents\nMerkle-bound to surfaces by per-chunk\ninclusion proofs (derivations.proof_blob)", fillcolor="#dff5e7"] - cache [label="PROVIDENCE CACHE — verified Q&A\n8-dim cache_key · audit_mode label ∈\n{STRICT, HYBRID, UNGROUNDED}", fillcolor="#fff3d6"] + surface [label="Ingested documents\n(Wikipedia · HTML · TeX · git · claim-packs)\nchunked → hashed → full-text-indexed", fillcolor="#e9f3ff"] + core [label="Derived layers\n(summaries, indices, distillations)\nhash-bound back to source chunks", fillcolor="#dff5e7"] + cache [label="Answer cache\nverified Q&A · grounded / partly / not grounded\ncryptographic chain back to source bytes", fillcolor="#fff3d6"] - audit [label="AUDIT CHAIN — every state change\nappends sha256(prev || canonical(body))\nto audit_events (single SQLite file)", fillcolor="#fde7e9", shape=note] + audit [label="Audit chain\nevery write appends a hash-linked row;\na single tampered record breaks the chain", fillcolor="#fde7e9", shape=note] surface -> core [label="distill", fontsize=9] core -> cache [label="verify → cache", fontsize=9] diff --git a/docs/diagrams/pager-arch-stack.png b/docs/diagrams/pager-arch-stack.png index 1281414..c778679 100644 Binary files a/docs/diagrams/pager-arch-stack.png and b/docs/diagrams/pager-arch-stack.png differ diff --git a/docs/diagrams/pager-arch-stack.svg b/docs/diagrams/pager-arch-stack.svg index 4b39742..627ba76 100644 --- a/docs/diagrams/pager-arch-stack.svg +++ b/docs/diagrams/pager-arch-stack.svg @@ -4,77 +4,77 @@ - + arch_stack - + surface - -SURFACE — ingested documents -Wikipedia · HTML · TeX · git · claim-packs -chunked → Merkle-rooted → FTS5-indexed + +Ingested documents +(Wikipedia · HTML · TeX · git · claim-packs) +chunked → hashed → full-text-indexed core - -CORE — distilled documents -Merkle-bound to surfaces by per-chunk -inclusion proofs (derivations.proof_blob) + +Derived layers +(summaries, indices, distillations) +hash-bound back to source chunks surface->core - - -distill + + +distill audit - - - -AUDIT CHAIN — every state change -appends sha256(prev || canonical(body)) -to audit_events (single SQLite file) + + + +Audit chain +every write appends a hash-linked row; +a single tampered record breaks the chain surface->audit - - + + cache - -PROVIDENCE CACHE — verified Q&A -8-dim cache_key · audit_mode label ∈ -{STRICT, HYBRID, UNGROUNDED} + +Answer cache +verified Q&A · grounded / partly / not grounded +cryptographic chain back to source bytes core->cache - - -verify → cache + + +verify → cache core->audit - - + + cache->audit - - -every write + + +every write diff --git a/docs/diagrams/pager-verifier-flow.dot b/docs/diagrams/pager-verifier-flow.dot index 63d8699..e751b97 100644 --- a/docs/diagrams/pager-verifier-flow.dot +++ b/docs/diagrams/pager-verifier-flow.dot @@ -1,5 +1,5 @@ // Compact verifier flow for the 2-pager appendix. -// (Question, source-chunks) → model → (text, pointers) → verifier → label. +// External-facing labels — no internal vocabulary. // Render: dot -Tsvg pager-verifier-flow.dot -o pager-verifier-flow.svg digraph verifier_flow { @@ -10,27 +10,27 @@ digraph verifier_flow { edge [fontname="Helvetica", fontsize=11] bgcolor="white" - question [label="QUESTION", fillcolor="#fff7e6", shape=note] - retrieval [label="retrieval\n(FTS5 + phrase route\n+ rivalry exclusion)", fillcolor="#e9f3ff"] - pointers [label="POINTER MAP\nE1 → span\nE2 → span\n…", fillcolor="#dff5e7", shape=note] - model [label="LLM\nsees: question +\npointer-tagged chunks\nemits: prose with [E1] [E2] tags\n(never types the quote string)", fillcolor="#f0e6ff"] - verifier [label="verifier\n(quote · span · entity\n· paraphrase — binary,\nno LLM call)", fillcolor="#fff3d6"] + question [label="Question", fillcolor="#fff7e6", shape=note] + retrieval [label="Retrieve\ncandidate source chunks", fillcolor="#e9f3ff"] + pointers [label="Labelled chunks\nE1 → text\nE2 → text\n…", fillcolor="#dff5e7", shape=note] + model [label="Language model\nsees: question + labelled chunks\nemits: prose tagged [E1] [E2]\n(never types the quote text)", fillcolor="#f0e6ff"] + verifier [label="Verifier\nfour mechanical strategies,\nno model in the loop", fillcolor="#fff3d6"] - label_strict [label="STRICT\nevery claim verifies", fillcolor="#28a745", fontcolor="white"] - label_hybrid [label="HYBRID\nsome verify, some don't", fillcolor="#ffc107"] - label_ungrounded [label="UNGROUNDED\nnone verify", fillcolor="#dc3545", fontcolor="white"] + label_grounded [label="grounded\nevery claim verifies", fillcolor="#28a745", fontcolor="white"] + label_partial [label="partly grounded\nsome verify, some don't", fillcolor="#ffc107"] + label_ungrounded [label="not grounded\nnone verify", fillcolor="#dc3545", fontcolor="white"] - audit [label="audit chain\n+ providence cache\n(8-dim key, governance-aware)", fillcolor="#fde7e9", shape=note] + audit [label="Audit chain\n+ answer cache\n(reproducible, replayable)", fillcolor="#fde7e9", shape=note] question -> retrieval -> pointers pointers -> model - model -> verifier [label="answer\n+ pointer IDs"] - verifier -> label_strict - verifier -> label_hybrid + model -> verifier [label="answer with [E1] [E2] tags"] + verifier -> label_grounded + verifier -> label_partial verifier -> label_ungrounded - label_strict -> audit [style=dashed, color="#888888"] - label_hybrid -> audit [style=dashed, color="#888888"] + label_grounded -> audit [style=dashed, color="#888888"] + label_partial -> audit [style=dashed, color="#888888"] label_ungrounded -> audit [style=dashed, color="#888888"] - {rank=same; label_strict; label_hybrid; label_ungrounded} + {rank=same; label_grounded; label_partial; label_ungrounded} } diff --git a/docs/diagrams/pager-verifier-flow.png b/docs/diagrams/pager-verifier-flow.png index c3f435d..b0f3893 100644 Binary files a/docs/diagrams/pager-verifier-flow.png and b/docs/diagrams/pager-verifier-flow.png differ diff --git a/docs/diagrams/pager-verifier-flow.svg b/docs/diagrams/pager-verifier-flow.svg index e5fa750..138e050 100644 --- a/docs/diagrams/pager-verifier-flow.svg +++ b/docs/diagrams/pager-verifier-flow.svg @@ -4,149 +4,145 @@ - + verifier_flow - + question - - - -QUESTION + + + +Question retrieval - -retrieval -(FTS5 + phrase route -+ rivalry exclusion) + +Retrieve +candidate source chunks question->retrieval - - + + pointers - - - -POINTER MAP -E1 → span -E2 → span -… + + + +Labelled chunks +E1 → text +E2 → text +… retrieval->pointers - - + + model - -LLM -sees: question + -pointer-tagged chunks -emits: prose with [E1] [E2] tags -(never types the quote string) + +Language model +sees: question + labelled chunks +emits: prose tagged [E1] [E2] +(never types the quote text) pointers->model - - + + verifier - -verifier -(quote · span · entity -· paraphrase — binary, -no LLM call) + +Verifier +four mechanical strategies, +no model in the loop model->verifier - - -answer -+ pointer IDs + + +answer with [E1] [E2] tags - + -label_strict - -STRICT -every claim verifies +label_grounded + +grounded +every claim verifies - + -verifier->label_strict - - +verifier->label_grounded + + - + -label_hybrid - -HYBRID -some verify, some don't +label_partial + +partly grounded +some verify, some don't - + -verifier->label_hybrid - - +verifier->label_partial + + label_ungrounded - -UNGROUNDED -none verify + +not grounded +none verify verifier->label_ungrounded - - + + audit - - - -audit chain -+ providence cache -(8-dim key, governance-aware) + + + +Audit chain ++ answer cache +(reproducible, replayable) - + -label_strict->audit - - +label_grounded->audit + + - + -label_hybrid->audit - - +label_partial->audit + + label_ungrounded->audit - - + +