russell@unturf.com
326badf6d8
Three things in one commit because they're tightly coupled (README
points at the diagrams; diagrams index in modules/index.md points
back at README; module pages embed the diagrams).
(1) README — label refresh:
- Quickstart label changed from STRICT/HYBRID/UNGROUNDED to the
four-rung ladder POINTER-LINKED → ANCHOR-WARRANTED →
EVIDENCE-WARRANTED → UNGROUNDED with -PARTIAL suffix on HYBRID.
- Verifier section spells out both layers (schema trichotomy +
display ladder), the seven hard checks of claim_lattice, and
the five anchor classes of warrant.
- Architecture tree updated: concepts/ package added, qa/
sub-modules expanded (warrant, evidence, parse_claims, dag),
verify.py described as quote/span/entity/paraphrase + claim_lattice.
- Concept overlay description updated for corpus-derived layer
(concept_relations table, link_reciprocity extractor, 1.6%
tax cite).
- Test count: 326+ → 641+.
(2) docs/diagrams/ — Graphviz dot sources:
- aborist-modules.dot — top-level package graph (substrate /
storage / sources / retrieval / qa / mesh / cli)
- query-pipeline.dot — question → cache → retrieval → LLM →
verify → render → cache write, with phase budgets
- ingest-pipeline.dot — source doc → canonicalize → chunk →
Merkle → upsert (+ optional distill)
- verifier-ladder.dot — (audit_mode, violations) → display rung
decision tree
Existing mesh-*.dot kept as-is. Makefile `make docs` target
extended to also emit .svg alongside the existing .png so the
diagrams render in markdown viewers.
(3) docs/modules/ — per-module reference pages:
- index.md (links to every diagram + every module page)
- merkle.md, document.md, store.md, ingest.md, evict.md,
sources.md, search.md, concepts.md, qa.md, distill.md,
wikitext.md
Each page is a one-screenful concise reference: what the
module is for, public API, key invariants, embedded diagrams
where useful, link to source. Mesh stays at the existing
docs/mesh.md + docs/mesh-deploy.md (already comprehensive).
Tests: 641 passed (no code change).
185 lines
6.9 KiB
Markdown
185 lines
6.9 KiB
Markdown
# `aborist.qa`
|
||
|
||
The Q&A pipeline. Question → cache → retrieval → LLM → verify →
|
||
render → cache write. Lives in 9 sub-modules; this page is the
|
||
map.
|
||
|
||
|
||
|
||
## Sub-modules
|
||
|
||
### `qa.client` — LLM transport
|
||
|
||
`ChatClient` ABC with three concrete implementations:
|
||
|
||
- `StubClient` — deterministic test fixture; returns canned
|
||
responses keyed on the input. Used in unit tests to avoid network.
|
||
- `OpenAICompatibleClient` — talks to any OpenAI-shape `/v1/chat/completions`
|
||
endpoint (Hermes-3 on vLLM by default). Includes HTTP retry layer
|
||
(3× exponential backoff on 5xx).
|
||
- (Future) `AnthropicClient` — Claude API direct.
|
||
|
||
[`aborist/qa/client.py`](../../aborist/qa/client.py)
|
||
|
||
### `qa.keys` — the 8-dim cache_key
|
||
|
||
```
|
||
cache_key = sha256(
|
||
source_root | question_hash | model_profile_hash |
|
||
conversation_hash | governance_policy_hash |
|
||
schema_version | canonicalization_version | chunking_version
|
||
)
|
||
```
|
||
|
||
Two question-hash modes (`strict` vs `equivalence_class`) live here.
|
||
Bumping any of these eight dimensions invalidates prior records on
|
||
lookup. The `verifier_policy_hash` (v9.9 9th dim) is also
|
||
implemented here.
|
||
|
||
[`aborist/qa/keys.py`](../../aborist/qa/keys.py)
|
||
|
||
### `qa.runner` — `ask()` for single-doc Q&A
|
||
|
||
The simplest entry point. Take one document, ask one question, get
|
||
back an answer + audit_mode + cache record. Used by the CLI for
|
||
focused queries against one URI.
|
||
|
||
[`aborist/qa/runner.py`](../../aborist/qa/runner.py)
|
||
|
||
### `qa.query` — `query()` for multi-source RAG
|
||
|
||
The main retrieval entry point. Walks shards, runs FTS5 BM25 with
|
||
AND→OR fallback (with synonym-pool injection in OR mode), filters
|
||
by title relevance with 4 accept paths, reranks by body coverage +
|
||
title boost + source role + title purity, assembles a 60 KB context
|
||
budget, calls the LLM, runs the verifier, persists to
|
||
`providence_cache`.
|
||
|
||
[`aborist/qa/query.py`](../../aborist/qa/query.py)
|
||
|
||
### `qa.verify` — the layered verifier
|
||
|
||
Five strategies run in sequence; first to find evidence classifies:
|
||
1. `quote` — `"..."`-wrapped claims tested verbatim
|
||
2. `span` — bullet/sentence units substring-tested
|
||
3. `entity` — multi-word proper nouns with proximity gating
|
||
4. `paraphrase` — token coverage on prose-shaped spans (≥85%)
|
||
5. `claim_lattice` — pointer-line `[E1,E2]` or JSON; runs **seven
|
||
deterministic hard checks**:
|
||
1. parser succeeded
|
||
2. evidence_id resolves
|
||
3. source_role allowed
|
||
4. claim text non-empty
|
||
5. citation coverage threshold
|
||
6. pointer count cap (trim-and-verify)
|
||
7. anchor-class warrant (see `qa.warrant`)
|
||
|
||
The classifier output rolls up into the v9.8 trichotomy
|
||
`audit_mode ∈ {STRICT, HYBRID, UNGROUNDED}`. Display layer (in
|
||
`cli.py`) maps `(audit_mode, violations) → four-rung ladder`.
|
||
|
||
|
||
|
||
[`aborist/qa/verify.py`](../../aborist/qa/verify.py)
|
||
|
||
### `qa.warrant` — anchor-class warrant
|
||
|
||
Five lexical anchor classes the verifier composes:
|
||
|
||
- **Proper-noun** — relation-question shape; at least one
|
||
Title-Case anchor must appear in some cited span
|
||
- **Date** — claim has a 4-digit year + month name; ALL components
|
||
required in some cited span
|
||
- **Entity-list** — entity-list-shape question; ≥1 named entity
|
||
must anchor (demote-don't-reject)
|
||
- **Count** — count-shape question; count token must appear in
|
||
word OR digit form (digit↔word equivalence)
|
||
- **Cause** — why-shape question; ≥1 cause anchor (proper noun OR
|
||
≥5-char common noun outside stopword pool)
|
||
|
||
The warrant layer earns proof-path entry by staying **lexical** —
|
||
no NLI, no embeddings. Substring tests over already-canonicalized
|
||
spans. See `docs/concept-relations-design.md` (sibling section)
|
||
for the relationship to retrieval-time synonym expansion.
|
||
|
||
[`aborist/qa/warrant.py`](../../aborist/qa/warrant.py)
|
||
|
||
### `qa.evidence` — EvidenceObject + spotlight
|
||
|
||
Builds the runtime evidence map for claim-lattice modes. Each
|
||
chunk becomes one `EvidenceObject` carrying TWO ids:
|
||
|
||
- `pointer_id` — short prompt-facing tag (`E1`, `E2`, …)
|
||
- `evidence_id` — content-addressed `E########` (sha256-derived)
|
||
|
||
The model sees only `pointer_id`s in the prompt; the runtime maps
|
||
to `evidence_id` for the cache & run-DAG (run-stable identity).
|
||
|
||
The spotlight excerpt picks the load-bearing slice via **density
|
||
rank** — find ALL match positions for ALL claim content tokens,
|
||
pick the position with maximum distinct-token cluster within
|
||
±half-window. Replaces the older first-match-of-longest-token
|
||
approach which lost the load-bearing slice on noisy chunks.
|
||
|
||
[`aborist/qa/evidence.py`](../../aborist/qa/evidence.py)
|
||
|
||
### `qa.parse_claims` — pointer-line parser
|
||
|
||
Walks lines of the model output, pulls every `[E\d+]` and
|
||
`[E\d+,E\d+,…]` bracket payload, returns
|
||
`(claim_text, pointer_ids[])` per line. Lines without a tag get
|
||
`parse_status='NO_EVIDENCE_POINTER'` & count toward the denominator
|
||
so unsourced prose can't smuggle past the verifier.
|
||
|
||
[`aborist/qa/parse_claims.py`](../../aborist/qa/parse_claims.py)
|
||
|
||
### `qa.dag` — per-run Merkle DAG
|
||
|
||
Commits each provenance step independently as a stage hash. Two
|
||
shapes:
|
||
|
||
- **7-stage (quote mode):** question / retrieval / context /
|
||
prompt / answer / verify / final_label
|
||
- **9-stage (claim-lattice / CTI):** question / retrieval /
|
||
evidence_map / prompt / raw_answer / parsed_claim_lattice /
|
||
verify / render / final_label
|
||
|
||
The `run_dag_root` is persisted alongside every providence record;
|
||
`run_dag_blob` carries the full `{root, nodes}` JSON so an auditor
|
||
can recompute & verify any step.
|
||
|
||
[`aborist/qa/dag.py`](../../aborist/qa/dag.py)
|
||
|
||
### `qa.inspect` — read-only sidecar
|
||
|
||
Pulls source chunks for a given cache_key & classifies each
|
||
unverified span: `verbatim_in_base` / `verbatim_in_raw_only` /
|
||
`trailing_artifact` / `paraphrase` / `partial_paraphrase` /
|
||
`no_overlap`. Also includes the deflection-detection sidecar
|
||
(subject-anchor heuristic for adversarial-premise topic shift).
|
||
|
||
**Sidecars never write to `providence_cache` or `audit_events`** —
|
||
they're diagnostic only. That invariant is what keeps `audit_mode`
|
||
a binary classification rather than a soft score.
|
||
|
||
[`aborist/qa/inspect.py`](../../aborist/qa/inspect.py)
|
||
|
||
### `qa.concepts` — backwards-compat shim
|
||
|
||
Delegates to `aborist.concepts` (the corpus-derived synonym/rivalry
|
||
layer). Pre-2026-05-01 the data lived as hand-curated frozensets in
|
||
this file; now it's a per-shard SQLite table. The shim preserves
|
||
the legacy public API (`synonym_expand`, `rivalry_excluded`,
|
||
`has_compare_phrasing`) so call sites in `qa/query.py` didn't have
|
||
to change.
|
||
|
||
[`aborist/qa/concepts.py`](../../aborist/qa/concepts.py) →
|
||
[`aborist/concepts/`](../../aborist/concepts/)
|
||
|
||
## Source papers
|
||
|
||
- Whitepaper §13.8 covers the layered verifier in depth
|
||
- Whitepaper §13.9 covers claim-lattice / CTI mode
|
||
- `docs/cti-architecture.md` is the architecture reference
|
||
- `docs/seven-point-program.md` enumerates the seven hard checks
|
||
- `docs/concept-relations-design.md` covers the synonym layer
|