Each query/ask call now emits a 7-stage Merkle-DAG fingerprint stored
alongside the providence record. F from the toy-Hermes design pass.
Stages, in order:
question hash of question_hash (the 8-dim cache_key dim)
retrieval hash of sources summary (document_roots + roles +
scores + chunk_idx) — captures which docs ranked
context context_root (the source-Merkle for the assembly)
prompt conversation_hash
answer sha256(answer_text)
verify hash of verdict (audit_mode, verifier_method,
n_quotes, n_verified, claim_statuses)
final_label hash of (audit_mode, verifier_method, lookup_path)
run_dag_root = MerkleTree over those stage hashes (aborist conventions:
non-commutative HashCombine 0x03, leaf prefix 0x00, self-dup odd rule).
run_dag_blob = canonical JSON of {root, nodes} so an auditor can
recompute & verify (`verify_run_dag(blob)` returns True/False).
The DAG is NOT in cache_key. cache_key inputs determine the answer; the
answer determines the DAG — folding it back would create a cycle.
Instead it rides alongside as a per-record computation fingerprint.
Distinct from the linear `audit_events` chain (which tracks DB-wide
state changes); this is per-run computation provenance.
Schema: ALTER TABLE providence_cache ADD COLUMN run_dag_root TEXT;
ADD COLUMN run_dag_blob TEXT;
Idempotent migration in `_migrate_audit_mode`. Both rebuild templates
(VISUAL→UNGROUNDED dance, paraphrase verifier_method dance) updated to
include the new columns. Legacy records pre-2026-04-30 carry NULL.
Result dict gains `run_dag_root` so callers can verify without a DB
round-trip.
Tests:
- test_dag.py (9 tests): determinism, reactivity to each stage's input,
fixed stage order, round-trip verify, tamper-detection, JSON-string
acceptance.
- test_query.py: persistence on record + result, verify_run_dag round-
trip on the persisted blob.
473 tests pass (DAG +9, query +1, integration unchanged).
Two enhancements from the toy-Hermes design pass (2026-04-30):
A. Synthetic-elision-inside-quote diagnosis (sidecar only).
Distinct from interior_elision (model dropped a `(...)` aside source
carries) — synthetic_elision is the model writing literal `[...]`
between fragments of a `"..."` span, signaling self-elision while
claiming verbatim citation. The verifier still rejects (binary
discipline holds), but `aborist inspect` now reports
`diagnosis: synthetic_elision_inside_quote` with prefix/suffix
presence flags so an operator can judge whether the elided middle
was benign. Probe runs first in the classify-span chain (more
specific than trailing_artifact / interior_elision / paraphrase).
Catches the Brachiosaurus case: `"The film centers on the fictional
Isla Nublar [...] Universal Studios..."` — both halves are in source,
but the literal `[...]` isn't, so substring match correctly fails &
the sidecar tells the operator why.
C. Source-role classification + role-weighted context budget.
`_classify_source_role(title, qtokens_stem)` tags each top-K hit:
primary_answer_source 2.0× cap strong title-stem overlap
secondary_context_source 1.0× cap "list of", "characters",
"franchise", "history of"
noisy_background_source 0.5× cap "score", "music",
"video game", "merchandise"
sequel_background_source 0.5× cap "lost world", roman numerals
background_source 1.0× cap default
Order matters: noisy/sequel/secondary markers fire before the
primary check so peripheral pages with strong title overlap (e.g.
`Jurassic Park (film score)` shares 3 stems with the JP-film query)
don't claim a primary slot.
Cap loop now applies role weight on top of the baseline
`max_context_chars / top_k`. Total context still bounded by the
running `char_budget` — weights just shift how the budget gets
divided so primary pages get more text & noisy pages less, fixing
the case where a `(film score)` page consumed a primary slot.
`source_role` is persisted on `_Hit` and surfaces on
`merkle_proof.sources[*].source_role` in the providence record so
inspect & audits can see which slot each source occupied.
Tests:
- inspect: synthetic_elision_caught (Brachiosaurus regression),
synthetic_elision_does_not_fire_when_source_has_brackets (false-
positive guard).
- query: role classifier matrix (primary / secondary / noisy / sequel /
background) on JP-film-style titles, role persistence in sources list.
455 tests pass (sidecar +2, query +2).
Per fox: while iterating on retrieval/verifier knobs, a flag on the
query itself is more useful than a separate verb. One step reset:
change a knob, re-query, see fresh result.
Surface:
aborist query --burn ...
make query Q="..." BURN=1
make query-dry Q="..." BURN=1 (works on dry-run too)
Behavior in `aborist.qa.query.query()`:
- New `burn_existing: bool = False` parameter on query().
- After computing primary cache_key (per the active dedup mode) but
BEFORE the cache lookup, if burn_existing: DELETE the live row that
matches the primary cache_key & write one providence_burn audit
event with reason "query --burn (test-ergonomic mid-query bust)".
- The lookup then misses → fresh inference runs. Result reports
`burned_existing: 0|1` so the caller sees whether anything got
busted.
- The equivalence-class fallback cache_key is deliberately NOT
touched: prior alt-mode records stay as historic witnesses.
Tests (3): cache populates → cache hits → BURN=1 forces fresh
inference; --burn writes a providence_burn audit event; first-time
query with --burn is a clean no-op (burned_existing=0). 356 passed,
1 skipped.
Complementary to `aborist burn-kindergarten` (mass reset) — this is
the surgical version. Both are local-only by design; for cross-peer
invalidation use `make falsify` and let mesh sync broadcast.
Fox 2026-04-29: 'who is supermans girlfriend?' returned 7-of-8 unrelated
`Girlfriends`-titled articles (TV show, movies, songs); only 1 actual
Superman-related doc made it. Two coupled defects:
1. Title-overlap accept fired on ANY single-token match. A 2-token
query was admitting docs that shared only ONE qtoken. `Girlfriends`
passed because its title matched "girlfriend" even though no
"superman" anywhere in the doc.
2. `_body_density_passes` required at least HALF the qtokens — too
lenient for the 2-token case (1 of 2 = 50% = pass).
3. Possessive plural mismatch: question_hash strips apostrophes so
"superman's" → "supermans", but the corpus has bare "Superman".
`body.count("supermans")` returns 0 even on the canonical doc.
Three coordinated fixes in `aborist/qa/query.py`:
- New `_stem_token_for_match(t)` helper: strips a trailing `s` for
tokens > 4 chars (so `supermans` → `superman`, `girlfriends` →
`girlfriend`). Conservative: skips short tokens & double-s endings
to avoid `class` / `boss` / `pass` corruption.
- `_body_density_passes` now uses `_body_count_with_stem` (literal-
first, stem-fallback) AND a tightened breadth threshold:
≤ 2 tokens require ALL of them
3+ tokens require N - 1 (allow one weak signal token to miss)
- `_filter_by_title_relevance` mirrors the same breadth threshold for
title-overlap. Synonym fallback (any-match against synonym_expand)
is preserved ONLY for 1-token queries — otherwise a stray synonym
hit (e.g. AMD synonym matching an Intel-only title) over-recalls.
End-to-end on the actual corpus (3.4M articles, 8 shards):
before: 'supermans girlfriend' → 7 unrelated `Girlfriends` titles, 1 Superman doc
after: Lois_Lane in top-K (the canonical answer), other Superman-related
docs alongside, Girlfriends-only-titled docs filtered out
Tests:
- test_query_filter_requires_breadth_for_multi_token_queries — synthetic
3-doc corpus pinning the new behavior (Lois Lane keeps; girlfriends-tv
& superman-music both drop).
- test_query_filter_one_token_query_still_synonym_expands — pin the
1-token loose path is preserved (no over-tightening).
349 passed, 1 skipped.
Burned the two stale UNGROUNDED girlfriend-query cache records so a
fresh `make query` exercises the new filter end-to-end.
Different agents have different value functions on the same providence
records. Today aborist bakes one canonicalization policy into
governance_policy_hash and calls it universal. This adds two knobs that
let agents express their preference without breaking provenance.
- aborist/qa/keys.py: `canonical_question(q, mode=...)` and
`question_hash(q, mode=...)` accept "strict" (NFC + ws-collapse only;
every variant gets its own hash) or "equivalence_class" (default —
additionally lowercase + trailing-punct strip + article strip).
New constants: QUESTION_DEDUP_MODES, FIDELITY_MODES, defaults.
- aborist/qa/query.py + aborist/qa/runner.py: each call computes the
primary cache_key under policy["question_dedup"]. New `fidelity`
parameter:
"strict" only primary cache_key checked
"equivalence_class" primary first; if miss AND alternate mode
produces a different cache_key, try alternate
Cross-silo fallback works because _ckey_for_mode rewrites
policy["question_dedup"] to the alternate mode before computing
governance_policy_hash — so the fallback ckey matches what an agent
under that mode would have written. Lookups can find each other's
records when fidelity permits. Result dict gains `lookup_path` ∈
{"strict", "equivalence_class", "strict_fallback",
"equivalence_class_fallback", "miss"}.
- aborist/cli.py: `--question-dedup` and `--fidelity` flags on the
`query` subcommand. Human render annotates cache_hit lines that
came from a fallback ckey ("cached via equivalence_class_fallback").
- tests/test_query.py: three new regressions
* strict-policy + strict-fidelity: variants get distinct cache_keys
* cross-silo fallback: strict-policy reads eq-class-policy records
via fidelity=equivalence_class
* strict-fidelity refuses fallback (audit-grade): cache miss even
when the alternate silo has a hit
- CLAUDE.md: dedup-mode and fidelity convention bullets updated.
This is V1 + V2 of the substrate move sketched in conversation: write
policy determines which silo a record lives in; read fidelity determines
how loosely an agent walks across silos. Records stay exactly-keyed
(provenance hard); routing is per-agent (preference soft). Same
verifier_no_diagnostics discipline holds — no soft signal enters the
hard chain.
442 tests pass.
Fox caught on 2026-04-29: 'who is batman?' and 'who is the batman?' produced
different cache_keys despite question_hash collapsing both variants. The 8-dim
cache_key has TWO seams that touch question text — question_hash AND
conversation_hash (which hashes the messages list with the literal question
in the user turn). Only the former canonicalized; the latter took bytes as
written, so each variant got its own chash and missed cache.
Fix: decouple the two forms.
- aborist/qa/keys.py: extract canonical_question(text) → str; question_hash
composes _sha256(canonical_question(t)) so the equivalence class is
defined in one place.
- aborist/qa/query.py + aborist/qa/runner.py: build TWO message lists per
call. `messages` carries the verbatim question (Hermes sees the user's
natural phrasing — no grammar drift). `canonical_messages` substitutes
the canonical form, and conversation_hash hashes that. The LLM still
gets 'Who Is THE Batman?'; the cache_key collapses to 'who is batman'.
- tests/test_query.py: regression — three variants (with/without `?`,
with/without `the`) hit the same cache_key. First call populates,
later variants cache_hit. Captures stub messages to confirm the LLM
saw the verbatim question, not the canonical form.
437 tests pass.
Fox 2026-04-29: querying "who is batman" returned only ONE source
(List_of_Batman_comics — an 80 KB+ bibliography) despite top_k=8 &
the actual bio article being in the corpus. Greedy fill: hit #1
consumed the entire 60 KB budget, every subsequent doc dropped with
char_budget <= 0.
Fix in aborist/qa/query.py:
per_source_cap = max(1, max_context_chars // max(1, top_k))
for h in hits[:top_k]:
text = _load_doc_text(...)
if len(text) > per_source_cap:
text = text[:per_source_cap] # NEW: per-source cap first
if len(text) > char_budget:
text = text[:char_budget]
...
Each top_k hit gets at most max_context_chars/top_k chars (default
60K/8 = 7.5K each — plenty for a chunk or two of prose). Total
context ≤ max_context_chars by construction. top_k=1 preserves the
legacy behavior (single source can use the full budget).
End-to-end effect on Batman: the bio (Wikipedia/Batman article) lands
in context alongside List_of_Batman_comics; the model can paraphrase-
verify against the actual character introduction text instead of
fabricating from training.
Tests: 2 regressions in tests/test_query.py — multi-source delivery
when hit #1 is huge, and top_k=1 single-source still allowed full
budget. 337 passed, 1 skipped.
Burned the two stale Batman cache records (chain extended) so a
fresh `make query Q="who is batman?"` exercises the new path.
Adds aborist/qa/verify.py — three-strategy verifier (explicit quotes,
bullet/sentence spans, multi-word proper nouns) that lexically checks
every claim against retrieved context under norm-v1 + lowercase. The
strategy that fires is recorded as verifier_method for diagnostics.
Entity strategy gates classification via an entity_policy
(strict/hybrid/proximity/drop) so a single proper-noun match no longer
overclaims STRICT — proximity (default) requires a cluster of 3+
verified entities within 300 chars.
Wires into ask() and query(). System prompts now require verbatim
quoted spans for every factual claim, restated via a user-turn
grounding_reminder one message before sources arrive (recent
user-turn instructions outweigh decayed system-turn rules under long
context in Hermes).
providence_cache gains 5 columns (audit_mode, n_quotes, n_verified,
unverified_quotes, verifier_method) with CHECK constraints. A
connect-time _migrate_audit_mode() ALTERs legacy DBs idempotently.
Cache hits return the persisted audit_mode rather than asserting
STRICT unconditionally.
CLI:
emergent list VISUAL/HYBRID records; --aggregate ranks
unverified quotes by frequency (corpus-growth signal).
reclassify re-run the verifier against live providence records;
cold-source records skipped; --dry-run reports
transitions without writing; each change writes one
'providence_reclassify' audit event.
aborist/qa/query.py exposes query() — the user-facing RAG flow:
1. FTS5 search across all shards (chunks_fts can't be UNION'd as a
view, so each shard's index is queried independently and merged
by score).
2. Top-K distinct documents are selected within a max-context-chars
budget (default 60 KB so a 768-token response fits Hermes-3's
82 K context window comfortably).
3. context_root = Merkle root over the sorted source document_roots.
That's the v9.8 'source' dimension for multi-source answers —
a verifier can recompute it from the listed source roots.
4. 8-dim cache_key over (context_root, question_hash, model_profile,
conversation, governance_policy, schema, canonicalization,
chunking). Hit returns STRICT immediately; miss calls Hermes and
persists.
CLI: aborist [--shards-dir DIR] query "<question>"
Default qa_db is <shards-dir>/qa.db (or ~/.aborist/qa.db). Uses the
same OpenAICompatibleClient/StubClient as `ask`. --dry-run skips the
LLM and returns context-only.
Search escape fix: the prior FTS5 escape ANDed every token including
stopwords + punctuation, so "What is anarcho-capitalism?" required
the doc to literally contain "what" + "is" + "anarcho-capitalism?" —
zero hits. New tokenizer drops stopwords + punctuation and ORs the
remaining content tokens; BM25 ranks the multi-token matches highest.
Live demo against the 122k-doc 4-shard cluster:
Q "What is anarcho-capitalism?" 6.1 s wall miss / 0.45 s cache hit
Q "Who was George Washington?" 10.3 s wall miss
Both answers cite the source URIs Hermes was given.
57 tests passing (4 new query tests covering search → context →
cache → audit chain).
