JSON mode previously presented content-addressed evidence_ids
(``Eed1b6e396``-style) in the EVIDENCE block headers and expected
the same in the model's claim output. Hermes-3-8B was fabricating
plausible near-miss IDs (e.g. ``E1b6e396`` when the runtime had
``Eed1b6e396``) on cross-document relationship questions →
UNKNOWN_EVIDENCE_ID → UNGROUNDED, even when the answer text was
factually correct (e.g. "Homer Simpson's boss is Mr. Burns.").
Switching the prompt-facing surface to short pointer IDs (``E1``,
``E2``, …) — same as claim_lattice_pointer mode — closes the
hallucination loop:
- Pointer IDs are short, enumerable, and fabrication-obvious.
The model can't invent ``E27`` if only ``E1``-``E10`` were shown;
out-of-range IDs read as schema violations at first glance.
- The runtime still resolves each pointer_id to its content-
addressed evidence_id internally and stores THAT in
``evidence_id_pairs`` for the cache & run-DAG. Cache_keys stay
run-stable; only the prompt-facing string changes.
- JSON schema unchanged (``evidence_ids: [str, ...]``), so vLLM
guided_json continues to constrain output shape.
Live-test verification: ``who is homer simpson's boss?`` went from
JSON-mode UNGROUNDED 0/1 (hallucinated ``E1b6e396``) to STRICT 1/1
(model emits ``E1``, resolves cleanly). Homer fixture pin removed —
runs against the JSON default now.
Files touched:
- aborist/qa/evidence.py: render_evidence_block_for_json uses
e.pointer_id instead of e.evidence_id.
- aborist/qa/verify.py: verify_claim_lattice_json switched from
evidence_map_by_evidence_id to evidence_map_by_pointer_id;
captures both pointer_ids (model-emitted) and evidence_ids
(run-stable) in claim_statuses + evidence_id_pairs.
- aborist/qa/{runner,query}.py: claim_lattice_json_system_prompt
+ grounding_reminder describe pointer IDs; example shifts from
``E........`` placeholder to ``E1``.
- tests/test_verify_json.py: stub _ev() takes pointer_id; four
fixtures updated to set it.
- tests/test_qa_quality_live.py: Homer fixture unpinned (now
runs default JSON mode and grounds). Red-fish-blue-fish
fixture pinned to pointer mode — JSON mode hits a separate
token-budget runaway on "plot of X" prose-summary shapes
(~2/3 of samples blow max_tokens with whitespace spam after
the closing brace). Different failure mode, addressed in a
later commit.
460 unit tests + 11 live fixtures pass.
Post-retry / post-trim-and-verify bench showed the picture flipped:
JSON mode now leads on strict-rate (50%) and ties grounded count
(54) with zero errors, all at parity-or-better latency. Switching
the `make query` ANSWER_MODE default from `claim_lattice_pointer`
to `claim_lattice` so the human-facing CLI uses the strongest mode.
Library-level DEFAULT_ANSWER_MODE stays "quote" so unit tests using
StubClient aren't disrupted.
Doc updated with post-retry bench table + revised conclusion.
Live test harness:
- new test_homer_simpson_boss_is_mr_burns fixture (pinned to
pointer mode — JSON mode hallucinates evidence_ids on
cross-document relationships, pointer mode's short numeric
tags can't be fabricated). Documents the mode trade-off.
- test_laura_croft marker net broadened to absorb Hermes
single-sample variance (added "magazine", "video game",
"character", "fictional", "british" markers) plus a hard
"croft" anchor check. Both real entities still gate cleanly.
11/11 live fixtures pass at the new JSON default + pointer pin
on Homer. 460 unit tests + 11 live = 471 green; 10 skipped is
just the live tests in the default-skip path.
Adds test_red_fish_blue_fish_identifies_seuss_book under the new
"retrieval disambiguation" section. The query partially matches
many noise titles (Red Dwarf, Blue whale, Toronto Blue Jays, Red
Sea, Detroit Red Wings, marine aquarium fish list) so it stress-
tests the retrieval+rerank pipeline's ability to land on the right
article — `One Fish Two Fish Red Fish Blue Fish` by Dr. Seuss —
despite the surface noise. Asserts the answer is grounded and
contains a Seuss-specific marker (seuss, rhyming, children's book,
or creatures). 10 tests collected total.
New tests/test_qa_quality_live.py with 9 fixtures gated by
ABORIST_LIVE_TESTS=1 (the `make test-live` target sets it). Each test
runs ONE live query against Hermes + the configured shard set and
asserts:
* audit_mode is at least HYBRID (or UNGROUNDED for the honest-
refusal case)
* key entity tokens appear in the rendered answer
(e.g. "Torvalds" for the linux-kernel question, all four turtle
names + "Splinter" for the TMNT multi-part)
* for the Mars PM no-such-thing case, either UNGROUNDED OR a
refutation phrase ("no prime minister", "does not have", etc.)
— bare affirmative claims of a Mars PM fail the test as
hallucination
Why functional tests matter alongside unit + bench:
- unit tests (test_qa, test_query, test_claim_lattice) validate
plumbing with StubClient — can't tell you "did the model
actually answer correctly?"
- bench (`make bench-qa`) measures aggregate strict-rate /
grounded counts across many questions but doesn't assert
specific content
- these live fixtures sit between: each test is a named gate
around one known-good answer. When a future change improves
things, the bench number climbs AND every fixture passes (or
gets stricter assertions). When something regresses, the bench
number falls AND specific fixtures fail by name, telling you
where the regression landed. "Benchmax" rationale: the bench
is the scoreboard, the fixtures are the gates that translate
quality drift into named test failures.
Default `make test` is unaffected — 460 passed + 10 skipped (was 1).
Three layered fixes for the JP-dinosaurs lazy-anchor failure mode where
"what dinosaurs were in the first jurassic park film?" returned HYBRID
5/8 with citations to a video-game tie-in chunk (Operation Genesis):
1. Bare-name claim guard. New policy field
`claim_lattice_min_claim_content_tokens` (default 2). Claims with
fewer content tokens (>=4 chars, post-spotlight-stopword) classify
as SCHEMA_INVALID. "Triceratops. [E16]" had 1 content token; lexical
overlap on a single token is too thin to verify "X is in the FIRST
film" semantics. Forcing sentence shape raises the coverage bar so
off-topic chunks can no longer satisfy the citation. Threshold of 2
keeps narrow factoids viable (e.g. "Steve Jobs co-founded Apple"
tokenizes to 3 content tokens after stopword strip).
2. Smell → demote. The lazy-anchor smell sidecar was advisory only;
the verdict could be STRICT while every claim cited the same
magnet chunk. New policy fields
`claim_lattice_lazy_anchor_demote_threshold` (0.5) and
`claim_lattice_lazy_anchor_demote_min_pairs` (3) cap audit_mode at
HYBRID when ratio + pair-count both met. STRICT now requires
diverse anchoring across pointers; emits a LAZY_ANCHOR_DEMOTE
violation record so the auditor can see what fired.
3. Noisy title markers extended. `_NOISY_TITLE_MARKERS` now includes
"operation genesis", "the game", "video games" so spinoff/tie-in
articles (Jurassic Park: Operation Genesis, X (NES game), etc.)
classify as noisy_background_source. Default
claim_lattice_allowed_source_roles excludes that role, so noisy
chunks never enter the evidence map for pointer-mode queries.
Brittle (case-by-case markers) but addresses the JP failure
directly.
Live JP-dinosaurs after fix: UNGROUNDED 0/11 (was HYBRID 5/8 with
bogus citations). Honest refusal beats false confidence — model could
recover by writing sentence-shape claims; bare-name shortcut blocked.
Test fixtures using minimal placeholder claims ("Trex appears. [E1]")
opt out of the bare-name check via `min_claim_content_tokens=0` since
those tests exercise orthogonal behaviors. 460 tests pass.
JSON mode (answer_mode="claim_lattice") is now reachable end-to-end via
runner.ask: message build, optional vLLM guided_json extra_body, verifier
dispatch, and providence_cache persistence. Verifier returns
verifier_method="claim_lattice" (same as pointer variant) so both modes
share the existing CHECK constraint; answer_mode on the run-DAG and
json_fixups on the verdict disambiguate.
Pointer verifier now splits EVIDENCE_LINKED_PARTIAL into its own
partially_verified_quotes bucket so a claim never renders as both a
verified bullet and an unverified footer. CLI shows a "partially
grounded" section between the verified body and the unverified list.
Removes the strict no-double-quote check (`MANUAL_QUOTE_VIOLATION`) from
verify_claim_lattice. The rule was rejecting factually correct,
source-grounded claims for cosmetic punctuation: Hermes-3-8B paraphrases
prose but copies named-quoted phrases verbatim from source (e.g.
`"Constitution State"` lifted from a Connecticut chunk). Pre-fix,
`make query Q="tell me about connecticut"` reported `0/13 verified` on
a paragraph where every claim was correct and source-supported, just
because the model preserved source quote marks.
The two checks that remain handle the cases the manual_quote rule was
nominally meant to catch:
- claim_lattice_min_citation_coverage (Rule 5, default 0.30)
catches lazy-anchor citations whose only overlap with the cited
span is a single topical token
- claim_lattice_max_pointers_per_claim (Rule 6, default 2)
catches the encyclopedic-mega-claim where the model emits
`[E1,E2,...,E26]` after one sentence
Bench n=3 across 8 fixed questions vs pre-G0-hardening baseline (also
n=3): pointer-mode grounded count (STRICT+HYBRID) goes 15 → 21,
UNGROUNDED 9 → 3, broad-descriptive failures (connecticut, python)
fully cured. STRICT count goes 12 → 9 because previously-bogus STRICTs
(microsoft cited transit chunks that just shared one token, supermans
lazy-anchor magnet at 32/32) are now honestly reclassified to HYBRID.
Side changes:
- Ports max_pointers / min_citation_coverage policy fields and
verify_claim_lattice call into runner.py so single-document `ask`
matches multi-source `query` semantics (query.py already had them
via b39e79b).
- Worked Example 2 (broad-descriptive) added to pointer-mode prompt
so "tell me about X" has a template, plus Rules 7 (anti-echo) and
8 (max-2-pointers).
- Two manual_quote tests in tests/test_claim_lattice.py inverted to
document the new behavior (quotes-in-claim no longer block).
- `_has_manual_quote` retained — still used by verify_claim_lattice_json.
459 tests pass.
Brings the JSON variant back as a third ANSWER_MODE, distinct from the
pointer variant. The substrate exposes both: pointer for prose-
distribution / small-model paths (Hermes-3 8B), JSON for grammar-
constrained / large-reasoning-model paths (vLLM guided_json,
Claude/GPT-4 native JSON, Qwen 3.6 reasoner). Agents pick by setting
`policy["answer_mode"]`; both fold into governance_policy_hash so
records under different modes never alias.
Components:
- aborist/qa/verify.py:
* ANSWER_MODES = ("quote", "claim_lattice_pointer", "claim_lattice")
* `_lenient_json_parse(raw)` — defensive pre-parser. Strips markdown
fences, trims preamble/suffix to {/}, normalizes curly quotes,
fixes trailing commas. Returns (parsed, fixups[]) so the verifier
logs which drift had to be peeled. Lenient on syntax, strict on
semantics: parsed JSON still has to schema-check.
* CLAIM_LATTICE_JSON_SCHEMA — JSON Schema for the {"claims":[...]}
shape. Used by vLLM guided_json sampling-time constraint.
* `verify_claim_lattice_json(answer_json_text, evidence_map, ...)` —
runs the same hard checks as verify_claim_lattice (evidence_id
resolves, source_role allowed, no manual quotes, claim text non-
empty, claim textually overlaps evidence) but on content-
addressed evidence_ids directly. Returns the same verdict shape
plus a `json_fixups` list.
- aborist/qa/client.py: ChatClient Protocol & OpenAICompatibleClient
gain optional `extra_body` kwarg. Forwarded as additional fields in
the JSON request payload — opaque pass-through for vLLM-specific
knobs like `guided_json`. Endpoints that don't recognize a key
silently drop it. StubClient ignores; tests inspect via self.calls.
- aborist/qa/evidence.py: `render_evidence_block_for_json` and
`render_evidence_map_for_json` — JSON-mode prompts label blocks with
the content-addressed evidence_id (long hex) since that's what the
model cites in its JSON. Pointer mode keeps using the short
pointer_id.
- aborist/qa/query.py:
* Imports the JSON verifier + schema + JSON-mode evidence renderer.
* Message-build branch: `elif answer_mode == "claim_lattice"`
builds the same per-chunk evidence map as pointer mode, but
uses `claim_lattice_json_system_prompt` and labels blocks with
evidence_id.
* LLM call: when answer_mode=claim_lattice and policy
`claim_lattice_use_guided_json` is on, passes
`extra_body={"guided_json": SCHEMA}` so vLLM constrains output.
* Verifier dispatch: new `elif answer_mode == "claim_lattice"`
branch calls verify_claim_lattice_json.
* DAG persistence: lattice-mode raw_answer / parsed_lattice /
rendered_text threading now applies to both pointer and JSON.
* DEFAULT_QUERY_POLICY adds `claim_lattice_json_system_prompt`,
`claim_lattice_json_grounding_reminder`, and
`claim_lattice_use_guided_json` (default True).
- tests/test_verify_json.py: 14 tests covering the lenient parser
(strict pass-through, fence strip, preamble trim, curly-quote
normalize, trailing-comma fix, multi-fixup, hard-fail) and the
JSON verifier (STRICT on resolved claims, HYBRID on partial,
UNGROUNDED on schema invalid, fence recovery, manual-quote
violation, source-role block).
The whitepaper rewrite to 13.9.1 (substrate exposes both modalities,
both first-class) becomes accurate post-ship — JSON mode now exists
in code as it always existed in the architecture's intent.
554 tests pass.
Two layered fixes that close the lazy-anchor gap. Title-purity
moves the right SOURCES to the top of retrieval; the new hard
check makes the verifier ruthlessly honest about whether each
cited chunk actually mentions what its claim says.
1. _rerank_by_title_purity. New stage at the end of _rerank().
purity = |title_tokens ∩ query_tokens| / |title_tokens|; score
multiplied by (1 + 2*purity), so a title that IS the topic
("Jurassic Park (film)" against the dinosaur question) gets
3.0x and titles that drag in off-topic tokens like
"Jurassic Park: Operation Genesis" only get 2.0x. Composes
with the role-rank weights from the prior commit. Florida
query: list-pages dropped out of top-8 entirely. JP-dinosaurs
query: Operation Genesis sank from #4 to #8.
2. CITATION_MISMATCH hard check. Sixth deterministic check in
verify_claim_lattice — for each (claim, pointer) pair, at
least one content token from the claim text must textually
appear in the cited evidence span. Lexical only, no NER, no
embeddings; same _content_tokens extractor as the spotlight
so the proof-path stays on the soft/hard boundary. Catches
the magnet-chunk lazy-anchor where the model cites a chunk
whose text contains zero overlap with what its claim asserts.
New status CITATION_MISMATCH joins the per-claim taxonomy.
New violation kind CITATION_MISMATCH joins the run-DAG verify
payload. JP-dinosaurs verdict went from "13/14 verified
ratio 0.77 (cite E13 for everything)" pre-check to "1/14
verified" post-check — only Tyrannosaurus Rex's claim
actually overlapped its cited chunk. The architecture
stopped lying about evidence-linkage that the cited spans
don't textually support.
Pure-stopword claims pass the overlap check vacuously (no
content tokens means nothing to check against) — other hard
checks own that case.
Tests: +3 covering CITATION_MISMATCH (claim-token absent yields
fail, partial overlap yields HYBRID with EVIDENCE_LINKED_PARTIAL,
pure-stopword claim passes vacuously). Updated 7 prior tests
that used colloquial stub claims ("Trex") which don't textually
appear in the sample spans ("Tyrannosaurus rex"); they now use
claim text that overlaps the source content. 445 tests passing,
all 7 production shards report 0 chain breaks.
verify_claim_lattice now derives pointer_id_distribution and
lazy_anchor_ratio from the verified-or-partial claim_statuses and
returns them in the verdict. Surfaced through ask()/query() result
dicts; CLI human-render displays a one-liner when ratio >= 0.5
(threshold for "model is lazy-anchoring").
Sidecar discipline preserved: never persisted in providence_cache,
never threaded into build_run_dag's verify_payload (test pins this
invariant — sidecar values must not enter run_dag_root). The
distribution is recoverable from claim_statuses which IS persisted,
so future sidecar tools can recompute on demand without enlarging
the audit chain.
JP dinosaurs benchmark currently triggers the warning at 11/11
ratio 1.00 — Hermes-3-8B cites [E1] for every claim even with
G0.3's relevance ranking putting the dinosaur-mention chunk near
the top. The render-layer signal now makes that lazy-anchor
behavior explicit instead of leaving it for the auditor to spot
from the spotlight excerpts.
Tests: +4 covering the ratio across distributions (single-anchor /
diversified / no-verified-pairs) plus the run_dag isolation
invariant. Total 442 passing.
New policy["answer_mode"] = "claim_lattice_pointer" (default "quote").
The runtime builds an evidence map with two-layer ids — pointer-id
(E1, E2, ...) shown to the model, sha256-derived evidence-id used by
the cache & run-DAG — and the model emits pointer-line prose
("Claim. [E12]") referencing them. Renderer interpolates literal
source spans at display time. Synthetic-elision-by-construction-
impossible: the model never types the quote string.
Pieces:
- aborist/qa/evidence.py (new): EvidenceObject + spotlight excerpt
(claim-token-centered window into the cited span, falls back to
leading window when no token matches).
- aborist/qa/parse_claims.py (new): pointer-line parser walks
lines, pulls [E\d+] / [E\d+,E\d+,...] tags, returns ParsedClaim
with PARSED / NO_EVIDENCE_POINTER status. Strict regex refuses
fuzzy alternatives so honest UNGROUNDED beats lax acceptance.
- aborist/qa/verify.py: verify_claim_lattice. Hard checks only —
parser succeeded, evidence_id resolves, source_role allowed, no
manual quotes (any " char violates), claim text non-empty.
Returns evidence_id_pairs (content-addressed, run-stable) for the
run-DAG. Soft signals (entailment, completeness, predicate
compatibility) stay sidecar.
- aborist/qa/dag.py: 9-stage CTI shape when evidence_map_root is
supplied — question / retrieval / evidence_map / prompt /
raw_answer / parsed_claim_lattice / verify / render /
final_label. Quote mode keeps the original 7-stage shape so
pre-G0 run_dag_root values stay valid.
- aborist/qa/query.py: per-chunk evidence-map build with role-
weighted budget AND query-relevance ordering. Within each source,
chunks are ranked by (distinct_query_tokens_present,
total_mentions, doc_order_asc) so the chunk most likely to
textually support the question gets the lowest pointer id. Without
this re-rank Hermes-3-8B lazy-anchored on doc-order-first chunks
regardless of relevance.
- aborist/qa/runner.py: same answer_mode branch for the single-doc
ask() path. No iterative repair in pointer mode (one-shot
benchmark discipline).
- aborist/store.py: verifier_method CHECK extended with
'claim_lattice'. New _rebuild_providence_cache_claim_lattice
migration preserves run_dag_root / run_dag_blob across the
rebuild — older rebuilds dropped them.
- aborist/cli.py: --answer-mode {quote,claim_lattice_pointer} on
query and ask.
- Makefile: ANSWER_MODE knob; defaults to claim_lattice_pointer for
`make query` so the testing harness exercises the new path.
Library DEFAULT_POLICY / DEFAULT_QUERY_POLICY stay "quote" so
Python callers and unit tests aren't surprised.
Prompt: one-shot worked example (Apple founders) plus strict
no-quotes / no-JSON / no-markdown / plain-prose-only rules. Without
the worked example Hermes-3-8B drops the bracket protocol on roughly
half of runs; with it, JP dinosaurs benchmark went 0/13 -> 10/11 ->
16/17 verified across the iterations that hardened the pipeline.
Tests: +48 covering parser, two-layer ids, verifier failure modes
(UNKNOWN_EVIDENCE_ID / SOURCE_ROLE_BLOCKED / MANUAL_QUOTE_VIOLATION
/ NO_EVIDENCE_POINTER / SCHEMA_INVALID), spotlight rendering with
buried-term fixture, per-chunk evidence map, query-relevance
ordering, schema migration round-trip, 9-stage DAG shape divergence.
Total 438 passed, all 7 production shards report 0 chain breaks.
Known limitation: chunk boundaries can cut wikitext mid-template, so
mwparserfromhell-backed to_base() leaves orphan </ref> tags and
leading list markers in the visible spotlight excerpts. Verifier and
CTI architecture are unaffected; the leak is cosmetic. Proper fixes
are template-aware chunking (chunker bump invalidates prior records)
or an orphan-marker post-strip in aborist/wikitext.py — both out of
G0 scope.
Two enhancements continuing the toy-Hermes design pass:
#1 Chain-segment failure localization
aborist/qa/dag.py: `localize_failure(audit_mode, n_sources, n_quotes,
n_verified)` maps a non-STRICT verdict to the pipeline stage that
introduced the failure:
retrieval no admitted sources (gate over-rejected, or corpus
genuinely lacks the topic) → ingest more / relax breadth
context sources retrieved but no quotes extracted (per-source
cap dropped relevant content, or model declined to cite)
→ raise cap / tighten prompt
answer quotes extracted but didn't all verify (model fabricated,
paraphrased inside quotes, appended citations) → mechanical
+ re-prompt repair targets exactly this case
`failure_stage` lands on the run_dag's verify node payload AND on the
result dict so an operator can read the reason at a glance — `failure_stage='answer'`
means stop tuning the verifier & fix the model behavior. Debugging
becomes typed instead of vague.
#2 Re-prompt repair (second tier of the hybrid loop)
aborist/qa/repair.py: `reprompt_repair(...)` builds a feedback message
naming the failed quotes & asks the model to rewrite using only
verbatim citations. Hard rule: only fires when
`policy["repair_max_reprompts"] > 0` (default 0); caller enforces the
cap by looping at most that many times.
aborist/qa/query.py + aborist/qa/runner.py: after mechanical repair,
if the answer is still HYBRID/UNGROUNDED with unverified quotes,
loop up to `repair_max_reprompts` times. Each iteration: build
feedback (assistant turn with current answer + user turn with failed
spans), call LLM, verify. Accept the new answer if `n_verified`
strictly improved; otherwise break (the model's not converging,
don't waste cycles).
The mechanical + re-prompt combination handles the cases each tier
declines individually:
mechanical alone: synthetic_elision split, trailing_artifact trim,
no_overlap remove
+ re-prompt: paraphrase, partial_paraphrase, interior_elision
needing semantic judgment, fabrications the
model can recognize when shown its own quote
`repair_max_reprompts` lives in DEFAULT_QUERY_POLICY +
DEFAULT_POLICY so it folds into governance_policy_hash. Default 0
preserves single-shot semantics for callers that don't opt in. Each
re-prompt iteration adds a `{action: reprompt_rewrite, diagnosis:
model_feedback_loop}` entry to repair_changes; audit chain captures
the full transition through the existing providence_repair event.
Tests:
- dag: localize_failure across all four cases (STRICT, retrieval,
context, answer); failure_stage embedded in run_dag verify node.
- repair: stub client with sequenced answers (failing first, clean
on re-prompt) — assert two LLM calls, STRICT verdict, reprompt_rewrite
in the change log.
484 tests pass (dag +5, repair +1). The pre-existing test_burn flake
under full-suite ordering remains; passes in isolation.
The system was observational: verifier classified, sidecar diagnosed,
repair plans were emitted — but no loop ever closed. This adds the
hybrid repair stage from the toy-Hermes design pass: mechanical first
(deterministic string substitution from sidecar suggestions), behind
a `policy["repair_enabled"]` flag (off by default). Re-prompt fallback
is TODO.
aborist/qa/repair.py: `mechanical_repair(answer, unverified_quotes,
context)` walks each unverified quote through the sidecar classifier &
applies its `repair` action by string sub:
synthetic_elision_inside_quote (both halves verbatim)
`"prefix [...] suffix"` → `"prefix" ... "suffix"`
Two verbatim spans the verifier can independently check; the
model's [...] ellipsis-marker becomes prose between them.
trailing_artifact
`"prose. (Source: ...)"` → `"prose."`
Verbatim prefix kept; model-appended tail dropped.
no_overlap
Drop the line containing the bad quote entirely.
Skips include_aside_for_verbatim (needs precise source-span extraction;
defer to re-prompt path), paraphrase / partial_paraphrase (need prose
rewriting). Idempotent.
aborist/qa/query.py + aborist/qa/runner.py: optional pass after first
verify. When `repair_enabled=True` AND `audit_mode != "STRICT"` AND
unverified quotes exist:
1. Run mechanical_repair on the answer text.
2. If repair produced any changes, re-verify the repaired text.
3. If post-repair verdict isn't worse (n_verified didn't decrease),
accept the repair: persist the REPAIRED answer text instead of
the model's original. Cache_key inputs unchanged.
4. Audit chain gets one `providence_repair` event with the change
log + pre/post verdict so the original→repaired transition is
reconstructable.
Result dict gains `repair_changes` (list of change records) and
`pre_repair_audit_mode` (what the original was classified as).
`policy["repair_enabled"]` enters governance_policy_hash so on/off
agents share no cache silos.
Tests:
- mechanical_repair on each diagnosis (synthetic_elision, trailing_artifact,
no_overlap), idempotence on clean text.
- query() integration: repair_enabled=False (default) leaves answer
text unchanged; repair_enabled=True promotes a HYBRID/quote
synthetic_elision case to STRICT/quote, persists the repaired text,
emits the providence_repair audit event.
479 tests pass (+6 repair).
Each query/ask call now emits a 7-stage Merkle-DAG fingerprint stored
alongside the providence record. F from the toy-Hermes design pass.
Stages, in order:
question hash of question_hash (the 8-dim cache_key dim)
retrieval hash of sources summary (document_roots + roles +
scores + chunk_idx) — captures which docs ranked
context context_root (the source-Merkle for the assembly)
prompt conversation_hash
answer sha256(answer_text)
verify hash of verdict (audit_mode, verifier_method,
n_quotes, n_verified, claim_statuses)
final_label hash of (audit_mode, verifier_method, lookup_path)
run_dag_root = MerkleTree over those stage hashes (aborist conventions:
non-commutative HashCombine 0x03, leaf prefix 0x00, self-dup odd rule).
run_dag_blob = canonical JSON of {root, nodes} so an auditor can
recompute & verify (`verify_run_dag(blob)` returns True/False).
The DAG is NOT in cache_key. cache_key inputs determine the answer; the
answer determines the DAG — folding it back would create a cycle.
Instead it rides alongside as a per-record computation fingerprint.
Distinct from the linear `audit_events` chain (which tracks DB-wide
state changes); this is per-run computation provenance.
Schema: ALTER TABLE providence_cache ADD COLUMN run_dag_root TEXT;
ADD COLUMN run_dag_blob TEXT;
Idempotent migration in `_migrate_audit_mode`. Both rebuild templates
(VISUAL→UNGROUNDED dance, paraphrase verifier_method dance) updated to
include the new columns. Legacy records pre-2026-04-30 carry NULL.
Result dict gains `run_dag_root` so callers can verify without a DB
round-trip.
Tests:
- test_dag.py (9 tests): determinism, reactivity to each stage's input,
fixed stage order, round-trip verify, tamper-detection, JSON-string
acceptance.
- test_query.py: persistence on record + result, verify_run_dag round-
trip on the persisted blob.
473 tests pass (DAG +9, query +1, integration unchanged).
Two paired enhancements (D + E from the toy-Hermes design pass):
D. Per-claim status taxonomy on `verify_quotes`.
New `claim_statuses` field on every verdict — a per-evidence-unit
list with three labels:
VERIFIED_QUOTE unit substring-matched in normalized context
(any of quote/span/entity strategies)
SUPPORTED_PARAPHRASE unit cleared the paraphrase token-coverage
threshold (≥85% topical tokens present)
UNSUPPORTED unit didn't match anything
Diagnostic labels (QUOTE_INTEGRITY_FAILED, SOURCE_MISMATCH,
FALSIFIED) stay in the sidecar / falsification machinery — the
binary-verifier discipline holds. Empty list when no evidence
was extracted at all (verifier_method='none'). Backward-compatible:
existing fields (audit_mode, n_quotes, n_verified, unverified_quotes,
verifier_method) unchanged; current callers ignore the new field.
E. Repair-action plans on sidecar diagnoses.
Each `_classify_span` diagnosis now carries a `repair` field with
a concrete suggestion the operator can act on:
synthetic_elision_inside_quote → split_into_two_quotes
(when both halves verbatim)
→ trim_to_verified_half
(when only one half verbatim)
→ remove_claim
interior_elision → include_aside_for_verbatim
(with the dropped aside text)
trailing_artifact → trim_trailing_artifact
(with the kept_prefix string)
paraphrase → downgrade_to_paraphrase
partial_paraphrase → split_or_remove
no_overlap → remove_claim
Read-only suggestions — sidecar still doesn't write to providence_cache
or audit_events. The repair stage is recommendation, not mutation.
Operator (or an automated repair pass) decides whether to act.
Human render in `aborist inspect` shows `repair: <action> (<reason>)`
under each diagnosis line.
Tests:
- verify: claim_statuses_quote_path_labels_each_unit (per-quote VERIFIED
/ UNSUPPORTED), claim_statuses_paraphrase_method_flagged,
claim_statuses_empty_when_no_evidence.
- inspect: repair_synthetic_elision_split_when_both_halves_verbatim,
repair_interior_elision_includes_aside, repair_trailing_artifact_trim,
repair_no_overlap_remove.
462 tests pass (verify +3, inspect +4).
Two enhancements from the toy-Hermes design pass (2026-04-30):
A. Synthetic-elision-inside-quote diagnosis (sidecar only).
Distinct from interior_elision (model dropped a `(...)` aside source
carries) — synthetic_elision is the model writing literal `[...]`
between fragments of a `"..."` span, signaling self-elision while
claiming verbatim citation. The verifier still rejects (binary
discipline holds), but `aborist inspect` now reports
`diagnosis: synthetic_elision_inside_quote` with prefix/suffix
presence flags so an operator can judge whether the elided middle
was benign. Probe runs first in the classify-span chain (more
specific than trailing_artifact / interior_elision / paraphrase).
Catches the Brachiosaurus case: `"The film centers on the fictional
Isla Nublar [...] Universal Studios..."` — both halves are in source,
but the literal `[...]` isn't, so substring match correctly fails &
the sidecar tells the operator why.
C. Source-role classification + role-weighted context budget.
`_classify_source_role(title, qtokens_stem)` tags each top-K hit:
primary_answer_source 2.0× cap strong title-stem overlap
secondary_context_source 1.0× cap "list of", "characters",
"franchise", "history of"
noisy_background_source 0.5× cap "score", "music",
"video game", "merchandise"
sequel_background_source 0.5× cap "lost world", roman numerals
background_source 1.0× cap default
Order matters: noisy/sequel/secondary markers fire before the
primary check so peripheral pages with strong title overlap (e.g.
`Jurassic Park (film score)` shares 3 stems with the JP-film query)
don't claim a primary slot.
Cap loop now applies role weight on top of the baseline
`max_context_chars / top_k`. Total context still bounded by the
running `char_budget` — weights just shift how the budget gets
divided so primary pages get more text & noisy pages less, fixing
the case where a `(film score)` page consumed a primary slot.
`source_role` is persisted on `_Hit` and surfaces on
`merkle_proof.sources[*].source_role` in the providence record so
inspect & audits can see which slot each source occupied.
Tests:
- inspect: synthetic_elision_caught (Brachiosaurus regression),
synthetic_elision_does_not_fire_when_source_has_brackets (false-
positive guard).
- query: role classifier matrix (primary / secondary / noisy / sequel /
background) on JP-film-style titles, role persistence in sources list.
455 tests pass (sidecar +2, query +2).
Per fox: while iterating on retrieval/verifier knobs, a flag on the
query itself is more useful than a separate verb. One step reset:
change a knob, re-query, see fresh result.
Surface:
aborist query --burn ...
make query Q="..." BURN=1
make query-dry Q="..." BURN=1 (works on dry-run too)
Behavior in `aborist.qa.query.query()`:
- New `burn_existing: bool = False` parameter on query().
- After computing primary cache_key (per the active dedup mode) but
BEFORE the cache lookup, if burn_existing: DELETE the live row that
matches the primary cache_key & write one providence_burn audit
event with reason "query --burn (test-ergonomic mid-query bust)".
- The lookup then misses → fresh inference runs. Result reports
`burned_existing: 0|1` so the caller sees whether anything got
busted.
- The equivalence-class fallback cache_key is deliberately NOT
touched: prior alt-mode records stay as historic witnesses.
Tests (3): cache populates → cache hits → BURN=1 forces fresh
inference; --burn writes a providence_burn audit event; first-time
query with --burn is a clean no-op (burned_existing=0). 356 passed,
1 skipped.
Complementary to `aborist burn-kindergarten` (mass reset) — this is
the surgical version. Both are local-only by design; for cross-peer
invalidation use `make falsify` and let mesh sync broadcast.
Per fox: useful while iterating on retrieval/verifier knobs to wipe
recent test runs without finding each cache_key. Mirrors the mesh-sync
kindergarten window so what's still un-broadcast (private to this peer)
is exactly what's safe to bust without confusing peers.
Surface:
aborist burn-kindergarten [--kindergarten-seconds N]
[--reason '...'] [--by-actor X]
[--force] [--dry-run] [--verbose N]
make burn-kindergarten [SECONDS=3600] [FORCE=1] [DRY_RUN=1] [REASON='why']
Behavior:
- Walks every shard, finds providence_cache rows with
created_at >= now - SECONDS and falsification_state='live'.
- Each row goes through the existing _burn_cache_key — children gate
honored unless --force, audit event written per successful burn,
chain integrity preserved.
- Result JSON reports examined / burned / refused_has_children /
not_found counts plus a verbose tail of per-row results.
- --dry-run reports without writing or auditing.
- --kindergarten-seconds 0 = burn every live row (test reset).
Tests (4): selective burn (old kept, fresh burned), dry-run writes
nothing, 0-second window burns everything, children-gate refusal
without --force. 353 passed, 1 skipped.
Operational note: this command does NOT propagate to peers (burn is
local kindergarten cleanup by design, see docs/mesh.md). If you want
the remote effect, falsify each record individually & let mesh sync
broadcast the falsifications instead.
Fox 2026-04-29: 'who is supermans girlfriend?' returned 7-of-8 unrelated
`Girlfriends`-titled articles (TV show, movies, songs); only 1 actual
Superman-related doc made it. Two coupled defects:
1. Title-overlap accept fired on ANY single-token match. A 2-token
query was admitting docs that shared only ONE qtoken. `Girlfriends`
passed because its title matched "girlfriend" even though no
"superman" anywhere in the doc.
2. `_body_density_passes` required at least HALF the qtokens — too
lenient for the 2-token case (1 of 2 = 50% = pass).
3. Possessive plural mismatch: question_hash strips apostrophes so
"superman's" → "supermans", but the corpus has bare "Superman".
`body.count("supermans")` returns 0 even on the canonical doc.
Three coordinated fixes in `aborist/qa/query.py`:
- New `_stem_token_for_match(t)` helper: strips a trailing `s` for
tokens > 4 chars (so `supermans` → `superman`, `girlfriends` →
`girlfriend`). Conservative: skips short tokens & double-s endings
to avoid `class` / `boss` / `pass` corruption.
- `_body_density_passes` now uses `_body_count_with_stem` (literal-
first, stem-fallback) AND a tightened breadth threshold:
≤ 2 tokens require ALL of them
3+ tokens require N - 1 (allow one weak signal token to miss)
- `_filter_by_title_relevance` mirrors the same breadth threshold for
title-overlap. Synonym fallback (any-match against synonym_expand)
is preserved ONLY for 1-token queries — otherwise a stray synonym
hit (e.g. AMD synonym matching an Intel-only title) over-recalls.
End-to-end on the actual corpus (3.4M articles, 8 shards):
before: 'supermans girlfriend' → 7 unrelated `Girlfriends` titles, 1 Superman doc
after: Lois_Lane in top-K (the canonical answer), other Superman-related
docs alongside, Girlfriends-only-titled docs filtered out
Tests:
- test_query_filter_requires_breadth_for_multi_token_queries — synthetic
3-doc corpus pinning the new behavior (Lois Lane keeps; girlfriends-tv
& superman-music both drop).
- test_query_filter_one_token_query_still_synonym_expands — pin the
1-token loose path is preserved (no over-tightening).
349 passed, 1 skipped.
Burned the two stale UNGROUNDED girlfriend-query cache records so a
fresh `make query` exercises the new filter end-to-end.
Different agents have different value functions on the same providence
records. Today aborist bakes one canonicalization policy into
governance_policy_hash and calls it universal. This adds two knobs that
let agents express their preference without breaking provenance.
- aborist/qa/keys.py: `canonical_question(q, mode=...)` and
`question_hash(q, mode=...)` accept "strict" (NFC + ws-collapse only;
every variant gets its own hash) or "equivalence_class" (default —
additionally lowercase + trailing-punct strip + article strip).
New constants: QUESTION_DEDUP_MODES, FIDELITY_MODES, defaults.
- aborist/qa/query.py + aborist/qa/runner.py: each call computes the
primary cache_key under policy["question_dedup"]. New `fidelity`
parameter:
"strict" only primary cache_key checked
"equivalence_class" primary first; if miss AND alternate mode
produces a different cache_key, try alternate
Cross-silo fallback works because _ckey_for_mode rewrites
policy["question_dedup"] to the alternate mode before computing
governance_policy_hash — so the fallback ckey matches what an agent
under that mode would have written. Lookups can find each other's
records when fidelity permits. Result dict gains `lookup_path` ∈
{"strict", "equivalence_class", "strict_fallback",
"equivalence_class_fallback", "miss"}.
- aborist/cli.py: `--question-dedup` and `--fidelity` flags on the
`query` subcommand. Human render annotates cache_hit lines that
came from a fallback ckey ("cached via equivalence_class_fallback").
- tests/test_query.py: three new regressions
* strict-policy + strict-fidelity: variants get distinct cache_keys
* cross-silo fallback: strict-policy reads eq-class-policy records
via fidelity=equivalence_class
* strict-fidelity refuses fallback (audit-grade): cache miss even
when the alternate silo has a hit
- CLAUDE.md: dedup-mode and fidelity convention bullets updated.
This is V1 + V2 of the substrate move sketched in conversation: write
policy determines which silo a record lives in; read fidelity determines
how loosely an agent walks across silos. Records stay exactly-keyed
(provenance hard); routing is per-agent (preference soft). Same
verifier_no_diagnostics discipline holds — no soft signal enters the
hard chain.
442 tests pass.
Fox 2026-04-29 (Clark Kent / Superman query): the verifier flagged a
quoted span as unverified and the sidecar diagnosed it as 'paraphrase'
with token_coverage=1.0. Investigation showed the source actually said
`Clark Joseph Kent (middle name is also Jerome according to some
versions) is a fictional character...` and the model elided the
parenthetical aside, quoting `Clark Joseph Kent is a fictional
character...`. Every word in source, but the sequence has a 60-char
gap where the aside was. Distinct failure mode from real paraphrase
(token reordering) and trailing_artifact (model APPENDS).
Sidecar refinement, no verifier change. The binary verifier discipline
(memory rule `feedback_verifier_no_diagnostics`) holds — quote strategy
is still verbatim-only; soft signal stays in the inspect verb.
Algorithm walks every `(` in base. For each open paren at position P:
the longest k where base[:P] ends with span[:k] is the model's prefix;
the span tail (≥20 chars) must then match the source after the close
paren. First paren that satisfies both checks wins.
CLI human-render shows `matched: N prefix + M suffix chars
(parenthetical aside dropped)` plus the dropped_aside text so an
operator can decide at a glance whether the elision is benign.
Live record now reads:
[1] interior_elision
matched: 17 prefix + 138 suffix chars (parenthetical aside dropped)
dropped_aside: middle name is also jerome according to some versions
439 tests pass (sidecar +2, full suite still green).
Fox 2026-04-29: "part of the gossip protocol should be some default
delay that allows a user to catch and burn kindergarteners before
they are synced." Currently `mesh sync` enumerates the most-recent
N items with no age filter — a doc ingested 30 seconds ago goes
out on the next sync, & if a peer ingests it before the operator
notices a problem, burn no longer suffices (peer has its own copy).
Adds a sender-side kindergarten window:
--kindergarten-seconds N default 3600 (1 hour)
Records younger than `now - N` are held back from broadcast. Both
ANNOUNCE_ROOT (filtered on documents.ingest_ts) & ANNOUNCE_FALSIFICATION
(filtered on falsifications.at). N=0 = broadcast everything (cron-
friendly opt-out for operators preferring immediate propagation).
Result JSON now reports the held counts so the operator can see
what the window protected:
kindergarten_seconds: 3600
announced_roots: 12
kindergarten_held_roots: 3
announced_falsifications: 0
kindergarten_held_falsifications: 1
Sender-side discipline only — the receiver has no view into when
the sender created the record, so it can't enforce. Adding a
created_at on the envelope would let receivers reject too-fresh
gossip, but that's a future protocol bump (envelopes today don't
carry sender wall-clock; ts is the send time).
Tests:
- existing tests now pass `--kindergarten-seconds 0` so freshly-
ingested fixtures broadcast immediately for the test
- new test_sync_kindergarten_holds_fresh_records: a 30-second-old
doc is held; an artificially-aged doc broadcasts. announced=1,
held>=1.
- new test_sync_kindergarten_zero_broadcasts_everything: explicit
opt-out works.
342 passed, 1 skipped.
Fox caught on 2026-04-29: 'who is batman?' and 'who is the batman?' produced
different cache_keys despite question_hash collapsing both variants. The 8-dim
cache_key has TWO seams that touch question text — question_hash AND
conversation_hash (which hashes the messages list with the literal question
in the user turn). Only the former canonicalized; the latter took bytes as
written, so each variant got its own chash and missed cache.
Fix: decouple the two forms.
- aborist/qa/keys.py: extract canonical_question(text) → str; question_hash
composes _sha256(canonical_question(t)) so the equivalence class is
defined in one place.
- aborist/qa/query.py + aborist/qa/runner.py: build TWO message lists per
call. `messages` carries the verbatim question (Hermes sees the user's
natural phrasing — no grammar drift). `canonical_messages` substitutes
the canonical form, and conversation_hash hashes that. The LLM still
gets 'Who Is THE Batman?'; the cache_key collapses to 'who is batman'.
- tests/test_query.py: regression — three variants (with/without `?`,
with/without `the`) hit the same cache_key. First call populates,
later variants cache_hit. Captures stub messages to confirm the LLM
saw the verbatim question, not the canonical form.
437 tests pass.
Per fox 2026-04-29: the wire protocol has had ANNOUNCE_FALSIFICATION
since the foundation commit (and MeshWireClient.announce_falsification
since AEAD landed), but the user-facing `mesh sync` only ever fired
ANNOUNCE_ROOT. Falsifications local to one peer never reached others
unless an operator hand-rolled a Python script.
Now `mesh sync` enumerates BOTH categories:
- ANNOUNCE_ROOT most-recent --limit documents (existing path)
- ANNOUNCE_FALSIFICATION most-recent --limit falsifications (new)
Receivers verify Ed25519 sig + per-peer chain-of-claims as before,
write one mesh_received audit event per accepted envelope. Result
JSON now reports both counts:
announced_roots: N (was: announced)
announced_falsifications: N
sent_roots: [...] (was: sent)
sent_falsifications: [...]
Two opt-out flags so operators can scope the broadcast:
--no-roots only push falsifications
--no-falsifications only push roots
Burns are deliberately NOT propagated. Burn semantics are local
kindergarten cleanup ("delete a leaf I shouldn't have written") —
other peers may have legitimately ingested the doc independently.
Falsify is the audit-preserving alternative whose broadcast IS the
right cross-peer signal for "this answer is wrong."
Caveat (deferred): no per-peer dedup state yet. Re-running sync
re-broadcasts the same most-recent N falsifications; receivers get
duplicate mesh_received audit-log entries (no state corruption,
just log noise). A `mesh_sync_state` table tracking
last_falsify_announced_ts per peer URL is the natural follow-up
when the falsification volume grows.
Tests:
- existing test_sync_announces_local_roots updated for new field
names (announced_roots, announced_falsifications)
- new test_sync_announces_falsifications: bob falsifies, syncs,
alice's chain has the ANNOUNCE_FALSIFICATION envelope
- new test_sync_no_falsifications_flag_skips_them: --no-falsifications
skips the broadcast cleanly
339 passed, 1 skipped.
Fox 2026-04-29: querying "who is batman" returned only ONE source
(List_of_Batman_comics — an 80 KB+ bibliography) despite top_k=8 &
the actual bio article being in the corpus. Greedy fill: hit #1
consumed the entire 60 KB budget, every subsequent doc dropped with
char_budget <= 0.
Fix in aborist/qa/query.py:
per_source_cap = max(1, max_context_chars // max(1, top_k))
for h in hits[:top_k]:
text = _load_doc_text(...)
if len(text) > per_source_cap:
text = text[:per_source_cap] # NEW: per-source cap first
if len(text) > char_budget:
text = text[:char_budget]
...
Each top_k hit gets at most max_context_chars/top_k chars (default
60K/8 = 7.5K each — plenty for a chunk or two of prose). Total
context ≤ max_context_chars by construction. top_k=1 preserves the
legacy behavior (single source can use the full budget).
End-to-end effect on Batman: the bio (Wikipedia/Batman article) lands
in context alongside List_of_Batman_comics; the model can paraphrase-
verify against the actual character introduction text instead of
fabricating from training.
Tests: 2 regressions in tests/test_query.py — multi-source delivery
when hit #1 is huge, and top_k=1 single-source still allowed full
budget. 337 passed, 1 skipped.
Burned the two stale Batman cache records (chain extended) so a
fresh `make query Q="who is batman?"` exercises the new path.
Two related changes that tighten the dedup + grounding signals
without lowering quality bars.
1. question_hash drops standalone English articles (the/a/an).
Fox 2026-04-29: `who is the batman?` and `who is batman?`
produced different cache records; same question, different hash.
Articles are filler at the question-equivalence layer. Add a 4th
canonicalization step to question_hash: after lowercase + trailing
punctuation strip, split on whitespace & drop tokens equal to
"the" / "a" / "an", then rejoin.
Equivalence class now includes:
"who is X" ┐
"who is the X" │
"who is a X" │ -> same question_hash
"who is an X" │
"Who Is X？" ┘ (CJK question mark)
"thesis" stays untouched — exact-match standalone tokens only,
not substring. Conservative on i18n: ASCII English articles only;
"el / la / le / der / die / das" etc. await demand.
2. _token_coverage filters stopwords and per-token punctuation.
Fox asked: should we lower the 0.85 paraphrase threshold? Honest
answer: no — that would promote the Q1 Batman fabrication
("wealthy/businessman/resides" missing from corpus) to STRICT.
Tighten the signal instead so 0.85 means more.
- Per-token punctuation strip (.,;:!?\"()[]{}) so `wayne,` lines
up with bare `wayne` in context. Apostrophes deliberately stay
so `batman's` is distinct from `batman`.
- English stopword filter on length-≥4 fillers (from / with /
have / been / would / which / where / their / etc., curated set
in _ENGLISH_STOPWORDS). These match almost any English text &
inflate coverage scores when topical content is missing.
Net effect on Q1 Batman case: missing tokens are
`wealthy/businessman/resides` — all topical. Coverage stays well
below 0.85, span stays UNGROUNDED. Net effect on a stylistic
paraphrase (model wrote `from` instead of `with`): coverage
computed only over topical tokens, both copies match → 1.0 →
correctly promoted.
Tests: 8 new in tests/test_keys.py (article-strip equivalence
classes, substring preservation, distinct-topic non-collapse) +
2 in tests/test_verify.py (stopword filter doesn't inflate; 0.85
threshold still rejects fabrication). 335 passed, 1 skipped.
Note for fox: the underlying retrieval issue surfaced in Q1/Q2 is
independent — Batman main article IS in shard 002.db but FTS5 +
title rerank ranked List_of_Batman_comics higher. Different fix,
separate commit.
Two changes that make the verifier accept more honest grounding without
relaxing what STRICT means.
1. Strip trailing citation parentheticals at extraction time.
`extract_quotes` and `extract_claim_spans` now drop a single trailing
`(Source: ...)` / `(citing X)` / `(https://...)` parenthetical
before returning. The Pikachu real-corpus case: model wrote the
verbatim source sentence and appended `(Source: https://...)`. The
prose IS in the corpus — only the inserted citation broke substring
match. Now substring fires; STRICT recovered.
Conservative regex: only strips when the parenthetical content
starts with a citation cue word OR contains a URL. A genuine prose
parenthetical like "Pikachu (a Pokémon species) lives in forests"
stays.
2. Paraphrase strategy as a 4th verifier method.
In the span path, items that fail substring match get a
token-coverage probe. Tokens >=4 chars from the span; if >= 85% of
them appear in the normalized base context, count the span as
paraphrase-verified. Paraphrase items contribute to n_verified;
verifier_method flips to 'paraphrase' when any soft-verified items
are present so an auditor can tell.
Gated by `_is_prose_span`: a list of proper nouns ("Keanu Reeves,
Laurence Fishburne") has no lowercase content tokens >=4 chars and
falls through to the entity strategy where proximity policy can
tight-cluster check. Real prose spans ("Pikachu is a species of
Pokémon creatures...") have multiple lowercase content tokens and
qualify for paraphrase.
Soft-signal note (from CLAUDE.md soft/hard rule): paraphrase is
heuristic, not byte-equivalence. The hard chain still records the
classification — verifier_method='paraphrase' is the explicit marker
that an audit reader can use to distinguish lexical-verbatim from
paraphrase-overlap. Quote strategy stays verbatim-only — quotes ARE
quotes; paraphrase-in-quotes is the model's mistake to flag, not
auto-promote.
Schema: verifier_method CHECK constraint expanded to include
'paraphrase'. New idempotent migration `_rebuild_providence_cache_paraphrase`
mirrors `_rebuild_providence_cache_ungrounded` (table rebuild via
BEGIN IMMEDIATE; no value translation needed). Probes existing CHECK
from sqlite_master and no-ops if already expanded.
Tests: 9 new in tests/test_verify.py — 4 trailing-citation strip
(quote+span variants, URL-only parenthetical, end-to-end STRICT
recovery), 4 paraphrase strategy (high-coverage promote, low-coverage
reject, mixed-method label, quote-strategy preservation). Default
suite still green: 326 passed, 1 skipped.
Burned the live Pikachu cache record (audit chain extended) so a
fresh `make query Q="who is pikachu?"` exercises the new verifier
end-to-end.
Per fox: the verifier reports n/N verified but doesn't tell you WHY a
span didn't ground. Some are paraphrase, some are model-added
trailing artifacts (e.g. `(Source: https://...)` citations the model
appended to verbatim prose), some are full inventions. Operator
needs to see which is which to triage.
Read-only sidecar — no audit events, no providence_cache mutations,
no v9.8 field changes. Per the verifier-no-diagnostics memory rule:
hard chain stays binary; soft signals live in sidecar verbs that
never feed back.
Five diagnosis labels:
verbatim_in_base span IS in base context — verifier or
canonicalization bug (worth flagging loudly).
verbatim_in_raw_only raw wikitext match but base form differs —
wikitext-strip edge case.
trailing_artifact a >=60-char prefix matches; the tail (often a
model-added citation) doesn't. Surfaces the
tail explicitly.
paraphrase >=85% query tokens (>4 chars) present in
base context but not in this sequence.
Model rewrote source content.
partial_paraphrase 40-85% coverage — mixed sourced/emergent.
no_overlap <40% coverage — likely full invention.
Verified live against the real Pikachu cache record (5 sources,
71KB raw -> 34KB base): two spans flagged paraphrase (token_coverage
1.0 for both, just rewritten sequence), one flagged trailing_artifact
(100-char prefix matches; tail is `(Source: https://...)` citation).
That's the actual reason HYBRID — model paraphrased + appended
citations that aren't in the corpus.
Surface:
aborist inspect --cache-key <hex> [--qa-db ...] [--json]
make inspect KEY=<hex> [JSON=1]
Tests: 9 (6 classifier on synthetic contexts pinning each label,
3 end-to-end with seeded fixture record + verification that inspect
writes nothing). 318 passed, 1 skipped overall.
Two things fox surfaced from a "who is pikachu?" run:
1. Output emitted "Pok\\u00e9mon" instead of "Pokémon" — json.dumps
defaulted to ensure_ascii=True. Switched to ensure_ascii=False on
every user-facing dump in cli.py (41 sites). The one canonical-JSON
call (`separators=(",", ":")` for storing in providence_cache as
a JSON column, not for hashing) was deliberately left alone.
2. The actual answer was buried under cache_key / context_root / per-
source metadata / timings JSON. Default now renders human-readable:
who is pikachu?
HYBRID 1/2 verified via quote 9.2s (fresh)
Pikachu is a species of Pokémon creatures...
sources (3):
[1] Pikachu — en.wikipedia.org/wiki/Pikachu (002.db)
[2] List_of_Pokémon — en.wikipedia.org/wiki/List_of_Pokémon
...
unverified (1):
- "spans the model couldn't ground..."
cache_key: 35ab7d33… <run with --json for full record>
Pass `--json` to get the prior raw record (still ensure_ascii=False
so unicode renders cleanly there too — scripts parsing the output
see real chars; the JSON spec accepts either form).
Implementation:
- `_render_query_human(result, question)` — pure function, easy to
unit-test, no I/O. Truncates unverified spans over 100 chars,
omits empty sections, distinguishes cached vs fresh in the summary.
- `_strip_scheme` / `_short_path` — small helpers for source display.
- `--json` flag on the `query` subparser; default is human render.
Tests: 12 new in tests/test_cli_render.py — question position, summary
fields (audit/n_verified/method/elapsed/cache-status), unicode
literals (Pokémon not \\u00e9), source line shape, long-quote
truncation, empty-section omission, error fallback, short cache_key
with --json hint. 309 passed, 1 skipped overall.
Per fox: ASCII-only stripping leaves CJK full-width forms behind, so
"who is X？" and "who is X" still produce different cache_keys
despite being the same question. Expanded strip set:
ASCII: . ? ! , ; :
CJK: ？(U+FF1F) ！(U+FF01) 。(U+3002) 、(U+3001)
ellipsis: …(U+2026)
Pairs deliberately stay out of the set:
" ' ) ] }
Stripping one side of a pair breaks balance. `who said "X"?` after
stripping `?` is balanced; further stripping `"` would yield
`who said "X` — different equivalence class than the original. And
apostrophes carry meaning: `X's` is a different question from `X`.
Lifted the strip set into a module constant `_QUESTION_TRAILING_STRIP`
so anyone considering an addition has a documented anchor.
Tests: 11 new in tests/test_keys.py — 6 CJK / ellipsis equivalences,
5 pair-preservation cases (double-quote, single-quote, paren, bracket,
brace). 24 keys tests + 297 default suite, all passing.
Whitepaper sibling change in ~/git/unfirehose-nextjs-logger/whitepaper/
merkle-providence-reverse-rag-whitepaper.rst — expanded the
question_hash bullet to document the equivalence class. Not committed
in this commit (different repo).
Today `question_hash` canonicalizes (NFC, ws-collapse, lowercase) but
preserves trailing punctuation. So "who is X?" and "who is X" produce
different hashes, different cache_keys, different providence_cache
records — the cache misses on what's semantically the same question.
Add `rstrip(".?!,;:")` as the third canonicalization step inside
question_hash specifically. Local to question hashing — chunk leaf
hashes go through `canonicalize()` directly and are unaffected.
Equivalence class after this:
"who is X" \
"who is X?" |
"who is X." | -> all same question_hash, all same cache_key
"who is X!" |
"Who Is X" /
Internal punctuation is preserved on purpose: "X, then Y" carries
meaning that "X then Y" doesn't, even though both have the same
content tokens.
One-time impact: prior cached records whose canonical question ended
in punctuation become orphans on lookup (re-derive on next ask).
History stays on disk; nothing burned automatically.
New test file tests/test_keys.py with 13 tests pinning the question_hash
equivalence class plus light coverage of the other dim hashes
(model_profile, conversation, governance, cache_key 8-dim invariant).
286 passed, 1 skipped.
Per fox: a query for "who is Russell Ballestrini" classified STRICT by
grounding 6/6 quotes against `feeds/all.atom.xml` — a 230-chunk dump
of post metadata. The verifier was technically correct (every quoted
string IS in the feed) but the result was hollow: feeds list URLs,
they don't carry knowledge. The ACTUAL bios live at the linked posts.
Two-pass filter in _CrawledHtmlSource.iter_documents:
1. Pre-fetch URL-pattern check (_looks_like_feed_url):
- suffix matches: .atom, .atom.xml, .rss, .rss.xml, .rdf,
/feed.xml, /atom.xml, /rss.xml, /rss2.xml, wp-rss2.xml,
wp-atom.xml, wp-rdf.xml, wp-rss.xml
- substring matches in path: /feed/, /feeds/, /atom, /rss, /sitemap
2. Post-fetch Content-Type check (_looks_like_feed_response):
- rejects: application/atom+xml, application/rss+xml,
application/rdf+xml, application/xml, text/xml
- keeps: application/xhtml+xml (xhtml IS html, just stricter syntax)
Defense in depth — a feed served at a non-feed path (e.g.
/index.html returning text/xml) still gets dropped on Content-Type.
What stays out of the corpus:
/feeds/all.atom.xml (Atom feeds)
/sitemap.xml (XML sitemaps)
/wp-rss2.xml (WordPress RSS)
/feed/ (any feed alias)
What's still allowed:
/post-name/ (real prose pages)
/index.html (HTML)
/page.xhtml (xhtml)
Tests: 12 new parametric cases (8 path patterns + 6 Content-Types +
xhtml positive case). 36 bridge tests + 273 default suite, all
passing.
For fox: the existing feed entry already in the crawl shard was burned
manually via `aborist burn --kind document --root eaf7c8d5...` — 230
chunks gone. Re-running `make crawl-ingest` won't re-introduce it.
The lifted AsyncWebFetcher already had a fast_mode constructor knob;
just unbroken plumbing was missing. fast_mode trades politeness for
throughput:
- timeouts: 5s vs 15-60s
- parallel page workers: cpu_count() * 3 vs 1
- ignores robots.txt 'crawl-delay' directive
- robots.txt 'Disallow' is STILL honored (only the delay drops)
Surface:
aborist crawl --seed-url ... --fast
make crawl-ingest URL=... FAST=1
Tests: two regressions pin that fast=True / fast=False each translate
to the right AsyncWebFetcher(fast_mode=...) construction. Capturing
fetcher fixture to avoid network. 21 bridge tests + 273 default
suite, all passing.
Per fox: 20 was a wrong default — typical sites have many more pages
within depth 2 than that, and the cap was silently truncating real
crawls. New behavior:
aborist crawl --depth 2 # no page cap
aborist crawl --depth 2 --max-pages 50 # opt-in cap when needed
make crawl-ingest URL=... DEPTH=2 # no page cap
make crawl-ingest URL=... DEPTH=2 MAX=50 # explicit cap
Implementation: bridge's BFS loop treats max_pages=0 as "unbounded"
(while-condition becomes `queue and (max_pages == 0 or len(...) < max_pages)`).
Default flows from CLI argparse default=0 down to the bridge.
Tests: pinned max_pages=0 → no cap with a 30-page chain regression
test (19 passed in tests/crawler/test_bridge.py). Default suite still
273 passed, 1 skipped.
The verbatim AsyncWebFetcher.fetch_webpage defaults extract_links=False
and returns (html, []) when omitted. Bridge's BFS therefore terminated
at the seed — a real-world crawl of russell.ballestrini.net discovered
exactly 1 URL (the seed). One-line fix; the kwarg was always there in
the lifted API.
Added test_crawl_seed_passes_extract_links_true that records every
fetch_webpage call and asserts the kwarg is True. Pin so this can't
silently regress when bridge internals get reorganized.
Updated the stub fetcher signature in the existing tests to accept
the new kwarg without behavior change.
Glues the verbatim crawler lift to aborist's existing ingest pipeline,
plus adds a freshness probe that asks "does this need recrawling?"
without paying for a body fetch when the answer is no.
Bridge (aborist/sources/crawler/bridge.py):
- crawl_seed(seed, depth, max_pages): BFS-discovers same-domain URLs
via AsyncWebFetcher. Exact-hostname filter so subdomain crossover is
off by default. Caps at max_pages. URL dedup, fragment-strip
canonicalization. Sync wrapper around an async core so the CLI
remains a one-liner.
- ingest_crawled(conn, urls): runs the URL list through the standard
ingest_source() path so chunking_version / canonicalization_version
/ Merkle commitment all stay consistent. Captures ETag +
Last-Modified per response and writes one row per document_root
into the new document_http_meta table.
Schema (idempotent migration, mirrors _migrate_mesh_peer_chains):
document_http_meta (
document_root TEXT PRIMARY KEY,
etag TEXT,
last_modified TEXT,
last_fetched_at INTEGER NOT NULL,
last_status INTEGER,
last_checked_at INTEGER,
FK(document_root) -> documents(document_root) ON DELETE CASCADE
)
Recrawl-check (recrawl_check):
- Sends conditional HEAD with If-None-Match + If-Modified-Since.
304 -> fresh (no body transfer needed). 200 -> stale (server says
the body changed). 404/410 -> gone. Other -> unreachable.
- Updates last_status + last_checked_at so consecutive runs visit
oldest-first (NULLS FIRST in ORDER BY).
- Optional --domain filter so operators can probe one site at a time.
CLI:
aborist crawl --seed-url URL [--depth 2] [--max-pages 20] [--ingest]
aborist crawler recrawl-check [--domain DOMAIN] [--limit 100]
Makefile:
crawl-ingest URL=https://example.com [DEPTH=2 MAX=20]
recrawl-check [DOMAIN=example.com LIMIT=100]
Both targets gate on bootstrap-crawler so [crawler] extras land first.
Tests: 17 new in tests/crawler/test_bridge.py — same-domain filter,
BFS bounds (depth + max_pages), dedup, fragment normalization, ETag
capture, content-type filter, HEAD classification (304/200/404),
domain filter, last_checked_at update. All stub the network via a
fake httpx.Client; default suite still 273 passed, 1 skipped.
End-to-end test path:
make bootstrap-crawler
make crawl-ingest URL=https://russell.ballestrini.net DEPTH=2 MAX=20
make query Q="who is russell ballestrini?"
make recrawl-check DOMAIN=russell.ballestrini.net
Lifts the async web fetcher into aborist as an opt-in source. The
implementation comes directly from ~/git/agents.ai.unturf.com/core
(rev 2026-04-28); aborist's adaptations are minimal and documented in
aborist/sources/crawler/__init__.py:
core/async_web_fetcher.py -> aborist/sources/crawler/async_web_fetcher.py
core/web_fetch.py -> aborist/sources/crawler/web_fetch.py
Two source-side changes during the lift:
1. Heavy deps (aiohttp, bs4, miniuri) wrapped in try/except so a bare
`import aborist.sources.crawler` raises ImportError with the install
hint instead of leaking AttributeErrors deep in user code.
2. Chat-bot fetch triggers (`has_fresh_fetch_trigger`,
`has_web_fetch_trigger` from agents.ai.unturf.com/core/keywords)
replaced with NotImplementedError stubs. Aborist has no chat
surface — fetch intent is detected at the application layer. The
two test classes that exercised these triggers are
`@pytest.mark.skip`'d with the same rationale.
Not lifted: web_cache_manager.py — it backs page caching with
SQLAlchemy. Aborist has its own content-addressed cache via
providence_cache; no need to carry SQLAlchemy as a dep just for
crawled-page memoization.
Off by default:
- `[crawler]` extras section in pyproject.toml carries the heavy
deps. `[dev]` pulls them in so the crawler tests can run.
- `make test` ignores tests/crawler/ entirely.
- `make bootstrap-crawler` installs the extras into the venv.
- `make test-crawler` runs only the lifted tests after extras land.
Tests: 74 passed, 9 skipped (the chat-bot trigger tests deliberately
dropped). Default `make test` stays at 273 passed, 1 skipped.
Receiver now rejects gossip envelopes whose prev_event_hash doesn't
extend the sender's last-seen event_hash — fork detection lives at the
wire layer, not just in docs/mesh.md.
Schema: new mesh_peer_chains table (peer_member_id PK, last_event_hash,
last_seq, last_seen_at) with idempotent _migrate_mesh_peer_chains
matching the _migrate_audit_mode pattern.
Envelope: WireEnvelope gains prev_event_hash + event_hash, both
optional for back-compat. canonical_bytes() excludes event_hash to
avoid the circular dependency (the hash is computed FROM the canonical
bytes). New WireEnvelope.with_chain(prev_event_hash=...) builds a
linked envelope with event_hash = sha256(prev || canonical_bytes).
Sender (MeshWireClient._signed_envelope, MeshWireServer.handle_request
DELIVER_BODY): reads our latest_event_hash, builds the linked
envelope, then appends a 'mesh_sent' audit event whose body is the
envelope's canonical (event-hash-excluded) dict. By construction the
audit event_hash equals the envelope event_hash — wire chain-of-claims
and local audit chain stay in lockstep, so back-to-back sends advance
the chain naturally.
Receiver (handle_announce): when an envelope carries chain fields,
recompute event_hash and 400 on mismatch; look up
mesh_peer_chains[sender] and 409 on prev mismatch; on accept update
the row + write the existing mesh_received event.
Backward compat: legacy envelopes (event_hash=None) bypass chain
enforcement — kept the existing test_mesh_wire.py fixtures verbatim
since they construct WireEnvelope directly without with_chain(). All
production traffic goes through MeshWireClient and is always tracked.
v1 deferred (documented in module docstring): no multi-event catchup;
operator retries on 409. No cross-peer reconciliation.
Tests: 12 new in tests/test_mesh_chain.py covering the 7 required
cases plus canonical-bytes exclusion, migration idempotence, and
legacy-envelope fallback. 246 passed, 1 skipped overall.
Adds an opt-in confidentiality layer to gossip. Senders pass
encrypt=True on announce_root / announce_falsification; bodies travel
AEAD-encrypted (ChaCha20-Poly1305) under the per-epoch symmetric secret
recovered via the existing X25519 wrap. Cleartext stays the default —
all existing wire tests pass without modification.
WireEnvelope grows an optional encrypted_body field. canonical_bytes
omits the field when None (byte-identical to v1 cleartext format) and
includes it when set, so the Ed25519 signature binds the ciphertext —
a MITM cannot swap payloads. AAD is type|sender|epoch|ts so a captured
(nonce, ct) cannot be replayed under different envelope metadata.
state.recover_epoch_secret resolves the rotator from the audit chain
and re-unwraps this peer's slot. Sender + receiver share one path; no
new sender-side cache is required.
Closes the request half of the gossip loop. MeshWireClient already
verifies signature + Merkle root on delivered bodies; this verb adds
the CLI surface, gating, local re-ingest through the standard chunker,
and a 'mesh_pulled' audit event. Re-ingest is rejected if the local
document_root differs from the requested root.
burn --kind {providence,document,core}; default 'providence' preserves
the prior --cache-key surface (no breaking change).
Children gates for document/core leaves:
derivations.src_root downstream cores derived from this doc
edges.dst_root other docs link to this doc
providence_cache.source_root Q&A grounded in this doc
derivations has ON DELETE CASCADE on BOTH core_root and src_root, so a
naive DELETE FROM documents would silently cascade-prune downstream
derivations and orphan deeper cores. Gate before delete; --force to
override with the count recorded in the audit body.
Burn manually clears chunks_fts (FTS5 has no FK) and outbound edges
(no FK either) before deleting the document. chunks + merkle_nodes +
inbound derivations cascade via FK.
Audit event types are document_burn / core_burn — distinct from
providence_burn so chain consumers can tell leaf kinds apart.
CLAUDE.md "Cores never evict" remains intact: burn is operator-driven
removal, evict is the automated cold-tier compression that touches
only kind='surface'.
Makefile: KIND=providence|document|core selects the leaf type;
KEY for providence, ROOT for document/core. Single 'burn' target keeps
the surface DRY.
Tests (tests/test_burn_doc.py, 13 new):
- happy path: chunks + merkle_nodes vanish via cascade
- refused on derivation downstream / providence ref / incoming edge
- --force overrides and records counts
- core burn happy path (no children) preserves parent surface
- core burn refused with downstream derivation
- audit chain integrity preserved (no dangling prev_event_hash)
- CLI invocations for document, core, default-providence, refused
Full suite: 247 passed, 1 skipped.
Both ask() (single-doc) and query() (multi-source) now run the
assembled context through aborist.wikitext.to_base() before message
construction. Hermes sees clean prose instead of raw wikitext, which:
- lets the model quote verbatim (raw [[Cloud Strife]] was unquotable
in fluent prose; the model paraphrased to bare "Cloud Strife" and
the verifier flagged that as UNGROUNDED)
- shrinks token bills ~43% on Wikipedia chunks (measured: 125,152 →
71,771 chars on the FF7 main-character query)
- aligns LLM-input with what the verifier already strips, so the
prose-vs-prose substring check is end-to-end consistent
Gated on policy["base_version"] (= wikitext-base-v1) so:
- governance_policy_hash flows the choice — prior answers cached
under raw-wikitext policy retain distinct cache_keys, no migration
- operators can opt out (set base_version=None) for debugging
- environments without mwparserfromhell soft-fall to raw wikitext
with no other behavior change
Tests (4): default policy carries base_version (tripwire), ask() user
messages have wikitext markers stripped, opt-out reverts to raw
wikitext, governance_hash differs with vs without base_version.
Wires the gossip-wire core (commit f141bab) into the user-facing CLI:
aborist mesh serve --host 127.0.0.1 --port 8400
Spawns MeshWireServer, blocks until SIGINT. Refuses if mesh isn't
initialized or .enabled is off — same gate every other mesh verb
enforces. Stdout is unbuffered JSON status lines.
aborist mesh sync --peer http://other.example:8400 [--limit N]
Enumerates local documents (most-recent N, default 100), fires one
ANNOUNCE_ROOT per document via MeshWireClient, reports counts of
acknowledged vs. errored. Pulling missing roots back from the peer
is a v2 addition — this verb pushes only.
Tests (6): gating refusals on uninitialized + disabled mesh; sync
round-trip with two peers proving Alice's audit chain gains one
'mesh_received' per Bob announce; unreachable-peer reporting; serve
smoke-test via start_in_thread + GET /mesh/info.
Burn deletes a providence_cache row outright; falsify (the existing
verb) flips state and keeps the row. Burn is for early/scratch use
during a tree's genesis when downstream consumers haven't ingested
the answer yet.
Children gate: refuses if any falsifications reference the cache_key.
Caller can override with --force; the audit event records forced=True
plus the child count at burn time so the chain can reconstruct what
was orphaned. Refusal is non-destructive AND skips the audit event so
scripts can retry without polluting history.
Always writes a 'providence_burn' audit event regardless of force.
Body fields: cache_key, burned_audit_mode, burned_n_verified,
burned_state, question_text, reason, by_actor, child_falsifications_at_burn,
forced. The chain-break query (`make chain-check`) shows 0 after burn —
the new event extends the prior chain cleanly.
Surface:
aborist burn --cache-key <hex> [--reason '...'] [--force]
make burn KEY=<hex> REASON='...' [FORCE=1]
Tests (7): happy-path delete + audit event, refusal on children,
--force override, unknown-key not-found, chain-integrity preserved,
CLI happy path, CLI non-zero exit on refusal.
- aborist/qa/verify.py: three strategies tried in sequence. quote uses
sequential pairing (1st & 2nd `"`, 3rd & 4th, ...) which eliminates
the phantom inter-pair captures regex pairing produced on adjacent
quote pairs like `"title" prose "quote"`. span checks bullet/sentence
lines verbatim. entity matches multi-word proper nouns; gated by
policy ∈ {strict, hybrid, drop, proximity}. Default proximity
promotes to STRICT only when N=3 verified entities cluster within
W=300 chars in source — separates structural grounding (cast list,
infobox) from incidental mention (scattered plot summary).
- aborist/store.py: schema CHECK now `('STRICT','HYBRID','UNGROUNDED')`.
Migration helper rebuilds legacy `('STRICT','HYBRID','VISUAL')`
tables via temp-table copy, translating VISUAL → UNGROUNDED in the
SELECT. Idempotent — DDL inspection skips the rebuild on already-
migrated DBs.
- aborist/cli.py: new `aborist reclassify` re-runs the verifier
against existing live records under the current entity policy; no
LLM calls. --compare runs all four policies side-by-side, --dry-run
reports transitions without writing. Cold-source records skipped.
One providence_reclassify audit event per changed row.
- AuditMode.UNGROUNDED replaces VISUAL across search backend, FTS5,
test fixtures, CLAUDE.md. The substrate (Merkle-AGI) name was
about FOR-style visualization; in the RAG layer the semantic is
"no recoverable grounding," so the label now says that.
DEFAULT_QUERY_POLICY gains entity_policy + entity_proximity_n +
entity_proximity_window so any tuning folds into governance_policy_hash
and invalidates cache cleanly.
Implements the protocol contract pinned in docs/mesh.md. Each peer
runs a stdlib ThreadingHTTPServer; clients send Ed25519-signed
envelopes carrying one of:
ANNOUNCE_ROOT document_root + source_uri + versions
ANNOUNCE_DERIVATION core_root <- surface_roots (proof-blob hash)
ANNOUNCE_PROVIDENCE cache_key + audit_mode + answer_hash
ANNOUNCE_FALSIFICATION cache_key + reason
REQUEST_BODY/DELIVER_BODY pull-on-miss + Merkle-root verify
WireEnvelope canonicalizes via sorted-key separator-tight JSON; the
canonical bytes are what get signed. Receivers verify the signature
against sender_id's sign_pub looked up in mesh_roster at the
envelope's epoch_id — non-members of that epoch produce no valid
signature, so they're silently rejected (401, no audit event).
Each accepted ANNOUNCE_* writes one 'mesh_received' event into the
local audit chain whose body is the full signed envelope (the
remote's signature stays attached for non-repudiation). The
receiver's chain stays internally consistent because we append in
receive-order.
REQUEST_BODY/DELIVER_BODY: the responder ships the document text
plus per-chunk leaf hashes; the client re-derives the Merkle root
from those leaves and refuses to return any body whose leaves
don't reconstruct the requested root.
Tests: 17 unit (envelope canonicalization, sig verify against
roster, type validation, tampered-body rejection, audit-event
write) + 5 end-to-end (two real HTTP peers on ephemeral ports
exchanging announces and bodies). 22/22 green.
Deferred to later commits, per docs/mesh.md:
- CLI verbs (mesh serve, mesh sync) — pending downstream merge
- per-peer chain-of-claims tracking (catchup, replay, fork detection)
- optional AEAD encryption of message bodies (contract says optional)
Adds aborist/wikitext.py with to_base() — deterministic wikitext →
prose conversion via mwparserfromhell. Drops <ref>...</ref>,
[[File:...]], [[Image:...]], [[Category:...]] entirely; resolves
piped wikilinks to display text; collapses templates, formatting,
and HTML markup. extract_wikilinks() preserves the link graph for
the definition-cloud artifact (recoverable from any page on demand
without re-parsing wikitext at query time).
Wires to_base() into verify_quotes(). Without the strip, the
verifier compared the model's clean prose against [[Cloud Strife]],
[[Shinra Electric Power Company|Shinra]], etc. and falsely flagged
real source quotes as VISUAL. Concrete case from a make-query run:
'Cloud Strife, an unsociable mercenary who claims to be a former
1st Class member of Shinra's SOLDIER unit;' is verbatim in the
Final_Fantasy_VII article wikitext (modulo markup). With the strip
that quote now verifies; the genuine model hallucinations in the
same answer still flag honestly. Side-effect: 43% smaller context
size on average so less LLM token waste.
BASE_VERSION = 'wikitext-base-v1' is the algorithm pin. Bump when
the strip rules change. Soft-imported in verify.py so environments
without mwparserfromhell installed degrade gracefully (no strip,
same behavior as before this commit).
Build:
- pyproject.toml: new [wikitext] extras (mwparserfromhell>=0.6),
pulled in by [dev]
- Makefile: chain-check / chain-check-shards targets — fast
audit-chain integrity probe, counts dangling prev_event_hash
references; 0 = chain intact
Tests:
- tests/test_wikitext.py: 31 tests (rules, idempotence, real-corpus
fixture)
- tests/test_verify.py: 2 regression tests pinning the FF7 flip
(Cloud Strife quote: VISUAL → STRICT after strip; genuine
hallucination: stays VISUAL)
- tests/fixtures/ff7_characters_chunk0.wikitext: real chunk from a
shard, used to validate the strip on actual Wikipedia content
Adds aborist/qa/verify.py — three-strategy verifier (explicit quotes,
bullet/sentence spans, multi-word proper nouns) that lexically checks
every claim against retrieved context under norm-v1 + lowercase. The
strategy that fires is recorded as verifier_method for diagnostics.
Entity strategy gates classification via an entity_policy
(strict/hybrid/proximity/drop) so a single proper-noun match no longer
overclaims STRICT — proximity (default) requires a cluster of 3+
verified entities within 300 chars.
Wires into ask() and query(). System prompts now require verbatim
quoted spans for every factual claim, restated via a user-turn
grounding_reminder one message before sources arrive (recent
user-turn instructions outweigh decayed system-turn rules under long
context in Hermes).
providence_cache gains 5 columns (audit_mode, n_quotes, n_verified,
unverified_quotes, verifier_method) with CHECK constraints. A
connect-time _migrate_audit_mode() ALTERs legacy DBs idempotently.
Cache hits return the persisted audit_mode rather than asserting
STRICT unconditionally.
CLI:
emergent list VISUAL/HYBRID records; --aggregate ranks
unverified quotes by frequency (corpus-growth signal).
reclassify re-run the verifier against live providence records;
cold-source records skipped; --dry-run reports
transitions without writing; each change writes one
'providence_reclassify' audit event.
