arborist/tests
russell@unturf.com a0a55c8871
qa: mechanical repair loop — closes the single-shot gap
The system was observational: verifier classified, sidecar diagnosed,
repair plans were emitted — but no loop ever closed. This adds the
hybrid repair stage from the toy-Hermes design pass: mechanical first
(deterministic string substitution from sidecar suggestions), behind
a `policy["repair_enabled"]` flag (off by default). Re-prompt fallback
is TODO.

aborist/qa/repair.py: `mechanical_repair(answer, unverified_quotes,
context)` walks each unverified quote through the sidecar classifier &
applies its `repair` action by string sub:

    synthetic_elision_inside_quote (both halves verbatim)
        `"prefix [...] suffix"` → `"prefix" ... "suffix"`
        Two verbatim spans the verifier can independently check; the
        model's [...] ellipsis-marker becomes prose between them.
    trailing_artifact
        `"prose. (Source: ...)"` → `"prose."`
        Verbatim prefix kept; model-appended tail dropped.
    no_overlap
        Drop the line containing the bad quote entirely.

Skips include_aside_for_verbatim (needs precise source-span extraction;
defer to re-prompt path), paraphrase / partial_paraphrase (need prose
rewriting). Idempotent.

aborist/qa/query.py + aborist/qa/runner.py: optional pass after first
verify. When `repair_enabled=True` AND `audit_mode != "STRICT"` AND
unverified quotes exist:
    1. Run mechanical_repair on the answer text.
    2. If repair produced any changes, re-verify the repaired text.
    3. If post-repair verdict isn't worse (n_verified didn't decrease),
       accept the repair: persist the REPAIRED answer text instead of
       the model's original. Cache_key inputs unchanged.
    4. Audit chain gets one `providence_repair` event with the change
       log + pre/post verdict so the original→repaired transition is
       reconstructable.

Result dict gains `repair_changes` (list of change records) and
`pre_repair_audit_mode` (what the original was classified as).
`policy["repair_enabled"]` enters governance_policy_hash so on/off
agents share no cache silos.

Tests:
- mechanical_repair on each diagnosis (synthetic_elision, trailing_artifact,
  no_overlap), idempotence on clean text.
- query() integration: repair_enabled=False (default) leaves answer
  text unchanged; repair_enabled=True promotes a HYBRID/quote
  synthetic_elision case to STRICT/quote, persists the repaired text,
  emits the providence_repair audit event.

479 tests pass (+6 repair).
2026-04-29 19:14:06 -04:00
..
crawler crawler: skip feeds + sitemaps at ingest (they're discovery, not knowledge) 2026-04-29 05:55:28 -04:00
fixtures verify: strip wikitext from context before substring matching 2026-04-28 15:48:07 -04:00
__init__.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_burn.py cli: 'aborist burn-kindergarten' — mass-burn fresh providence rows 2026-04-29 16:56:59 -04:00
test_burn_doc.py cli: extend 'aborist burn' to documents and cores 2026-04-28 17:32:40 -04:00
test_cli_render.py cli: human render for query by default; --json for raw; ensure_ascii=False 2026-04-29 10:14:36 -04:00
test_compress.py storage cheats + TF-IDF retrieval fix 2026-04-27 17:24:51 -04:00
test_concepts.py concept overlay: synonym expansion + rivalry exclusion 2026-04-27 12:34:00 -04:00
test_dag.py qa: per-run Merkle-DAG provenance — run_dag_root on every providence record 2026-04-29 18:44:17 -04:00
test_distill.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_distill_recursive.py recursive distillation: core -> depth+1 core 2026-04-27 07:54:30 -04:00
test_evict.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_grok_source.py add Grok export source: conversations + media posts 2026-04-27 13:49:42 -04:00
test_html_source.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_ingest.py verify: layered strategies + entity policies, rename VISUAL → UNGROUNDED 2026-04-28 16:58:31 -04:00
test_inspect.py qa: per-claim status taxonomy on verifier + repair-action plans on sidecar 2026-04-29 18:37:52 -04:00
test_keys.py qa: question_hash strips articles + verifier coverage drops stopwords 2026-04-29 14:00:44 -04:00
test_llm_context_base.py qa: pipe wikitext-base-v1 into LLM context (was verifier-only) 2026-04-28 17:08:49 -04:00
test_merkle.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_mesh.py mesh: cryptographic foundation, off by default 2026-04-27 19:00:24 -04:00
test_mesh_aead.py mesh: optional AEAD body encryption under per-epoch shared secret 2026-04-28 17:34:39 -04:00
test_mesh_chain.py mesh: per-peer audit chain-of-claims tracking on the wire 2026-04-28 19:41:25 -04:00
test_mesh_cli.py mesh: sync default-holds records younger than 1 hour (kindergarten window) 2026-04-29 16:07:24 -04:00
test_mesh_cli_pull.py mesh: 'pull' CLI verb — fetch document body from a peer 2026-04-28 17:33:27 -04:00
test_mesh_wire.py mesh: HTTP gossip wire — signed envelopes + 5 message types 2026-04-28 16:57:04 -04:00
test_mesh_wire_e2e.py mesh: HTTP gossip wire — signed envelopes + 5 message types 2026-04-28 16:57:04 -04:00
test_migration_audit_mode.py verify: layered strategies + entity policies, rename VISUAL → UNGROUNDED 2026-04-28 16:58:31 -04:00
test_qa.py qa: post-LLM faithfulness verifier sets STRICT/HYBRID/VISUAL audit_mode 2026-04-28 15:45:08 -04:00
test_query.py qa: per-run Merkle-DAG provenance — run_dag_root on every providence record 2026-04-29 18:44:17 -04:00
test_reclassify.py verify: layered strategies + entity policies, rename VISUAL → UNGROUNDED 2026-04-28 16:58:31 -04:00
test_repair.py qa: mechanical repair loop — closes the single-shot gap 2026-04-29 19:14:06 -04:00
test_resume.py resumable ingest + per-shard audit chain integrity 2026-04-27 11:29:27 -04:00
test_snapshot.py add corpus-level snapshots: single-hash identity for the forest 2026-04-27 21:29:10 -04:00
test_tfidf.py add TF-IDF keyword distiller 2026-04-27 07:55:53 -04:00
test_vcs_source.py add git and mercurial repo sources for self-play 2026-04-27 18:17:39 -04:00
test_verify.py qa: per-claim status taxonomy on verifier + repair-action plans on sidecar 2026-04-29 18:37:52 -04:00
test_versioned_ingest.py storage cheats + TF-IDF retrieval fix 2026-04-27 17:24:51 -04:00
test_wikipedia_old.py prepare full Wikipedia 2003-05-16 ingest: cur + old (revisions) 2026-04-27 08:10:42 -04:00
test_wikipedia_xml.py add Phase IV Wikipedia XML + abstract sources 2026-04-27 17:24:28 -04:00
test_wikitext.py verify: layered strategies + entity policies, rename VISUAL → UNGROUNDED 2026-04-28 16:58:31 -04:00