arborist/tests
russell@unturf.com a1dd330455
qa: tighten TITLE_MISMATCH + DEFLECTION demotes from emergent-log review
Emergent stress-test log surfaced two failure shapes the curated
bench can't reach. Reviewer (claude-opus-4-7 + fox) appended
teacher: blocks to four log entries; two of them named real
defects in the demote chain. This commit ships those fixes plus
the teacher-block annotations.

Cases reviewed (2026-05-02 emergent_log.jsonl):

  ✅ Menkar/sterilizers/trifle           — working as designed
  🟡 swallowtail/upbraided/rockiest      — metaphorical deflection
  🔴 cashback/widescreens/within         — warrant gap on inferential claims
  🔴 comeliness/fetish/investitures      — topic shift earned ANCHOR-WARRANTED unfairly

Two code changes:

1) verify_claim_lattice + verify_claim_lattice_json: when EVERY
   resolving claim has TITLE_MISMATCH, demote audit_mode →
   UNGROUNDED. Previously HYBRID. Catches the cashback case
   ('widescreens offer cashback' cited to a generic Coupon
   article — n_verified=1 overclaimed when the citation was
   meaningless).

2) cli._ladder_rung_for_lattice: DEFLECTION_DETECTED moves from
   soft-demote (cap at ANCHOR-WARRANTED) to hard-demote (cap at
   POINTER-LINKED) alongside WARRANT_MISSING and TITLE_MISMATCH.
   Catches the comeliness case (model fully shifted topic to
   'Rock & Chips'; pre-fix the verified Rock & Chips claims
   landed at ANCHOR-WARRANTED unfairly).

Test updates:
  - tests/test_verify_json.py:test_verify_json_title_mismatch_demotes_to_ungrounded
    (renamed; expected outcome flipped HYBRID → UNGROUNDED)
  - 4 fixtures in tests/test_claim_lattice.py and 1 in
    tests/test_verify_json.py: claim text augmented with 'in the
    film' so the Jurassic-Park-titled fixture's title shares a
    stem with the claim. These tests were testing other paths
    (NO_EVIDENCE_POINTER, EVIDENCE_LINKED_PARTIAL, UNKNOWN_EVIDENCE_ID)
    and the title-mismatch was incidental to the fixture choice.

Teacher blocks on the four log entries record the verdict +
failure_class + recommendation + notes, with reviewer + date.
The two ✅/🟡 entries have no recommended code change; the two 🔴
entries reference this commit as the fix.

751/34 tests pass.
2026-05-02 13:33:48 -04:00
..
crawler crawler: skip feeds + sitemaps at ingest (they're discovery, not knowledge) 2026-04-29 05:55:28 -04:00
fixtures verify: strip wikitext from context before substring matching 2026-04-28 15:48:07 -04:00
__init__.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_bench_qa_sweep.py bench: log-scale buckets to 1M; per-mode peak-bucket recommendation 2026-05-01 21:55:06 -04:00
test_burn.py cli: 'aborist burn-kindergarten' — mass-burn fresh providence rows 2026-04-29 16:56:59 -04:00
test_burn_doc.py cli: extend 'aborist burn' to documents and cores 2026-04-28 17:32:40 -04:00
test_claim_lattice.py qa: tighten TITLE_MISMATCH + DEFLECTION demotes from emergent-log review 2026-05-02 13:33:48 -04:00
test_cli_render.py qa(frame): #000002 land — reference-frame polarity contract (D3 → ✓) 2026-05-01 20:18:13 -04:00
test_compress.py storage cheats + TF-IDF retrieval fix 2026-04-27 17:24:51 -04:00
test_concepts.py qa(concepts): corpus-derived concept_relations table replaces frozensets 2026-05-01 21:14:53 -04:00
test_dag.py qa: chain-segment failure localization + re-prompt repair tier 2026-04-29 19:27:33 -04:00
test_directives.py qa(provenance): #000001 land — retrieval_plan_hash binds D4 2026-05-01 19:26:01 -04:00
test_distill.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_distill_recursive.py recursive distillation: core -> depth+1 core 2026-04-27 07:54:30 -04:00
test_evict.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_frame.py qa(frame): #000002 land — reference-frame polarity contract (D3 → ✓) 2026-05-01 20:18:13 -04:00
test_grok_source.py add Grok export source: conversations + media posts 2026-04-27 13:49:42 -04:00
test_html_source.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_ingest.py verify: layered strategies + entity policies, rename VISUAL → UNGROUNDED 2026-04-28 16:58:31 -04:00
test_inspect.py qa/inspect: title-relevance sidecar — flag retrieval-driven hallucinations 2026-05-01 10:19:44 -04:00
test_keys.py arch: layer-cake docs + verifier_policy_hash + old-maps principle 2026-05-01 12:10:29 -04:00
test_llm_context_base.py qa: pipe wikitext-base-v1 into LLM context (was verifier-only) 2026-04-28 17:08:49 -04:00
test_merkle.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_mesh.py mesh: cryptographic foundation, off by default 2026-04-27 19:00:24 -04:00
test_mesh_aead.py mesh: optional AEAD body encryption under per-epoch shared secret 2026-04-28 17:34:39 -04:00
test_mesh_chain.py mesh: per-peer audit chain-of-claims tracking on the wire 2026-04-28 19:41:25 -04:00
test_mesh_cli.py mesh: sync default-holds records younger than 1 hour (kindergarten window) 2026-04-29 16:07:24 -04:00
test_mesh_cli_pull.py mesh: 'pull' CLI verb — fetch document body from a peer 2026-04-28 17:33:27 -04:00
test_mesh_wire.py mesh: HTTP gossip wire — signed envelopes + 5 message types 2026-04-28 16:57:04 -04:00
test_mesh_wire_e2e.py mesh: HTTP gossip wire — signed envelopes + 5 message types 2026-04-28 16:57:04 -04:00
test_migration_audit_mode.py verify: layered strategies + entity policies, rename VISUAL → UNGROUNDED 2026-04-28 16:58:31 -04:00
test_providence_source.py docs: consolidate self-reference design — flat MVP + fact-Core distillation 2026-05-02 12:39:42 -04:00
test_qa.py qa: post-LLM faithfulness verifier sets STRICT/HYBRID/VISUAL audit_mode 2026-04-28 15:45:08 -04:00
test_qa_quality_live.py qa(verifier): promote deflection sidecar to soft-demote violation 2026-05-01 23:44:32 -04:00
test_query.py qa(retrieval): fix _search_titles SQLite tree-depth overflow on long queries 2026-05-01 22:19:20 -04:00
test_reclassify.py verify: layered strategies + entity policies, rename VISUAL → UNGROUNDED 2026-04-28 16:58:31 -04:00
test_repair.py qa: chain-segment failure localization + re-prompt repair tier 2026-04-29 19:27:33 -04:00
test_resume.py resumable ingest + per-shard audit chain integrity 2026-04-27 11:29:27 -04:00
test_snapshot.py add corpus-level snapshots: single-hash identity for the forest 2026-04-27 21:29:10 -04:00
test_tfidf.py add TF-IDF keyword distiller 2026-04-27 07:55:53 -04:00
test_vcs_source.py add git and mercurial repo sources for self-play 2026-04-27 18:17:39 -04:00
test_verify.py qa: per-claim status taxonomy on verifier + repair-action plans on sidecar 2026-04-29 18:37:52 -04:00
test_verify_json.py qa: tighten TITLE_MISMATCH + DEFLECTION demotes from emergent-log review 2026-05-02 13:33:48 -04:00
test_versioned_ingest.py storage cheats + TF-IDF retrieval fix 2026-04-27 17:24:51 -04:00
test_warrant.py qa(warrant): #000003 land — anchor-class generalization (D6 → ✓) 2026-05-01 19:16:06 -04:00
test_wikipedia_old.py prepare full Wikipedia 2003-05-16 ingest: cur + old (revisions) 2026-04-27 08:10:42 -04:00
test_wikipedia_xml.py add Phase IV Wikipedia XML + abstract sources 2026-04-27 17:24:28 -04:00
test_wikitext.py verify: layered strategies + entity policies, rename VISUAL → UNGROUNDED 2026-04-28 16:58:31 -04:00