arborist/tests
russell@unturf.com 3a9aa729c5
qa: per-source context cap so a huge top-ranked doc doesn't monopolize
Fox 2026-04-29: querying "who is batman" returned only ONE source
(List_of_Batman_comics — an 80 KB+ bibliography) despite top_k=8 &
the actual bio article being in the corpus. Greedy fill: hit #1
consumed the entire 60 KB budget, every subsequent doc dropped with
char_budget <= 0.

Fix in aborist/qa/query.py:
  per_source_cap = max(1, max_context_chars // max(1, top_k))
  for h in hits[:top_k]:
      text = _load_doc_text(...)
      if len(text) > per_source_cap:
          text = text[:per_source_cap]   # NEW: per-source cap first
      if len(text) > char_budget:
          text = text[:char_budget]
      ...

Each top_k hit gets at most max_context_chars/top_k chars (default
60K/8 = 7.5K each — plenty for a chunk or two of prose). Total
context ≤ max_context_chars by construction. top_k=1 preserves the
legacy behavior (single source can use the full budget).

End-to-end effect on Batman: the bio (Wikipedia/Batman article) lands
in context alongside List_of_Batman_comics; the model can paraphrase-
verify against the actual character introduction text instead of
fabricating from training.

Tests: 2 regressions in tests/test_query.py — multi-source delivery
when hit #1 is huge, and top_k=1 single-source still allowed full
budget. 337 passed, 1 skipped.

Burned the two stale Batman cache records (chain extended) so a
fresh `make query Q="who is batman?"` exercises the new path.
2026-04-29 14:11:12 -04:00
..
crawler crawler: skip feeds + sitemaps at ingest (they're discovery, not knowledge) 2026-04-29 05:55:28 -04:00
fixtures verify: strip wikitext from context before substring matching 2026-04-28 15:48:07 -04:00
__init__.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_burn.py cli: 'aborist burn' — kindergarten leaf delete with audit event 2026-04-28 17:03:30 -04:00
test_burn_doc.py cli: extend 'aborist burn' to documents and cores 2026-04-28 17:32:40 -04:00
test_cli_render.py cli: human render for query by default; --json for raw; ensure_ascii=False 2026-04-29 10:14:36 -04:00
test_compress.py storage cheats + TF-IDF retrieval fix 2026-04-27 17:24:51 -04:00
test_concepts.py concept overlay: synonym expansion + rivalry exclusion 2026-04-27 12:34:00 -04:00
test_distill.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_distill_recursive.py recursive distillation: core -> depth+1 core 2026-04-27 07:54:30 -04:00
test_evict.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_grok_source.py add Grok export source: conversations + media posts 2026-04-27 13:49:42 -04:00
test_html_source.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_ingest.py verify: layered strategies + entity policies, rename VISUAL → UNGROUNDED 2026-04-28 16:58:31 -04:00
test_inspect.py qa: 'aborist inspect' — sidecar diagnose for unverified spans 2026-04-29 10:29:06 -04:00
test_keys.py qa: question_hash strips articles + verifier coverage drops stopwords 2026-04-29 14:00:44 -04:00
test_llm_context_base.py qa: pipe wikitext-base-v1 into LLM context (was verifier-only) 2026-04-28 17:08:49 -04:00
test_merkle.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_mesh.py mesh: cryptographic foundation, off by default 2026-04-27 19:00:24 -04:00
test_mesh_aead.py mesh: optional AEAD body encryption under per-epoch shared secret 2026-04-28 17:34:39 -04:00
test_mesh_chain.py mesh: per-peer audit chain-of-claims tracking on the wire 2026-04-28 19:41:25 -04:00
test_mesh_cli.py mesh: 'serve' and 'sync' CLI verbs over the wire layer 2026-04-28 17:05:31 -04:00
test_mesh_cli_pull.py mesh: 'pull' CLI verb — fetch document body from a peer 2026-04-28 17:33:27 -04:00
test_mesh_wire.py mesh: HTTP gossip wire — signed envelopes + 5 message types 2026-04-28 16:57:04 -04:00
test_mesh_wire_e2e.py mesh: HTTP gossip wire — signed envelopes + 5 message types 2026-04-28 16:57:04 -04:00
test_migration_audit_mode.py verify: layered strategies + entity policies, rename VISUAL → UNGROUNDED 2026-04-28 16:58:31 -04:00
test_qa.py qa: post-LLM faithfulness verifier sets STRICT/HYBRID/VISUAL audit_mode 2026-04-28 15:45:08 -04:00
test_query.py qa: per-source context cap so a huge top-ranked doc doesn't monopolize 2026-04-29 14:11:12 -04:00
test_reclassify.py verify: layered strategies + entity policies, rename VISUAL → UNGROUNDED 2026-04-28 16:58:31 -04:00
test_resume.py resumable ingest + per-shard audit chain integrity 2026-04-27 11:29:27 -04:00
test_snapshot.py add corpus-level snapshots: single-hash identity for the forest 2026-04-27 21:29:10 -04:00
test_tfidf.py add TF-IDF keyword distiller 2026-04-27 07:55:53 -04:00
test_vcs_source.py add git and mercurial repo sources for self-play 2026-04-27 18:17:39 -04:00
test_verify.py qa: question_hash strips articles + verifier coverage drops stopwords 2026-04-29 14:00:44 -04:00
test_versioned_ingest.py storage cheats + TF-IDF retrieval fix 2026-04-27 17:24:51 -04:00
test_wikipedia_old.py prepare full Wikipedia 2003-05-16 ingest: cur + old (revisions) 2026-04-27 08:10:42 -04:00
test_wikipedia_xml.py add Phase IV Wikipedia XML + abstract sources 2026-04-27 17:24:28 -04:00
test_wikitext.py verify: layered strategies + entity policies, rename VISUAL → UNGROUNDED 2026-04-28 16:58:31 -04:00