qa/corpus_query: multi-route retrieval when policy.multi_route=True (6c)

When the caller passes policy={"multi_route": True}, run_query now
fans out across four retrieval routes in parallel and merges:

  1. fts_body          — body BM25 (the only route in pre-6c)
  2. fts_title         — title-only BM25 via documents_fts
  3. fts_phrase        — verbatim 4-gram phrase MATCH; closes the
                         allusion gap ("always been at war" → 1984)
  4. core_keyword_match — TF-IDF core route for neologisms

Each route is fail-open: NotSupportedError → []. core_keyword
returns [] on SidecarBucketCorpus (no derivations in slim sidecar);
phrase / title / body all work cloud-side via the slim FTS5 sidecar.

Merge by MIN bm25 per document_root (FTS5 returns negative; lower
wins). core_keyword's positive match_count scores are kept only as
a tiebreaker when no FTS5 route surfaced that doc — handled
explicitly via score-sign discrimination since the scales are
incomparable.

Lifted helper: question_phrases(question, n=4) from query.py's
_question_phrases into arborist.qa.retrieval_routes — pure-stdlib
n-token window extractor, no stopword stripping (the diagnostic
signal IS the stopword).

policy=None / policy={} (no multi_route flag) keep the existing
body-only path — byte-identity gate from step 5 still green. 263
existing tests + 1 new multi-route test pass.

Still missing for full legacy parity: filter_by_title_relevance
integration in run_query (the 5-accept-path filter is available in
retrieval_routes.py since step 3 but isn't wired into the
orchestrator yet). That's the next sub-step — without it, the
multi-route merge over-recalls on noisy title overlaps.
This commit is contained in:
russell@unturf.com 2026-05-31 12:48:35 -04:00
parent d3b78025ff
commit 5fdd573c0a
No known key found for this signature in database
3 changed files with 170 additions and 10 deletions

View file

@ -205,6 +205,25 @@ def test_run_query_policy_classifies_source_role(corpus):
assert src["source_role"] == "primary_answer_source"
def test_run_query_multi_route_merges_body_and_title(corpus):
"""policy={"multi_route": True} fans body + title + phrase +
core_keyword. core_keyword raises NotSupportedError on
SqliteShardCorpus when the shard has no derivations fail-open
means the merge still produces hits from body and title.
For the Anarchism fixture (no derivations), this test just
confirms multi-route doesn't blow up and the merged hit set is
non-empty."""
result = run_query(
corpus, "anarchism",
StubClient(answer="Anarchism is a philosophy. [E1]"),
model_id="stub", top_k=2,
policy={"multi_route": True},
)
assert result["sources"]
assert any("Anarchism" in s["title"] for s in result["sources"])
def test_run_query_policy_ignores_unknown_keys(corpus):
"""policy with verifier-irrelevant keys (e.g. base_version that
Phase 1 step 6a doesn't honor yet) must not blow up — unknown keys