qa/corpus_query: multi-route retrieval when policy.multi_route=True (6c)
When the caller passes policy={"multi_route": True}, run_query now
fans out across four retrieval routes in parallel and merges:
1. fts_body — body BM25 (the only route in pre-6c)
2. fts_title — title-only BM25 via documents_fts
3. fts_phrase — verbatim 4-gram phrase MATCH; closes the
allusion gap ("always been at war" → 1984)
4. core_keyword_match — TF-IDF core route for neologisms
Each route is fail-open: NotSupportedError → []. core_keyword
returns [] on SidecarBucketCorpus (no derivations in slim sidecar);
phrase / title / body all work cloud-side via the slim FTS5 sidecar.
Merge by MIN bm25 per document_root (FTS5 returns negative; lower
wins). core_keyword's positive match_count scores are kept only as
a tiebreaker when no FTS5 route surfaced that doc — handled
explicitly via score-sign discrimination since the scales are
incomparable.
Lifted helper: question_phrases(question, n=4) from query.py's
_question_phrases into arborist.qa.retrieval_routes — pure-stdlib
n-token window extractor, no stopword stripping (the diagnostic
signal IS the stopword).
policy=None / policy={} (no multi_route flag) keep the existing
body-only path — byte-identity gate from step 5 still green. 263
existing tests + 1 new multi-route test pass.
Still missing for full legacy parity: filter_by_title_relevance
integration in run_query (the 5-accept-path filter is available in
retrieval_routes.py since step 3 but isn't wired into the
orchestrator yet). That's the next sub-step — without it, the
multi-route merge over-recalls on noisy title overlaps.
This commit is contained in:
parent
d3b78025ff
commit
5fdd573c0a
3 changed files with 170 additions and 10 deletions
|
|
@ -205,6 +205,25 @@ def test_run_query_policy_classifies_source_role(corpus):
|
|||
assert src["source_role"] == "primary_answer_source"
|
||||
|
||||
|
||||
def test_run_query_multi_route_merges_body_and_title(corpus):
|
||||
"""policy={"multi_route": True} fans body + title + phrase +
|
||||
core_keyword. core_keyword raises NotSupportedError on
|
||||
SqliteShardCorpus when the shard has no derivations — fail-open
|
||||
means the merge still produces hits from body and title.
|
||||
|
||||
For the Anarchism fixture (no derivations), this test just
|
||||
confirms multi-route doesn't blow up and the merged hit set is
|
||||
non-empty."""
|
||||
result = run_query(
|
||||
corpus, "anarchism",
|
||||
StubClient(answer="Anarchism is a philosophy. [E1]"),
|
||||
model_id="stub", top_k=2,
|
||||
policy={"multi_route": True},
|
||||
)
|
||||
assert result["sources"]
|
||||
assert any("Anarchism" in s["title"] for s in result["sources"])
|
||||
|
||||
|
||||
def test_run_query_policy_ignores_unknown_keys(corpus):
|
||||
"""policy with verifier-irrelevant keys (e.g. base_version that
|
||||
Phase 1 step 6a doesn't honor yet) must not blow up — unknown keys
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue