qa/corpus_query: role-classified + role-weighted budget (step 6b)
When policy is provided (any non-None dict), the orchestrator now:
1. Classifies each hit's source_role via
arborist.qa.source_roles.classify_source_role(title, qtokens_stem,
document_uri). Same heuristic legacy query() uses — noisy/sequel/
secondary markers fire first, then breadth-of-title-stem-coverage
decides primary vs background.
2. Splits max_context_chars by SOURCE_ROLE_BUDGET_WEIGHTS
proportionally (primary 2.0, noisy/sequel 0.5, others 1.0).
Primary answer source claims ~2× the slice — same shape as
legacy query()'s per-source cap.
policy=None preserves the pre-step-6b shape EXACTLY: rank-based
roles (rank 1 = primary, else background), flat
`max_context_chars / len(hits)` budget. Byte-identity gate from
step 5 remains green.
Tested: policy={} on the Anarchism single-token fixture classifies
the top hit as primary_answer_source (matches legacy). Existing
259 tests + new role-class test = 262 in the gate.
This commit is contained in:
parent
72d111796f
commit
03f248cc82
2 changed files with 63 additions and 5 deletions
|
|
@ -188,6 +188,23 @@ def test_run_query_policy_forwards_verifier_kwarg(corpus):
|
|||
), f"expected TOO_MANY_CLAIMS violation in capped run; got {violations!r}"
|
||||
|
||||
|
||||
def test_run_query_policy_classifies_source_role(corpus):
|
||||
"""With policy provided, hit roles come from the title-based
|
||||
classifier (arborist.qa.source_roles.classify_source_role), not
|
||||
a flat 'rank 1 = primary, else = background' default. For the
|
||||
'anarchism' query and the Anarchism fixture title, the top hit
|
||||
must classify as primary_answer_source either way (matches
|
||||
the legacy behavior for single-token clean-title queries)."""
|
||||
result = run_query(
|
||||
corpus, "anarchism",
|
||||
StubClient(answer="Anarchism is a philosophy. [E1]"),
|
||||
model_id="stub", top_k=2, policy={},
|
||||
)
|
||||
src = result["sources"][0]
|
||||
# Classifier identifies clean Anarchism title as primary.
|
||||
assert src["source_role"] == "primary_answer_source"
|
||||
|
||||
|
||||
def test_run_query_policy_ignores_unknown_keys(corpus):
|
||||
"""policy with verifier-irrelevant keys (e.g. base_version that
|
||||
Phase 1 step 6a doesn't honor yet) must not blow up — unknown keys
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue