qa: lock phrase-route non-regression tests + open ticket #000002 (Module L)

Two follow-ups to the phrase-pattern retrieval fix (commit 1b8677d)
covering items 6 and 10-11 of fox's 2026-05-01 architectural review:

(1) Non-regression tests for the phrase route:
  - test_phrase_route_skipped_when_question_shorter_than_min_n
    pins the structural false-positive guard: the n=5/n=6 minimum
    means a 4-token literal-geography query lacks enough tokens to
    trigger the route at all.
  - test_phrase_route_does_not_hijack_literal_geography_query
    end-to-end: a 4-token "oceania east asia geography" query on
    a synthetic 2-doc corpus surfaces only the geography-stub doc;
    the orwell-stub doc (whose body has the diagnostic 5-gram) is
    correctly NOT pulled in by the phrase route on a literal query.

(2) docs/ticket-000002-reference-frame-polarity-contract.md
    Captures fox's Module L proposal verbatim as Appendix A and
    extracts the implementation sketch into the standard ticket
    body (problem statement, abstraction, CTI interpretation, three
    pieces of code to write, test list, scope boundaries).

    The phrase route closed the RETRIEVAL side of reference-frame
    failure. Module L addresses the ANSWER side: today's substrate
    answers Orwell queries as "the text does not directly state..."
    when it should produce multi-frame answers distinguishing
    Party propaganda from fictional-actual continuity. Forecast
    cost ~3-4 hours; risk medium (prompt augmentation interaction
    with claim_lattice prompt).

    Module M = ticket #000001 (route provenance binding); not
    duplicated. Module N (FP guards) partially landed via the
    tests above; remaining tests folded into ticket #000002's
    test list. Module H (relation warrant lite) lacks scope
    detail; deferred without a ticket.

(3) docs/TICKETS.md updated: index gains #000002 row, Next ID
    bumped to 000003.
This commit is contained in:
russell@unturf.com 2026-05-01 14:03:39 -04:00
parent 1b8677d3d5
commit a99ac4388b
No known key found for this signature in database
3 changed files with 724 additions and 1 deletions

View file

@ -1070,6 +1070,68 @@ def test_phrase_match_surfaces_topical_doc(tmp_path):
assert "test://orwell-stub" in uris
def test_phrase_route_skipped_when_question_shorter_than_min_n():
"""False-positive guard: a 4-token geography question lacks enough
tokens to trigger the n=5/n=6 phrase route. Short conventional
queries route through body-BM25 + title-LIKE only phrase routing
is structurally biased toward longer allusion-shape questions."""
from aborist.qa.query import _question_phrases
# 4 tokens after extraction → empty 5-gram and 6-gram outputs.
assert _question_phrases("oceania east asia geography", n=5) == []
assert _question_phrases("oceania east asia geography", n=6) == []
# 5 tokens → exactly one 5-gram, zero 6-grams.
out_5 = _question_phrases("oceania population east asia trade", n=5)
assert len(out_5) == 1
assert out_5[0] == "oceania population east asia trade"
out_6 = _question_phrases("oceania population east asia trade", n=6)
assert out_6 == []
def test_phrase_route_does_not_hijack_literal_geography_query(tmp_path):
"""Critical false-positive guard. A literal geography query about
Oceania + East Asia must NOT pull in an Orwell-flavored stub doc
just because both contain geographic tokens. The phrase route
only fires for verbatim 5+ token sequences from the question;
a different geography question shouldn't accidentally invoke it."""
main_db = tmp_path / "corpus.db"
qa_db = tmp_path / "qa.db"
conn = connect(main_db)
try:
ingest_source(conn, FakeSource([
_doc(
"test://orwell-stub",
# Body has the diagnostic Orwell 5-gram, but the title
# is title-irrelevant to a geography query.
"The novel narrates that Oceania always been at war with "
"Eastasia though the alliances had previously rotated. " * 6
),
_doc(
"test://geography-stub",
"Geographic descriptions of regions called Oceania and East "
"Asia. Topics: trade, population, climate, demographics. " * 12,
),
]))
finally:
conn.close()
# Literal geography query — short, no Orwell phrase.
result = query(
question="oceania east asia geography",
qa_db=qa_db,
chat_client=StubClient(answer="A geography answer."),
model_id="m",
single_db=main_db,
top_k=5,
)
uris = [s["document_uri"] for s in result["sources"]]
# Geography stub should be present (literal query, literal source).
assert "test://geography-stub" in uris
# Orwell stub should NOT have been surfaced via phrase route on
# a literal-geography query — the phrase route only activates on
# verbatim 5+ token sequences from the question, and "oceania
# east asia geography" is too short to produce any.
def test_filter_keeps_phrase_match_root_with_no_title_overlap():
"""Direct unit test for accept-path 4: a hit whose title shares
zero content tokens with the question, but whose document_root is