verify: strip possessive apostrophes before title-overlap stemming

_claim_title_overlap's _stem only stripped trailing 's' on words >4
chars. 'homer's' (mid-apostrophe possessive) became 'homer'' after
stem, which didn't match 'homer' from the title — both claims of a
Simpsons answer cited articles that contained the right content but
the title-overlap check failed, demoting audit_mode UNGROUNDED via
the existing 'all resolving claims TITLE_MISMATCH' rule.

Fix: normalize apostrophes (ASCII and Unicode right-single-quote)
to empty before length/suffix check. Localized to _claim_title_overlap's
inline _stem — no other callers of _content_tokens affected.

Repro / regression:
  before: _claim_title_overlap("Homer's boss", "Dancin' Homer") → False
  after : _claim_title_overlap("Homer's boss", "Dancin' Homer") → True
  unrelated still False; plural collapse still True; 115 verifier tests pass.

Live demo (cloud-ask 'who is homer simpsons boss?'):
  before: UNGROUNDED 2/2  (false rejection of two correct claims)
  after : POINTER-LINKED-PARTIAL 2/2  (Scorpio claim still title-
          mismatched against 'You Only Move Twice' — the episode-
          naming convention legitimately ducks token overlap there;
          that demote is honest)
This commit is contained in:
russell@unturf.com 2026-05-30 17:00:24 -04:00
parent 0e7f599279
commit e0837da9f4
No known key found for this signature in database

View file

@ -1168,7 +1168,21 @@ def _claim_title_overlap(claim_text: str, source_title: str | None) -> bool:
# collapse the same way ('movies' vs 'movie', 'simpsons' vs
# 'simpson'). Defined in qa/query.py to avoid an import cycle:
# inline a minimal copy here instead.
#
# 2026-05-30: also normalize possessive apostrophes that the
# tokenizer leaves in mid-word ("homer's" → "homers" before stem).
# Without this, claim "Homer's boss" cited to title "Dancin' Homer"
# tripped TITLE_MISMATCH because "homer's" stemmed to "homer'"
# which didn't match "homer" stemmed from the title. The bug
# cascades: when EVERY resolving claim trips TITLE_MISMATCH, the
# verifier demotes audit_mode → UNGROUNDED even though the
# citations are honest, producing the "UNGROUNDED 2/2" inversion.
def _stem(t: str) -> str:
# Strip apostrophes anywhere — possessives, Unicode quotes
# ("Dancin'" or "Dancin"). Done before length/suffix
# check so "homer's" (7 chars, mid apostrophe) becomes
# "homers" then stems to "homer".
t = t.replace("'", "").replace("", "")
if len(t) > 4 and t.endswith("s") and not t.endswith("ss"):
return t[:-1]
return t