arborist/docs/tickets
russell@unturf.com 39bebe3fdb
qa(verify): Rule 9 — SUBJECT_TOKENS_ABSENT premise-parroting demote
Closes the first confirmed EVIDENCE-WARRANTED false-positive
surfaced by the 200-cycle bench-emergent run on
`steer/reply/correcter` (Ticket #000006 amend 2026-05-02b). The
model parroted three question-distinctive tokens (correcter,
steer, reply) into its claim while citing a glossary article whose
33.5K-char content contains ZERO occurrences of any of them.
Generic linguistic vocabulary (language, communication, terms,
relationships) carried Rule 5's citation-coverage check on its
own; the actual subject tokens rode along unverified.

New per-claim check `_parroted_subject_tokens_absent`: for each
resolving claim, compute the question∩claim content-token set,
then check substring presence in the union of cited evidence
spans (lower-cased, mirroring Rule 5). When ≥ threshold parroted
tokens are absent, emit `SUBJECT_TOKENS_ABSENT` and demote STRICT
→ HYBRID. Default threshold = 3 — single-token absence is often
stem-variant noise; three+ is the parrot fingerprint.

Plumbing:
- New default `DEFAULT_SUBJECT_TOKENS_ABSENT_THRESHOLD = 3`
- Both `verify_claim_lattice` and `verify_claim_lattice_json`
  gain `subject_tokens_absent_threshold` kwarg + per-claim check
  block (mirrors TITLE_MISMATCH plumbing, sits right after it
  in the rule order)
- `claim_lattice_subject_tokens_absent_threshold` policy field
  added to `DEFAULT_QUERY_POLICY` and `DEFAULT_POLICY`; folds
  into `governance_policy_hash` and (via _VERIFIER_POLICY_FIELDS)
  `verifier_policy_hash`
- All four runner/query call sites pass the policy-derived value

Live verification (cache-split cleanly via policy-hash bump):
  pre-fix cache_key 08dbd2c1… : STRICT (false positive)
  post-fix cache_key 6a519636… : UNGROUNDED

Three new unit tests in `tests/test_verify_json.py`:
- threshold-meeting parrot demotes STRICT → HYBRID
- no-op when question is None
- below-threshold absence stays STRICT

Full suite: 776 passed, 34 skipped.
2026-05-02 15:51:58 -04:00
..
ticket-000001-retrieval-keywords-audit-gap.md docs: tickets/ subdir + delete naming-deferral, ref-frame, coverage-audit 2026-05-02 12:51:09 -04:00
ticket-000002-reference-frame-polarity-contract.md docs: tickets/ subdir + delete naming-deferral, ref-frame, coverage-audit 2026-05-02 12:51:09 -04:00
ticket-000003-anchor-class-warrant.md docs: tickets/ subdir + delete naming-deferral, ref-frame, coverage-audit 2026-05-02 12:51:09 -04:00
ticket-000004-directive-coverage-in-bench.md docs: tickets/ subdir + delete naming-deferral, ref-frame, coverage-audit 2026-05-02 12:51:09 -04:00
ticket-000005-label-ladder-migration.md docs: tickets/ subdir + delete naming-deferral, ref-frame, coverage-audit 2026-05-02 12:51:09 -04:00
ticket-000006-bench-emergent-findings.md qa(verify): Rule 9 — SUBJECT_TOKENS_ABSENT premise-parroting demote 2026-05-02 15:51:58 -04:00
ticket-000007-query-layer-hyphen-fold.md qa(query): #000007 land — query-layer hyphen folding 2026-05-02 14:12:46 -04:00