arborist/docs
russell@unturf.com 39bebe3fdb
qa(verify): Rule 9 — SUBJECT_TOKENS_ABSENT premise-parroting demote
Closes the first confirmed EVIDENCE-WARRANTED false-positive
surfaced by the 200-cycle bench-emergent run on
`steer/reply/correcter` (Ticket #000006 amend 2026-05-02b). The
model parroted three question-distinctive tokens (correcter,
steer, reply) into its claim while citing a glossary article whose
33.5K-char content contains ZERO occurrences of any of them.
Generic linguistic vocabulary (language, communication, terms,
relationships) carried Rule 5's citation-coverage check on its
own; the actual subject tokens rode along unverified.

New per-claim check `_parroted_subject_tokens_absent`: for each
resolving claim, compute the question∩claim content-token set,
then check substring presence in the union of cited evidence
spans (lower-cased, mirroring Rule 5). When ≥ threshold parroted
tokens are absent, emit `SUBJECT_TOKENS_ABSENT` and demote STRICT
→ HYBRID. Default threshold = 3 — single-token absence is often
stem-variant noise; three+ is the parrot fingerprint.

Plumbing:
- New default `DEFAULT_SUBJECT_TOKENS_ABSENT_THRESHOLD = 3`
- Both `verify_claim_lattice` and `verify_claim_lattice_json`
  gain `subject_tokens_absent_threshold` kwarg + per-claim check
  block (mirrors TITLE_MISMATCH plumbing, sits right after it
  in the rule order)
- `claim_lattice_subject_tokens_absent_threshold` policy field
  added to `DEFAULT_QUERY_POLICY` and `DEFAULT_POLICY`; folds
  into `governance_policy_hash` and (via _VERIFIER_POLICY_FIELDS)
  `verifier_policy_hash`
- All four runner/query call sites pass the policy-derived value

Live verification (cache-split cleanly via policy-hash bump):
  pre-fix cache_key 08dbd2c1… : STRICT (false positive)
  post-fix cache_key 6a519636… : UNGROUNDED

Three new unit tests in `tests/test_verify_json.py`:
- threshold-meeting parrot demotes STRICT → HYBRID
- no-op when question is None
- below-threshold absence stays STRICT

Full suite: 776 passed, 34 skipped.
2026-05-02 15:51:58 -04:00
..
diagrams docs: README label refresh + per-module reference + Graphviz diagrams 2026-05-01 23:19:01 -04:00
tickets qa(verify): Rule 9 — SUBJECT_TOKENS_ABSENT premise-parroting demote 2026-05-02 15:51:58 -04:00
bench-emergent-design.md bench: emergent stress-test — 3-word triangulation (blue-moon cadence) 2026-05-02 12:42:56 -04:00
bench-maxing.md docs(bench): rename qa-modes-bench-2026-05-02.md → qa-modes-bench.md 2026-05-02 12:27:38 -04:00
concept-relations-design.md make: speed up tests + automate concept backfill (test 38s→11s) 2026-05-02 10:11:16 -04:00
cti-architecture.md docs: tickets/ subdir + delete naming-deferral, ref-frame, coverage-audit 2026-05-02 12:51:09 -04:00
mesh-deploy.md docs: mesh-deploy runbook for two-host gossip setup 2026-04-28 17:24:37 -04:00
mesh.md docs: mesh multiplayer protocol with dot diagrams 2026-04-28 12:54:36 -04:00
modules.md docs: consolidate docs/modules/* → docs/modules.md (single file) 2026-05-02 13:23:43 -04:00
qa-modes-bench-2026-04-30.md docs(qa): final stop-sequence bench row, day-journey net summary 2026-04-30 16:32:41 -04:00
qa-modes-bench.md docs(bench): rename qa-modes-bench-2026-05-02.md → qa-modes-bench.md 2026-05-02 12:27:38 -04:00
self-reference-design.md docs: consolidate self-reference design — flat MVP + fact-Core distillation 2026-05-02 12:39:42 -04:00
seven-point-program.md qa(frame): #000002 land — reference-frame polarity contract (D3 → ✓) 2026-05-01 20:18:13 -04:00
TICKETS.md ticket(#000006): move into tickets/ subdir + 134-cycle rolling amend 2026-05-02 14:42:42 -04:00
verifier-semantic-gap-design.md docs(qa): design proposal for verifier semantic-gap NLI sidecar 2026-04-30 19:50:05 -04:00