russell@unturf.com
a1dd330455
Emergent stress-test log surfaced two failure shapes the curated bench can't reach. Reviewer (claude-opus-4-7 + fox) appended teacher: blocks to four log entries; two of them named real defects in the demote chain. This commit ships those fixes plus the teacher-block annotations. Cases reviewed (2026-05-02 emergent_log.jsonl): ✅ Menkar/sterilizers/trifle — working as designed 🟡 swallowtail/upbraided/rockiest — metaphorical deflection 🔴 cashback/widescreens/within — warrant gap on inferential claims 🔴 comeliness/fetish/investitures — topic shift earned ANCHOR-WARRANTED unfairly Two code changes: 1) verify_claim_lattice + verify_claim_lattice_json: when EVERY resolving claim has TITLE_MISMATCH, demote audit_mode → UNGROUNDED. Previously HYBRID. Catches the cashback case ('widescreens offer cashback' cited to a generic Coupon article — n_verified=1 overclaimed when the citation was meaningless). 2) cli._ladder_rung_for_lattice: DEFLECTION_DETECTED moves from soft-demote (cap at ANCHOR-WARRANTED) to hard-demote (cap at POINTER-LINKED) alongside WARRANT_MISSING and TITLE_MISMATCH. Catches the comeliness case (model fully shifted topic to 'Rock & Chips'; pre-fix the verified Rock & Chips claims landed at ANCHOR-WARRANTED unfairly). Test updates: - tests/test_verify_json.py:test_verify_json_title_mismatch_demotes_to_ungrounded (renamed; expected outcome flipped HYBRID → UNGROUNDED) - 4 fixtures in tests/test_claim_lattice.py and 1 in tests/test_verify_json.py: claim text augmented with 'in the film' so the Jurassic-Park-titled fixture's title shares a stem with the claim. These tests were testing other paths (NO_EVIDENCE_POINTER, EVIDENCE_LINKED_PARTIAL, UNKNOWN_EVIDENCE_ID) and the title-mismatch was incidental to the fixture choice. Teacher blocks on the four log entries record the verdict + failure_class + recommendation + notes, with reviewer + date. The two ✅/🟡 entries have no recommended code change; the two 🔴 entries reference this commit as the fix. 751/34 tests pass. |
||
|---|---|---|
| .. | ||
| emergent_log.jsonl | ||
| qa_questions.txt | ||
| qa_questions_smoke.txt | ||
| qa_sweep.py | ||
| run.sh | ||