engineering/arborist
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
arborist/bench/fixtures/5f/falsification-v1.jsonl
russell@unturf.com 883289d00e
#000025 Phase 1e: 5F falsification motif coverage (§10.12 closed) 
Pre-Phase-1e: falsification-v1.jsonl covered 10 motif tags across 50
fixtures (the high-traffic warrant/title/anchor/format set). Phase
1e adds 12 fixtures (5f-fal-051..062) for the previously-uncovered
motifs from the verifier+soft-demote registries:

  CITATION_MISMATCH            DEFLECTION_DETECTED
  MANUAL_QUOTE_VIOLATION       SCHEMA_INVALID
  SOURCE_ROLE_BLOCKED          SUBJECT_TOKENS_ABSENT
  TOO_MANY_EVIDENCE_IDS        UNKNOWN_EVIDENCE_ID
  BROAD_QUANTIFIER_RUNAWAY     BROAD_QUANTIFIER_CAP_APPLIED
  BROAD_QUANTIFIER_SCOPE_UNBOUND  BROAD_QUANTIFIER_REJECTED

Coverage now: 22 unique motif tags across 62 fixtures.

Harness changes:
- test_bench_batteries.py: bump pass_count assertion 50 → 62 in both
  falsification tests; add test_5f_falsification_covers_every_documented_motif
  that pins the motif set against the verifier+soft-demote registries
  so adding a new violation upstream surfaces here as a missing
  fixture (loud signal, no silent drift).
- test_session_integration.py: bump full-suite total 662 → 674.

Closes #000025 §10.12 (every documented failure-motif tag).
Still open in Phase 1b: §10.11 (real shard finetuning chains),
§10.13 (Feedback Loop latency/efficiency against real workload),
§10.14 (threshold handoff to #000012).
2026-05-10 16:04:34 -04:00

63 lines
22 KiB
JSON
Raw Blame History

{"_meta": {"battery": "5f", "sub_battery": "falsification", "version": "v1", "task_count": 62, "notes": "Planted-bad-record fixtures with expected violation tags. Each fixture pinned to verifier_method_root so verifier-shape changes warn rather than false-fail. Phase 1c (2026-05-09): expanded 10→30. Phase 1d (2026-05-09): expanded 30→50. Phase 1e (2026-05-10): added 12 fixtures (5f-fal-051..062) covering the previously-uncovered motif tags from the verifier+soft-demote registries — CITATION_MISMATCH, DEFLECTION_DETECTED, MANUAL_QUOTE_VIOLATION, SCHEMA_INVALID, SOURCE_ROLE_BLOCKED, SUBJECT_TOKENS_ABSENT, TOO_MANY_EVIDENCE_IDS, UNKNOWN_EVIDENCE_ID, BROAD_QUANTIFIER_{RUNAWAY,CAP_APPLIED,SCOPE_UNBOUND,REJECTED}. Closes #000025 §10.12 (every documented failure-motif tag)."}}
{"id":"5f-fal-001","battery":"5f","sub_battery":"falsification","version":"v1","carrier":"providence_record","domain":"claim_lattice","pi_star_ref":"claim-lattice@v1","record":"planted_bad_001","observed_violations":["WARRANT_MISSING","TITLE_MISMATCH"],"expected_reason":"WARRANT_MISSING","verifier_method_root":"warrant-v1","expected":"pass"}
{"id":"5f-fal-002","battery":"5f","sub_battery":"falsification","version":"v1","carrier":"providence_record","domain":"claim_lattice","pi_star_ref":"claim-lattice@v1","record":"planted_bad_002","observed_violations":["TITLE_MISMATCH"],"expected_reason":"TITLE_MISMATCH","verifier_method_root":"warrant-v1","expected":"pass"}
{"id":"5f-fal-003","battery":"5f","sub_battery":"falsification","version":"v1","carrier":"providence_record","domain":"claim_lattice","pi_star_ref":"claim-lattice@v1","record":"planted_bad_003","observed_violations":["TOO_MANY_CLAIMS"],"expected_reason":"TOO_MANY_CLAIMS","verifier_method_root":"warrant-v1","expected":"pass"}
{"id":"5f-fal-004","battery":"5f","sub_battery":"falsification","version":"v1","carrier":"providence_record","domain":"claim_lattice","pi_star_ref":"claim-lattice@v1","record":"planted_bad_004","observed_violations":["BARE_NAME_CLAIM"],"expected_reason":"BARE_NAME_CLAIM","verifier_method_root":"warrant-v1","expected":"pass"}
{"id":"5f-fal-005","battery":"5f","sub_battery":"falsification","version":"v1","carrier":"providence_record","domain":"claim_lattice","pi_star_ref":"claim-lattice@v1","record":"planted_bad_005","observed_violations":["LAZY_ANCHOR_DEMOTED","TITLE_MISMATCH"],"expected_reason":"LAZY_ANCHOR_DEMOTED","verifier_method_root":"warrant-v1","expected":"pass"}
{"id":"5f-fal-006","battery":"5f","sub_battery":"falsification","version":"v1","carrier":"providence_record","domain":"claim_lattice","pi_star_ref":"claim-lattice@v1","record":"planted_bad_006","observed_violations":["POINTER_OVERFLOW_TRIMMED"],"expected_reason":"POINTER_OVERFLOW_TRIMMED","verifier_method_root":"warrant-v1","expected":"pass"}
{"id":"5f-fal-007","battery":"5f","sub_battery":"falsification","version":"v1","carrier":"providence_record","domain":"claim_lattice","pi_star_ref":"claim-lattice@v1","record":"planted_bad_007","observed_violations":[],"expected_reason":"WARRANT_MISSING","verifier_method_root":"warrant-v1","expected":"fail"}
{"id":"5f-fal-008","battery":"5f","sub_battery":"falsification","version":"v1","carrier":"providence_record","domain":"claim_lattice","pi_star_ref":"claim-lattice@v1","record":"planted_bad_008","observed_violations":["UNRELATED_TAG"],"expected_reason":"WARRANT_MISSING","verifier_method_root":"warrant-v1","expected":"fail"}
{"id":"5f-fal-009","battery":"5f","sub_battery":"falsification","version":"v1","carrier":"providence_record","domain":"claim_lattice","pi_star_ref":"claim-lattice@v1","record":"planted_bad_009","observed_violations":["FORMAT_COLLAPSED","WARRANT_MISSING"],"expected_reason":"FORMAT_COLLAPSED","verifier_method_root":"warrant-v1","expected":"pass"}
{"id":"5f-fal-010","battery":"5f","sub_battery":"falsification","version":"v1","carrier":"providence_record","domain":"claim_lattice","pi_star_ref":"claim-lattice@v1","record":"planted_bad_010","observed_violations":["NO_EVIDENCE_POINTER"],"expected_reason":"NO_EVIDENCE_POINTER","verifier_method_root":"warrant-v1","expected":"pass"}
{"id": "5f-fal-011", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_011", "observed_violations": ["WARRANT_MISSING"], "expected_reason": "WARRANT_MISSING", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-012", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_012", "observed_violations": ["TITLE_MISMATCH"], "expected_reason": "TITLE_MISMATCH", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-013", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_013", "observed_violations": ["TOO_MANY_CLAIMS", "WARRANT_MISSING"], "expected_reason": "TOO_MANY_CLAIMS", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-014", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_014", "observed_violations": ["BARE_NAME_CLAIM"], "expected_reason": "BARE_NAME_CLAIM", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-015", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_015", "observed_violations": ["FORMAT_COLLAPSED"], "expected_reason": "FORMAT_COLLAPSED", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-016", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_016", "observed_violations": ["NO_EVIDENCE_POINTER"], "expected_reason": "NO_EVIDENCE_POINTER", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-017", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_017", "observed_violations": ["LAZY_ANCHOR_DEMOTED"], "expected_reason": "LAZY_ANCHOR_DEMOTED", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-018", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_018", "observed_violations": ["POINTER_OVERFLOW_TRIMMED"], "expected_reason": "POINTER_OVERFLOW_TRIMMED", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-019", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_019", "observed_violations": ["WARRANT_MISSING", "TITLE_MISMATCH", "TOO_MANY_CLAIMS"], "expected_reason": "WARRANT_MISSING", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-020", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_020", "observed_violations": ["BARE_NAME_CLAIM", "TITLE_MISMATCH"], "expected_reason": "BARE_NAME_CLAIM", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-021", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_021", "observed_violations": ["UNGROUNDED"], "expected_reason": "UNGROUNDED", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-022", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_022", "observed_violations": ["HYBRID_PARAPHRASE"], "expected_reason": "HYBRID_PARAPHRASE", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-023", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_023", "observed_violations": ["STRICT_QUOTE", "LAZY_ANCHOR_DEMOTED"], "expected_reason": "LAZY_ANCHOR_DEMOTED", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-024", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_024", "observed_violations": [], "expected_reason": "WARRANT_MISSING", "verifier_method_root": "warrant-v1", "expected": "fail"}
{"id": "5f-fal-025", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_025", "observed_violations": ["UNRELATED_TAG"], "expected_reason": "TITLE_MISMATCH", "verifier_method_root": "warrant-v1", "expected": "fail"}
{"id": "5f-fal-026", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_026", "observed_violations": ["WARRANT_MISSING"], "expected_reason": "TITLE_MISMATCH", "verifier_method_root": "warrant-v1", "expected": "fail"}
{"id": "5f-fal-027", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_027", "observed_violations": ["FORMAT_COLLAPSED"], "expected_reason": "WARRANT_MISSING", "verifier_method_root": "warrant-v1", "expected": "fail"}
{"id": "5f-fal-028", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_028", "observed_violations": ["TITLE_MISMATCH", "BARE_NAME_CLAIM"], "expected_reason": "TOO_MANY_CLAIMS", "verifier_method_root": "warrant-v1", "expected": "fail"}
{"id": "5f-fal-029", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_029", "observed_violations": [], "expected_reason": "BARE_NAME_CLAIM", "verifier_method_root": "warrant-v1", "expected": "fail"}
{"id": "5f-fal-030", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_030", "observed_violations": ["NO_EVIDENCE_POINTER"], "expected_reason": "FORMAT_COLLAPSED", "verifier_method_root": "warrant-v1", "expected": "fail"}
{"id": "5f-fal-031", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_031", "observed_violations": ["WARRANT_MISSING"], "expected_reason": "WARRANT_MISSING", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-032", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_032", "observed_violations": ["TITLE_MISMATCH"], "expected_reason": "TITLE_MISMATCH", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-033", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_033", "observed_violations": ["TOO_MANY_CLAIMS"], "expected_reason": "TOO_MANY_CLAIMS", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-034", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_034", "observed_violations": ["BARE_NAME_CLAIM"], "expected_reason": "BARE_NAME_CLAIM", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-035", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_035", "observed_violations": ["FORMAT_COLLAPSED"], "expected_reason": "FORMAT_COLLAPSED", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-036", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_036", "observed_violations": ["NO_EVIDENCE_POINTER"], "expected_reason": "NO_EVIDENCE_POINTER", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-037", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_037", "observed_violations": ["LAZY_ANCHOR_DEMOTED"], "expected_reason": "LAZY_ANCHOR_DEMOTED", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-038", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_038", "observed_violations": ["POINTER_OVERFLOW_TRIMMED"], "expected_reason": "POINTER_OVERFLOW_TRIMMED", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-039", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_039", "observed_violations": ["UNGROUNDED"], "expected_reason": "UNGROUNDED", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-040", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_040", "observed_violations": ["HYBRID_PARAPHRASE"], "expected_reason": "HYBRID_PARAPHRASE", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-041", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_041", "observed_violations": ["WARRANT_MISSING"], "expected_reason": "WARRANT_MISSING", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-042", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_042", "observed_violations": ["TITLE_MISMATCH"], "expected_reason": "TITLE_MISMATCH", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-043", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_043", "observed_violations": ["TOO_MANY_CLAIMS"], "expected_reason": "TOO_MANY_CLAIMS", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-044", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_044", "observed_violations": ["BARE_NAME_CLAIM"], "expected_reason": "BARE_NAME_CLAIM", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-045", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_045", "observed_violations": ["FORMAT_COLLAPSED"], "expected_reason": "FORMAT_COLLAPSED", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-046", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_046", "observed_violations": ["NO_EVIDENCE_POINTER"], "expected_reason": "NO_EVIDENCE_POINTER", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-047", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_047", "observed_violations": ["LAZY_ANCHOR_DEMOTED"], "expected_reason": "LAZY_ANCHOR_DEMOTED", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-048", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_048", "observed_violations": ["POINTER_OVERFLOW_TRIMMED"], "expected_reason": "POINTER_OVERFLOW_TRIMMED", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-049", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_049", "observed_violations": ["UNGROUNDED"], "expected_reason": "UNGROUNDED", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-050", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_050", "observed_violations": ["HYBRID_PARAPHRASE"], "expected_reason": "HYBRID_PARAPHRASE", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-051", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_051", "observed_violations": ["CITATION_MISMATCH"], "expected_reason": "CITATION_MISMATCH", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-052", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_052", "observed_violations": ["DEFLECTION_DETECTED"], "expected_reason": "DEFLECTION_DETECTED", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-053", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_053", "observed_violations": ["MANUAL_QUOTE_VIOLATION"], "expected_reason": "MANUAL_QUOTE_VIOLATION", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-054", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_054", "observed_violations": ["SCHEMA_INVALID"], "expected_reason": "SCHEMA_INVALID", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-055", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_055", "observed_violations": ["SOURCE_ROLE_BLOCKED"], "expected_reason": "SOURCE_ROLE_BLOCKED", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-056", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_056", "observed_violations": ["SUBJECT_TOKENS_ABSENT"], "expected_reason": "SUBJECT_TOKENS_ABSENT", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-057", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_057", "observed_violations": ["TOO_MANY_EVIDENCE_IDS"], "expected_reason": "TOO_MANY_EVIDENCE_IDS", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-058", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_058", "observed_violations": ["UNKNOWN_EVIDENCE_ID"], "expected_reason": "UNKNOWN_EVIDENCE_ID", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-059", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_059", "observed_violations": ["BROAD_QUANTIFIER_RUNAWAY"], "expected_reason": "BROAD_QUANTIFIER_RUNAWAY", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-060", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_060", "observed_violations": ["BROAD_QUANTIFIER_CAP_APPLIED"], "expected_reason": "BROAD_QUANTIFIER_CAP_APPLIED", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-061", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_061", "observed_violations": ["BROAD_QUANTIFIER_SCOPE_UNBOUND"], "expected_reason": "BROAD_QUANTIFIER_SCOPE_UNBOUND", "verifier_method_root": "warrant-v1", "expected": "pass"}
{"id": "5f-fal-062", "battery": "5f", "sub_battery": "falsification", "version": "v1", "carrier": "providence_record", "domain": "claim_lattice", "pi_star_ref": "claim-lattice@v1", "record": "planted_bad_062", "observed_violations": ["BROAD_QUANTIFIER_REJECTED"], "expected_reason": "BROAD_QUANTIFIER_REJECTED", "verifier_method_root": "warrant-v1", "expected": "pass"}
Reference in a new issue View git blame Copy permalink