russell@unturf.com
883289d00e
Pre-Phase-1e: falsification-v1.jsonl covered 10 motif tags across 50 fixtures (the high-traffic warrant/title/anchor/format set). Phase 1e adds 12 fixtures (5f-fal-051..062) for the previously-uncovered motifs from the verifier+soft-demote registries: CITATION_MISMATCH DEFLECTION_DETECTED MANUAL_QUOTE_VIOLATION SCHEMA_INVALID SOURCE_ROLE_BLOCKED SUBJECT_TOKENS_ABSENT TOO_MANY_EVIDENCE_IDS UNKNOWN_EVIDENCE_ID BROAD_QUANTIFIER_RUNAWAY BROAD_QUANTIFIER_CAP_APPLIED BROAD_QUANTIFIER_SCOPE_UNBOUND BROAD_QUANTIFIER_REJECTED Coverage now: 22 unique motif tags across 62 fixtures. Harness changes: - test_bench_batteries.py: bump pass_count assertion 50 → 62 in both falsification tests; add test_5f_falsification_covers_every_documented_motif that pins the motif set against the verifier+soft-demote registries so adding a new violation upstream surfaces here as a missing fixture (loud signal, no silent drift). - test_session_integration.py: bump full-suite total 662 → 674. Closes #000025 §10.12 (every documented failure-motif tag). Still open in Phase 1b: §10.11 (real shard finetuning chains), §10.13 (Feedback Loop latency/efficiency against real workload), §10.14 (threshold handoff to #000012). |
||
|---|---|---|
| .. | ||
| batteries | ||
| fixtures | ||
| results | ||
| scripts | ||
| emergent_log.jsonl | ||
| prometheus_sigma_trigger_probe.py | ||
| qa_questions.txt | ||
| qa_questions_canonical_witness_npower.txt | ||
| qa_questions_metacog_subset.txt | ||
| qa_questions_progressive_and.txt | ||
| qa_questions_quantifier_baseline.txt | ||
| qa_questions_quantifier_subset.txt | ||
| qa_questions_smoke.txt | ||
| qa_questions_warrant_chain_aggressive.txt | ||
| qa_questions_warrant_chain_paraphrase.txt | ||
| qa_questions_warrant_chain_probe.txt | ||
| qa_sweep.py | ||
| run.sh | ||