engineering/arborist
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

#000025 Phase 1e: 5F falsification motif coverage (§10.12 closed)

Browse source 
Pre-Phase-1e: falsification-v1.jsonl covered 10 motif tags across 50
fixtures (the high-traffic warrant/title/anchor/format set). Phase
1e adds 12 fixtures (5f-fal-051..062) for the previously-uncovered
motifs from the verifier+soft-demote registries:

  CITATION_MISMATCH            DEFLECTION_DETECTED
  MANUAL_QUOTE_VIOLATION       SCHEMA_INVALID
  SOURCE_ROLE_BLOCKED          SUBJECT_TOKENS_ABSENT
  TOO_MANY_EVIDENCE_IDS        UNKNOWN_EVIDENCE_ID
  BROAD_QUANTIFIER_RUNAWAY     BROAD_QUANTIFIER_CAP_APPLIED
  BROAD_QUANTIFIER_SCOPE_UNBOUND  BROAD_QUANTIFIER_REJECTED

Coverage now: 22 unique motif tags across 62 fixtures.

Harness changes:
- test_bench_batteries.py: bump pass_count assertion 50 → 62 in both
  falsification tests; add test_5f_falsification_covers_every_documented_motif
  that pins the motif set against the verifier+soft-demote registries
  so adding a new violation upstream surfaces here as a missing
  fixture (loud signal, no silent drift).
- test_session_integration.py: bump full-suite total 662 → 674.

Closes #000025 §10.12 (every documented failure-motif tag).
Still open in Phase 1b: §10.11 (real shard finetuning chains),
§10.13 (Feedback Loop latency/efficiency against real workload),
§10.14 (threshold handoff to #000012).
This commit is contained in:
russell@unturf.com 2026-05-10 16:04:34 -04:00
parent 30a9488578
commit 883289d00e
No known key found for this signature in database
5 changed files with 100 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/TICKETS.md
View file

@ -92,7 +92,7 @@ Newest first. Update on every open/close.
| #000028 | Multi-modality witness for canonical shapes | closed · landed 2026-05-09 + follow-ups (capital ledger · sample rate) | 2026-05-08 | — |
| #000027 | Canonical projections persist to providence_cache | closed · landed 2026-05-09 | 2026-05-08 | — |
| #000026 | Real-shard workload baseline + search latency | closed · Phase 1 + 2 + 3 landed 2026-05-09 (Phase 3 in `60b5748`) | 2026-05-08 | — |
| #000025 | 5F battery (Function · Finetuning · Falsification · Formulate · Feedback Loop) | in progress · Phase 1a + 1b.2 + 1c + 1d landed 2026-05-09 | 2026-05-07 | — |
| #000025 | 5F battery (Function · Finetuning · Falsification · Formulate · Feedback Loop) | in progress · Phase 1a + 1b.2 + 1c + 1d landed 2026-05-09; Phase 1e (motif coverage, §10.12 closed) landed 2026-05-10 | 2026-05-07 | — |
| #000024 | 5T Phase 1b + Dav1DPrometheus vocabulary alignment | closed · landed 2026-05-08 | 2026-05-07 | — |
| #000023 | 5S Phase 1b: Syllogism · Synthesis · Semiotics | closed · landed 2026-05-08 | 2026-05-07 | — |
| #000022 | Adapter LossReport (PRD I9 analogue) | closed · landed 2026-05-07 | 2026-05-07 | — |

32
docs/tickets/ticket-000025-5f-battery.md
View file

@ -1,6 +1,6 @@
# Ticket #000025 — 5F battery: Function · Finetuning · Falsification · Formulate · Feedback Loop
**Status:** in progress · Phase 1a landed 2026-05-08; Phase 1b open
**Status:** in progress · Phase 1a + 1b.2 + 1c + 1d + 1e (motif coverage, 2026-05-10) landed; §10.11 (real shard finetuning chains), §10.13 (Feedback Loop latency/efficiency calibrated to real workload), §10.14 (threshold handoff to #000012) still open
**Opened:** 2026-05-07
**Revised:** 2026-05-08 (cross-modality + state-space synthesis
folded in per review response)
@ -544,6 +544,36 @@ limit.
`make bench-5f-live` aggregates all five live fixtures.
### Phase 1e — motif coverage (2026-05-10)
§10.12 closure: every documented failure-motif tag has at least one
falsification fixture as `expected_reason`. Pre-Phase-1e the
falsification-v1.jsonl pack covered 10 motifs (the high-traffic
warrant/title/anchor/format set); Phase 1e adds 12 fixtures
(`5f-fal-051..062`) for the previously-uncovered motifs surfaced by
auditing `arborist/qa/verify.py:violations.append` callsites + the
`_SOFT_DEMOTE_VIOLATION_KINDS` registry in `arborist/cli.py`:
```
CITATION_MISMATCH DEFLECTION_DETECTED
MANUAL_QUOTE_VIOLATION SCHEMA_INVALID
SOURCE_ROLE_BLOCKED SUBJECT_TOKENS_ABSENT
TOO_MANY_EVIDENCE_IDS UNKNOWN_EVIDENCE_ID
BROAD_QUANTIFIER_RUNAWAY BROAD_QUANTIFIER_CAP_APPLIED
BROAD_QUANTIFIER_SCOPE_UNBOUND BROAD_QUANTIFIER_REJECTED
```
Pack now covers 22 unique motif tags across 62 fixtures (was
10/50). `test_5f_falsification_covers_every_documented_motif` in
`tests/test_bench_batteries.py` pins the motif set against the
verifier + soft-demote registries so adding a new violation
upstream surfaces here as a missing fixture (loud signal, no silent
drift).
Phase 1e closes §10.12. Remaining open: §10.11 (real shard
finetuning chains), §10.13 (Feedback Loop latency/efficiency
against real workload), §10.14 (threshold handoff to #000012).
---
## 11. Status