bench: 5R battery — closes ticket #000021 (15-sub-battery suite complete)
Phase 2 of #000021. React/Rearrange/Restore/Replicate/Resonate over the workspace surface — selfmodel_records (#000014) + memory_records (#000017), both landed earlier today. Closes the gap that gated 5R since the substrate work shipped. Sub-battery semantics (per SQD whitepaper §9.3 + ticket #000021 §4.2): - React: incorporate new fact/constraint. Workspace = (snapshot_t0, snapshot_t1, expected_delta). Pass = added_facts present + removed_facts absent in t+1. - Rearrange: restructure without semantic shift. Re-canonicalize different surface forms through a named π*; pass = bytes match expected_equivalent flag. Tests the order-invariance contracts in SelfModel (capability_claim_hashes sorted) and Memory (branches sorted by branch_id). - Restore: retrieve prior fact. Workspace = (history[], current_facts[]). Pass = fact in current OR any historical snapshot. - Replicate: independent canonical encodings via π*. Same input run N times must yield byte-equal output. Tests determinism contract. - Resonate: variance across N runs. Deterministic π*'s yield distinct=1; expected_max_distinct=1 enforces zero-variance contract. Surface: - bench/batteries/b_5r.py (5 deterministic runners; no LLM-as-judge) - bench/fixtures/5r/{react,rearrange,restore,replicate,resonate}-v1.jsonl (30 each = 150 new fixtures) - runner.py registers 5r in _BATTERIES + _DEFAULT_FIXTURES - Makefile: bench-5r + bench-suite (5S+5T+5F+5R aggregate) Final tally: 5S syntax/semantics/syllogism/synthesis/semiotics 108 5T transfer/transfer-learning/triangulation/... 154 5F function/finetuning/falsification/... 50 5R react/rearrange/restore/replicate/resonate 150 TOTAL: 462 fixtures across 21 sub-batteries — 100% pass. Tests: 6 new in tests/test_bench_batteries.py + adjustment to test_session_integration.py for the 312→462 count + 5R sub-battery presence assertion. Full suite: 1192 passed, 36 skipped. Closes #000021. Phase 3 (external-corpus expansion) remains open under the ticket but does not gate closure — the complete Dav1DPrometheus surface is now executable infrastructure.
This commit is contained in:
parent
6e42dbf026
commit
6e20c792c4
12 changed files with 635 additions and 13 deletions
|
|
@ -65,7 +65,7 @@ Newest first. Update on every open/close.
|
|||
| #000024 | 5T Phase 1b + Dav1DPrometheus vocabulary alignment | closed · landed 2026-05-08 | 2026-05-07 | — |
|
||||
| #000023 | 5S Phase 1b: Syllogism · Synthesis · Semiotics | closed · landed 2026-05-08 | 2026-05-07 | — |
|
||||
| #000022 | Adapter LossReport (PRD I9 analogue) | closed · landed 2026-05-07 | 2026-05-07 | — |
|
||||
| #000021 | 5S/5T/5R benchmark fixtures + harness | in progress · Phase 1a landed 2026-05-07 | 2026-05-07 | — |
|
||||
| #000021 | 5S/5T/5R benchmark fixtures + harness | closed · Phase 1a + 2 landed 2026-05-08 | 2026-05-07 | — |
|
||||
| #000020 | Capital-cost ledger (8-capital queues) | closed · landed 2026-05-07 | 2026-05-07 | — |
|
||||
| #000019 | Specification methodology for π* and V | closed · landed 2026-05-07 | 2026-05-07 | — |
|
||||
| #000018 | Adversarial soft-hash covert-channel analysis | open · awaiting go/no-go | 2026-05-07 | — |
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue