bench: 5R battery — closes ticket #000021 (15-sub-battery suite complete)
Phase 2 of #000021. React/Rearrange/Restore/Replicate/Resonate over the workspace surface — selfmodel_records (#000014) + memory_records (#000017), both landed earlier today. Closes the gap that gated 5R since the substrate work shipped. Sub-battery semantics (per SQD whitepaper §9.3 + ticket #000021 §4.2): - React: incorporate new fact/constraint. Workspace = (snapshot_t0, snapshot_t1, expected_delta). Pass = added_facts present + removed_facts absent in t+1. - Rearrange: restructure without semantic shift. Re-canonicalize different surface forms through a named π*; pass = bytes match expected_equivalent flag. Tests the order-invariance contracts in SelfModel (capability_claim_hashes sorted) and Memory (branches sorted by branch_id). - Restore: retrieve prior fact. Workspace = (history[], current_facts[]). Pass = fact in current OR any historical snapshot. - Replicate: independent canonical encodings via π*. Same input run N times must yield byte-equal output. Tests determinism contract. - Resonate: variance across N runs. Deterministic π*'s yield distinct=1; expected_max_distinct=1 enforces zero-variance contract. Surface: - bench/batteries/b_5r.py (5 deterministic runners; no LLM-as-judge) - bench/fixtures/5r/{react,rearrange,restore,replicate,resonate}-v1.jsonl (30 each = 150 new fixtures) - runner.py registers 5r in _BATTERIES + _DEFAULT_FIXTURES - Makefile: bench-5r + bench-suite (5S+5T+5F+5R aggregate) Final tally: 5S syntax/semantics/syllogism/synthesis/semiotics 108 5T transfer/transfer-learning/triangulation/... 154 5F function/finetuning/falsification/... 50 5R react/rearrange/restore/replicate/resonate 150 TOTAL: 462 fixtures across 21 sub-batteries — 100% pass. Tests: 6 new in tests/test_bench_batteries.py + adjustment to test_session_integration.py for the 312→462 count + 5R sub-battery presence assertion. Full suite: 1192 passed, 36 skipped. Closes #000021. Phase 3 (external-corpus expansion) remains open under the ticket but does not gate closure — the complete Dav1DPrometheus surface is now executable infrastructure.
This commit is contained in:
parent
6e42dbf026
commit
6e20c792c4
12 changed files with 635 additions and 13 deletions
11
Makefile
11
Makefile
|
|
@ -266,9 +266,18 @@ bench-5f: bootstrap ## 5F battery (Function+Finetuning+Falsification+Formulate+F
|
|||
PYTHONUNBUFFERED=1 $(PY) -m bench.batteries.runner --battery 5f --sub formulate --fixtures bench/fixtures/5f/formulate-v1.jsonl
|
||||
PYTHONUNBUFFERED=1 $(PY) -m bench.batteries.runner --battery 5f --sub feedback-loop --fixtures bench/fixtures/5f/feedback-loop-v1.jsonl
|
||||
|
||||
bench-5r: bootstrap ## 5R battery (React+Rearrange+Restore+Replicate+Resonate)
|
||||
PYTHONUNBUFFERED=1 $(PY) -m bench.batteries.runner --battery 5r --sub react --fixtures bench/fixtures/5r/react-v1.jsonl
|
||||
PYTHONUNBUFFERED=1 $(PY) -m bench.batteries.runner --battery 5r --sub rearrange --fixtures bench/fixtures/5r/rearrange-v1.jsonl
|
||||
PYTHONUNBUFFERED=1 $(PY) -m bench.batteries.runner --battery 5r --sub restore --fixtures bench/fixtures/5r/restore-v1.jsonl
|
||||
PYTHONUNBUFFERED=1 $(PY) -m bench.batteries.runner --battery 5r --sub replicate --fixtures bench/fixtures/5r/replicate-v1.jsonl
|
||||
PYTHONUNBUFFERED=1 $(PY) -m bench.batteries.runner --battery 5r --sub resonate --fixtures bench/fixtures/5r/resonate-v1.jsonl
|
||||
|
||||
bench-5s5t: bench-5s bench-5t ## 5S + 5T (Phase-1b vocabulary)
|
||||
|
||||
bench-5s5t5f: bench-5s bench-5t bench-5f ## complete Dav1DPrometheus suite (5S + 5T + 5F)
|
||||
bench-5s5t5f: bench-5s bench-5t bench-5f ## 5S + 5T + 5F (operational triad)
|
||||
|
||||
bench-suite: bench-5s bench-5t bench-5f bench-5r ## complete Dav1DPrometheus suite (5S + 5T + 5F + 5R)
|
||||
|
||||
|
||||
verify-shards: bootstrap ## cross-shard Merkle round-trip on a random sample
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue