russell@unturf.com
6e20c792c4
Phase 2 of #000021. React/Rearrange/Restore/Replicate/Resonate over the workspace surface — selfmodel_records (#000014) + memory_records (#000017), both landed earlier today. Closes the gap that gated 5R since the substrate work shipped. Sub-battery semantics (per SQD whitepaper §9.3 + ticket #000021 §4.2): - React: incorporate new fact/constraint. Workspace = (snapshot_t0, snapshot_t1, expected_delta). Pass = added_facts present + removed_facts absent in t+1. - Rearrange: restructure without semantic shift. Re-canonicalize different surface forms through a named π*; pass = bytes match expected_equivalent flag. Tests the order-invariance contracts in SelfModel (capability_claim_hashes sorted) and Memory (branches sorted by branch_id). - Restore: retrieve prior fact. Workspace = (history[], current_facts[]). Pass = fact in current OR any historical snapshot. - Replicate: independent canonical encodings via π*. Same input run N times must yield byte-equal output. Tests determinism contract. - Resonate: variance across N runs. Deterministic π*'s yield distinct=1; expected_max_distinct=1 enforces zero-variance contract. Surface: - bench/batteries/b_5r.py (5 deterministic runners; no LLM-as-judge) - bench/fixtures/5r/{react,rearrange,restore,replicate,resonate}-v1.jsonl (30 each = 150 new fixtures) - runner.py registers 5r in _BATTERIES + _DEFAULT_FIXTURES - Makefile: bench-5r + bench-suite (5S+5T+5F+5R aggregate) Final tally: 5S syntax/semantics/syllogism/synthesis/semiotics 108 5T transfer/transfer-learning/triangulation/... 154 5F function/finetuning/falsification/... 50 5R react/rearrange/restore/replicate/resonate 150 TOTAL: 462 fixtures across 21 sub-batteries — 100% pass. Tests: 6 new in tests/test_bench_batteries.py + adjustment to test_session_integration.py for the 312→462 count + 5R sub-battery presence assertion. Full suite: 1192 passed, 36 skipped. Closes #000021. Phase 3 (external-corpus expansion) remains open under the ticket but does not gate closure — the complete Dav1DPrometheus surface is now executable infrastructure.
31 lines
9.2 KiB
JSON
31 lines
9.2 KiB
JSON
{"_meta":{"battery":"5r","sub_battery":"react","version":"v1","task_count":30,"notes":"Workspace incorporates a new fact/observation between snapshots. expected_delta lists added/removed facts the t+1 snapshot must reflect."}}
|
|
{"id":"5r-react-001","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["A","B"]},"snapshot_t1":{"facts":["A","B","C"]},"expected_delta":{"added_facts":["C"],"removed_facts":[]}}
|
|
{"id":"5r-react-002","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["x","y","z"]},"snapshot_t1":{"facts":["x","y","z","new1","new2"]},"expected_delta":{"added_facts":["new1","new2"],"removed_facts":[]}}
|
|
{"id":"5r-react-003","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["fact-1","fact-2","fact-3"]},"snapshot_t1":{"facts":["fact-2","fact-3"]},"expected_delta":{"added_facts":[],"removed_facts":["fact-1"]}}
|
|
{"id":"5r-react-004","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["claim-a"]},"snapshot_t1":{"facts":["claim-b","claim-c"]},"expected_delta":{"added_facts":["claim-b","claim-c"],"removed_facts":["claim-a"]}}
|
|
{"id":"5r-react-005","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":[]},"snapshot_t1":{"facts":["genesis fact"]},"expected_delta":{"added_facts":["genesis fact"],"removed_facts":[]}}
|
|
{"id":"5r-react-006","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["initial"]},"snapshot_t1":{"facts":["initial"]},"expected_delta":{"added_facts":[],"removed_facts":[]}}
|
|
{"id":"5r-react-007","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["alpha","beta"]},"snapshot_t1":{"facts":["alpha","beta","gamma","delta","epsilon"]},"expected_delta":{"added_facts":["gamma","delta","epsilon"],"removed_facts":[]}}
|
|
{"id":"5r-react-008","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["status=active"]},"snapshot_t1":{"facts":["status=archived"]},"expected_delta":{"added_facts":["status=archived"],"removed_facts":["status=active"]}}
|
|
{"id":"5r-react-009","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["price=10","stock=100"]},"snapshot_t1":{"facts":["price=12","stock=95"]},"expected_delta":{"added_facts":["price=12","stock=95"],"removed_facts":["price=10","stock=100"]}}
|
|
{"id":"5r-react-010","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["entry-1"]},"snapshot_t1":{"facts":["entry-1","entry-2"]},"expected_delta":{"added_facts":["entry-2"],"removed_facts":[]}}
|
|
{"id":"5r-react-011","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["red","green","blue"]},"snapshot_t1":{"facts":["red","green","blue","alpha"]},"expected_delta":{"added_facts":["alpha"],"removed_facts":[]}}
|
|
{"id":"5r-react-012","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["one","two","three","four"]},"snapshot_t1":{"facts":["one","two"]},"expected_delta":{"added_facts":[],"removed_facts":["three","four"]}}
|
|
{"id":"5r-react-013","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["motif-A"]},"snapshot_t1":{"facts":["motif-A","motif-B","motif-C"]},"expected_delta":{"added_facts":["motif-B","motif-C"],"removed_facts":[]}}
|
|
{"id":"5r-react-014","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["claim-x is true"]},"snapshot_t1":{"facts":["claim-x is false"]},"expected_delta":{"added_facts":["claim-x is false"],"removed_facts":["claim-x is true"]}}
|
|
{"id":"5r-react-015","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["q1","q2"]},"snapshot_t1":{"facts":["q1","q2","q3"]},"expected_delta":{"added_facts":["q3"],"removed_facts":[]}}
|
|
{"id":"5r-react-016","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["k1=v1","k2=v2","k3=v3"]},"snapshot_t1":{"facts":["k1=v1","k2=v2"]},"expected_delta":{"added_facts":[],"removed_facts":["k3=v3"]}}
|
|
{"id":"5r-react-017","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["foo"]},"snapshot_t1":{"facts":["foo","bar","baz","qux"]},"expected_delta":{"added_facts":["bar","baz","qux"],"removed_facts":[]}}
|
|
{"id":"5r-react-018","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["state=active"]},"snapshot_t1":{"facts":["state=stale"]},"expected_delta":{"added_facts":["state=stale"],"removed_facts":["state=active"]}}
|
|
{"id":"5r-react-019","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["clean"]},"snapshot_t1":{"facts":["clean","new","fresh"]},"expected_delta":{"added_facts":["new","fresh"],"removed_facts":[]}}
|
|
{"id":"5r-react-020","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["a","b","c","d","e"]},"snapshot_t1":{"facts":["c","d","e"]},"expected_delta":{"added_facts":[],"removed_facts":["a","b"]}}
|
|
{"id":"5r-react-021","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["alpha"]},"snapshot_t1":{"facts":["alpha","beta"]},"expected":"fail","expected_delta":{"added_facts":["nonexistent"],"removed_facts":[]}}
|
|
{"id":"5r-react-022","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["x"]},"snapshot_t1":{"facts":["x","y"]},"expected":"fail","expected_delta":{"added_facts":[],"removed_facts":["x"]}}
|
|
{"id":"5r-react-023","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":[]},"snapshot_t1":{"facts":["e1","e2"]},"expected_delta":{"added_facts":["e1","e2"],"removed_facts":[]}}
|
|
{"id":"5r-react-024","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["root"]},"snapshot_t1":{"facts":["root","branch1","branch2"]},"expected_delta":{"added_facts":["branch1","branch2"],"removed_facts":[]}}
|
|
{"id":"5r-react-025","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["S1","S2","S3"]},"snapshot_t1":{"facts":["S1","S2","S3","S4","S5","S6"]},"expected_delta":{"added_facts":["S4","S5","S6"],"removed_facts":[]}}
|
|
{"id":"5r-react-026","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["U"]},"snapshot_t1":{"facts":["U","V"]},"expected_delta":{"added_facts":["V"],"removed_facts":[]}}
|
|
{"id":"5r-react-027","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["before"]},"snapshot_t1":{"facts":["after"]},"expected_delta":{"added_facts":["after"],"removed_facts":["before"]}}
|
|
{"id":"5r-react-028","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["item-A","item-B","item-C","item-D"]},"snapshot_t1":{"facts":["item-A","item-D"]},"expected_delta":{"added_facts":[],"removed_facts":["item-B","item-C"]}}
|
|
{"id":"5r-react-029","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["lone"]},"snapshot_t1":{"facts":[]},"expected_delta":{"added_facts":[],"removed_facts":["lone"]}}
|
|
{"id":"5r-react-030","battery":"5r","sub_battery":"react","version":"v1","carrier":"memory_snapshot","domain":"memory_root","pi_star_ref":"pi_memory_v1","snapshot_t0":{"facts":["p","q","r"]},"snapshot_t1":{"facts":["p","q","r"]},"expected_delta":{"added_facts":[],"removed_facts":[]}}
|