russell@unturf.com
d6f3834ab7
Implements the cross-modality discipline that ticket #000015 spelled out. Until now only text + claim_lattice + memory carriers existed in code; the multimodal story was theoretical. code-py-ast@v1 algorithm: 1. Parse UTF-8 bytes via stdlib ast.parse. 2. Walk the AST; emit deterministic S-expression "(NodeType field1=val1 ...)" with sorted fields, lists as "[elem0 elem1 ...]", primitives via repr(). 3. Source positions (lineno/col_offset) skipped naturally — not in ast._fields. Equivalence classes preserved (verified by tests): - Whitespace, indentation, blank lines - Comments - String quote style ('x' vs "x") - Operator spacing (1+2 vs 1 + 2) - Trailing semicolons (x=1;y=2 vs x=1\ny=2) Equivalence classes kept distinct: - Identifier names (def foo vs def bar) - Operator types (Add vs Sub) - Argument order in calls (f(x,y) vs f(y,x)) Projective, not invertible — canonical bytes are S-expression text, NOT valid Python. Re-canonicalizing the canonical output is undefined; idempotency tests run on the original raw input only. Surface: - arborist/pi_star/code.py — full implementation, replaces stub - bench/batteries/base.py — PHASE_1_CARRIERS adds "code" - bench/batteries/b_5s.py — runner accepts both pi_star_ref (cross-modality canonical) and pi_star (Phase 1a legacy) keys; backward-compat shim - bench/fixtures/5s/syntax-code-v1.jsonl — 10 Python parse-pass fixtures - bench/fixtures/5s/semantics-code-v1.jsonl — 12 equivalence-class fixtures (whitespace/comments/quote-style/operator collapse; identifier/operator/argument-order remain distinct) - Makefile: bench-5s-code target - tests/test_pi_star.py: stubs parametrize drops code-py-ast (graduated); 11 new tests for code-py-ast@v1 covering equivalence classes, distinguishing classes, syntax-error rejection, non-bytes rejection, determinism, empty-source handling, equivalence_class_id format Three remaining stubs (logic-kernel, time-series-quantized, tabular-pinned) keep their NotImplementedError contract. Full suite: 1221 passed, 36 skipped. Cross-modality discipline now has actual code-level proof, not just schema metadata.
13 lines
2.9 KiB
JSON
13 lines
2.9 KiB
JSON
{"_meta":{"battery":"5s","sub_battery":"semantics","version":"v1","task_count":12,"notes":"Code-carrier semantics: equivalence-class checks through code-py-ast@v1. Whitespace/comments/quote-style collapse; identifier/operator changes don't."}}
|
|
{"id":"5s-sem-code-001","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"def foo(): pass","input_b":"def foo():\n pass","expected_equivalent":true}
|
|
{"id":"5s-sem-code-002","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"x = 1","input_b":"x = 1 # comment","expected_equivalent":true}
|
|
{"id":"5s-sem-code-003","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"s = 'hello'","input_b":"s = \"hello\"","expected_equivalent":true}
|
|
{"id":"5s-sem-code-004","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"y = 1+2","input_b":"y = 1 + 2","expected_equivalent":true}
|
|
{"id":"5s-sem-code-005","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"x = [1, 2, 3]","input_b":"x = [\n 1,\n 2,\n 3,\n]","expected_equivalent":true}
|
|
{"id":"5s-sem-code-006","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"def foo(): pass","input_b":"def bar(): pass","expected_equivalent":false}
|
|
{"id":"5s-sem-code-007","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"y = a + b","input_b":"y = a - b","expected_equivalent":false}
|
|
{"id":"5s-sem-code-008","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"f(x, y)","input_b":"f(y, x)","expected_equivalent":false}
|
|
{"id":"5s-sem-code-009","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"return 42","input_b":"return 42","expected_equivalent":true}
|
|
{"id":"5s-sem-code-010","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"x = 1 ; y = 2","input_b":"x = 1\ny = 2","expected_equivalent":true}
|
|
{"id":"5s-sem-code-011","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"x = 1","input_b":"x = 2","expected_equivalent":false}
|
|
{"id":"5s-sem-code-012","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"class A:\n pass","input_b":"class B:\n pass","expected_equivalent":false}
|