pi_star: code-py-ast@v1 graduates from stub — first non-text canonicalizer
Implements the cross-modality discipline that ticket #000015 spelled out. Until now only text + claim_lattice + memory carriers existed in code; the multimodal story was theoretical. code-py-ast@v1 algorithm: 1. Parse UTF-8 bytes via stdlib ast.parse. 2. Walk the AST; emit deterministic S-expression "(NodeType field1=val1 ...)" with sorted fields, lists as "[elem0 elem1 ...]", primitives via repr(). 3. Source positions (lineno/col_offset) skipped naturally — not in ast._fields. Equivalence classes preserved (verified by tests): - Whitespace, indentation, blank lines - Comments - String quote style ('x' vs "x") - Operator spacing (1+2 vs 1 + 2) - Trailing semicolons (x=1;y=2 vs x=1\ny=2) Equivalence classes kept distinct: - Identifier names (def foo vs def bar) - Operator types (Add vs Sub) - Argument order in calls (f(x,y) vs f(y,x)) Projective, not invertible — canonical bytes are S-expression text, NOT valid Python. Re-canonicalizing the canonical output is undefined; idempotency tests run on the original raw input only. Surface: - arborist/pi_star/code.py — full implementation, replaces stub - bench/batteries/base.py — PHASE_1_CARRIERS adds "code" - bench/batteries/b_5s.py — runner accepts both pi_star_ref (cross-modality canonical) and pi_star (Phase 1a legacy) keys; backward-compat shim - bench/fixtures/5s/syntax-code-v1.jsonl — 10 Python parse-pass fixtures - bench/fixtures/5s/semantics-code-v1.jsonl — 12 equivalence-class fixtures (whitespace/comments/quote-style/operator collapse; identifier/operator/argument-order remain distinct) - Makefile: bench-5s-code target - tests/test_pi_star.py: stubs parametrize drops code-py-ast (graduated); 11 new tests for code-py-ast@v1 covering equivalence classes, distinguishing classes, syntax-error rejection, non-bytes rejection, determinism, empty-source handling, equivalence_class_id format Three remaining stubs (logic-kernel, time-series-quantized, tabular-pinned) keep their NotImplementedError contract. Full suite: 1221 passed, 36 skipped. Cross-modality discipline now has actual code-level proof, not just schema metadata.
This commit is contained in:
parent
8c618bbbc8
commit
d6f3834ab7
7 changed files with 231 additions and 14 deletions
6
Makefile
6
Makefile
|
|
@ -266,6 +266,12 @@ bench-5f: bootstrap ## 5F battery (Function+Finetuning+Falsification+Formulate+F
|
|||
PYTHONUNBUFFERED=1 $(PY) -m bench.batteries.runner --battery 5f --sub formulate --fixtures bench/fixtures/5f/formulate-v1.jsonl
|
||||
PYTHONUNBUFFERED=1 $(PY) -m bench.batteries.runner --battery 5f --sub feedback-loop --fixtures bench/fixtures/5f/feedback-loop-v1.jsonl
|
||||
|
||||
bench-5s-code: bootstrap ## 5S code-carrier (Syntax + Semantics through code-py-ast@v1)
|
||||
PYTHONUNBUFFERED=1 $(PY) -m bench.batteries.runner --battery 5s --sub syntax \
|
||||
--fixtures bench/fixtures/5s/syntax-code-v1.jsonl
|
||||
PYTHONUNBUFFERED=1 $(PY) -m bench.batteries.runner --battery 5s --sub semantics \
|
||||
--fixtures bench/fixtures/5s/semantics-code-v1.jsonl
|
||||
|
||||
bench-5f-formulate-live: bootstrap ## 5F Formulate via live arborist.qa.parse_claims (Phase 1b.2)
|
||||
PYTHONUNBUFFERED=1 $(PY) -m bench.batteries.runner --battery 5f --sub formulate \
|
||||
--fixtures bench/fixtures/5f/formulate-live-v1.jsonl
|
||||
|
|
|
|||
|
|
@ -1,16 +1,62 @@
|
|||
"""``code-py-ast@v1`` π* (stub).
|
||||
"""``code-py-ast@v1`` π* — Python source → canonical AST S-expression.
|
||||
|
||||
Domain: ``code``. Implementation lands in a follow-up ticket; this
|
||||
stub registers the namespace and raises :class:`NotImplementedError`
|
||||
on canonicalize.
|
||||
Domain: ``code``. First non-text canonical projection registered in
|
||||
arborist. Activates the cross-modality discipline that ticket
|
||||
#000015 spelled out — until now it was only theoretical (text +
|
||||
claim_lattice + memory carriers).
|
||||
|
||||
Planned semantics: parse Python source → AST → canonical S-expression
|
||||
serialization (sorted keyword args, normalized literals, stripped
|
||||
comments). Equivalent code → identical canonical bytes.
|
||||
Algorithm
|
||||
---------
|
||||
1. Parse ``raw`` UTF-8 bytes via stdlib :mod:`ast`.
|
||||
2. Walk every node, emit a deterministic S-expression of the form
|
||||
``(NodeType field1=val1 field2=val2 ...)`` where:
|
||||
- ``_fields`` are walked in **lexical order** (stable across
|
||||
Python versions).
|
||||
- Source positions (``lineno``, ``col_offset``,
|
||||
``end_lineno``, ``end_col_offset``, ``type_comment``) are NOT
|
||||
in ``_fields`` so they're naturally excluded.
|
||||
- Lists serialize as ``[elem0 elem1 ...]``.
|
||||
- Constants serialize via :func:`repr` (Python guarantees
|
||||
deterministic repr for primitives).
|
||||
3. Result is UTF-8-encoded bytes.
|
||||
|
||||
Equivalence classes preserved
|
||||
-----------------------------
|
||||
- Whitespace + indentation differences (don't enter AST).
|
||||
- Comments (don't enter AST).
|
||||
- String quote style (``'x'`` vs ``"x"`` parse to identical
|
||||
``Constant`` nodes).
|
||||
- Numeric literal formatting (``1`` vs ``01_000`` collapse to
|
||||
identical ``Constant`` value, modulo Python parser).
|
||||
- Trailing semicolons.
|
||||
|
||||
Equivalence classes kept distinct
|
||||
---------------------------------
|
||||
- Identifier names (``Name.id`` differs).
|
||||
- Operator types (``Add`` vs ``Sub`` distinct ``BinOp.op`` nodes).
|
||||
- Argument order in calls (positional order is semantic).
|
||||
|
||||
Round-trip property
|
||||
-------------------
|
||||
This π* is **projective**, not invertible — canonical bytes are an
|
||||
S-expression, NOT valid Python source. Re-canonicalizing the
|
||||
canonical output is undefined (the S-expression isn't Python). v1
|
||||
runners that need round-trip stability test on the *original raw*
|
||||
input only.
|
||||
|
||||
Versioning
|
||||
----------
|
||||
``code-py-ast@v1`` pins this algorithm. Any change (sort order,
|
||||
field exclusion, list bracket style, literal repr) requires a
|
||||
new version number — never mutate v1 in place.
|
||||
|
||||
Source: ticket #000015 §2.1 declared the stub; this commit lands
|
||||
the first real non-text canonicalizer.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
from dataclasses import dataclass
|
||||
|
||||
from arborist.pi_star.protocol import PiStarError
|
||||
|
|
@ -24,10 +70,54 @@ class CodePyAstV1:
|
|||
domain: str = "code"
|
||||
|
||||
def canonicalize(self, raw: bytes) -> bytes:
|
||||
raise NotImplementedError(
|
||||
"code-py-ast@v1 is a stub; implementation ticket pending. "
|
||||
"See docs/tickets/ticket-000015-pi-star-domain-library.md."
|
||||
)
|
||||
if not isinstance(raw, (bytes, bytearray)):
|
||||
raise PiStarError(
|
||||
"code-py-ast@v1 expects bytes; got "
|
||||
f"{type(raw).__name__}"
|
||||
)
|
||||
try:
|
||||
source = raw.decode("utf-8", errors="surrogatepass")
|
||||
except UnicodeDecodeError as exc: # pragma: no cover — defensive
|
||||
raise PiStarError(f"input not valid UTF-8: {exc}") from exc
|
||||
try:
|
||||
tree = ast.parse(source)
|
||||
except SyntaxError as exc:
|
||||
raise PiStarError(
|
||||
f"code-py-ast@v1 input is not valid Python: {exc}"
|
||||
) from exc
|
||||
return _canonical_sexp(tree).encode("utf-8", errors="surrogatepass")
|
||||
|
||||
|
||||
def _canonical_sexp(node) -> str:
|
||||
"""Walk an AST; emit a deterministic S-expression. Field order
|
||||
is lexical so the encoding is stable regardless of how the AST
|
||||
constructor populated attributes."""
|
||||
if isinstance(node, ast.AST):
|
||||
cls_name = type(node).__name__
|
||||
# Walk _fields in lexical order. ast nodes set this tuple at
|
||||
# class-definition time; using sorted() makes us robust if
|
||||
# Python ever reorders.
|
||||
parts = []
|
||||
for fname in sorted(node._fields):
|
||||
fval = getattr(node, fname, None)
|
||||
parts.append(f"{fname}={_canonical_sexp(fval)}")
|
||||
return f"({cls_name} {' '.join(parts)})" if parts else f"({cls_name})"
|
||||
if isinstance(node, list):
|
||||
return "[" + " ".join(_canonical_sexp(x) for x in node) + "]"
|
||||
if node is None:
|
||||
return "None"
|
||||
if isinstance(node, bool):
|
||||
# bool BEFORE int (bool is a subclass of int).
|
||||
return "True" if node else "False"
|
||||
if isinstance(node, (int, float, complex)):
|
||||
return repr(node)
|
||||
if isinstance(node, str):
|
||||
return repr(node)
|
||||
if isinstance(node, bytes):
|
||||
return repr(node)
|
||||
# Catch-all for anything we didn't anticipate. repr() is
|
||||
# deterministic for stdlib types.
|
||||
return repr(node)
|
||||
|
||||
|
||||
register(CodePyAstV1())
|
||||
|
|
|
|||
|
|
@ -72,7 +72,7 @@ def run_syntax(fixtures_path: Path) -> BatteryResult:
|
|||
continue
|
||||
detail: dict = {}
|
||||
try:
|
||||
pi_star = get(task["pi_star"])
|
||||
pi_star = get(task.get("pi_star_ref", task.get("pi_star")))
|
||||
pi_star.canonicalize(task["input"].encode("utf-8"))
|
||||
ok = task.get("expected", "pass") == "pass"
|
||||
if not ok:
|
||||
|
|
@ -124,7 +124,7 @@ def run_semantics(fixtures_path: Path) -> BatteryResult:
|
|||
continue
|
||||
detail: dict = {}
|
||||
try:
|
||||
pi_star = get(task["pi_star"])
|
||||
pi_star = get(task.get("pi_star_ref", task.get("pi_star")))
|
||||
ca = pi_star.canonicalize(task["input_a"].encode("utf-8"))
|
||||
cb = pi_star.canonicalize(task["input_b"].encode("utf-8"))
|
||||
equivalent = ca == cb
|
||||
|
|
|
|||
|
|
@ -110,6 +110,8 @@ PHASE_1_CARRIERS = frozenset({
|
|||
"qa_answer",
|
||||
"capability_transition",
|
||||
"memory_root",
|
||||
# First non-text carrier, landed via code-py-ast@v1 graduation.
|
||||
"code",
|
||||
})
|
||||
|
||||
|
||||
|
|
|
|||
13
bench/fixtures/5s/semantics-code-v1.jsonl
Normal file
13
bench/fixtures/5s/semantics-code-v1.jsonl
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
{"_meta":{"battery":"5s","sub_battery":"semantics","version":"v1","task_count":12,"notes":"Code-carrier semantics: equivalence-class checks through code-py-ast@v1. Whitespace/comments/quote-style collapse; identifier/operator changes don't."}}
|
||||
{"id":"5s-sem-code-001","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"def foo(): pass","input_b":"def foo():\n pass","expected_equivalent":true}
|
||||
{"id":"5s-sem-code-002","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"x = 1","input_b":"x = 1 # comment","expected_equivalent":true}
|
||||
{"id":"5s-sem-code-003","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"s = 'hello'","input_b":"s = \"hello\"","expected_equivalent":true}
|
||||
{"id":"5s-sem-code-004","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"y = 1+2","input_b":"y = 1 + 2","expected_equivalent":true}
|
||||
{"id":"5s-sem-code-005","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"x = [1, 2, 3]","input_b":"x = [\n 1,\n 2,\n 3,\n]","expected_equivalent":true}
|
||||
{"id":"5s-sem-code-006","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"def foo(): pass","input_b":"def bar(): pass","expected_equivalent":false}
|
||||
{"id":"5s-sem-code-007","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"y = a + b","input_b":"y = a - b","expected_equivalent":false}
|
||||
{"id":"5s-sem-code-008","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"f(x, y)","input_b":"f(y, x)","expected_equivalent":false}
|
||||
{"id":"5s-sem-code-009","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"return 42","input_b":"return 42","expected_equivalent":true}
|
||||
{"id":"5s-sem-code-010","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"x = 1 ; y = 2","input_b":"x = 1\ny = 2","expected_equivalent":true}
|
||||
{"id":"5s-sem-code-011","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"x = 1","input_b":"x = 2","expected_equivalent":false}
|
||||
{"id":"5s-sem-code-012","battery":"5s","sub_battery":"semantics","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input_a":"class A:\n pass","input_b":"class B:\n pass","expected_equivalent":false}
|
||||
11
bench/fixtures/5s/syntax-code-v1.jsonl
Normal file
11
bench/fixtures/5s/syntax-code-v1.jsonl
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
{"_meta":{"battery":"5s","sub_battery":"syntax","version":"v1","task_count":10,"notes":"First code-carrier fixtures. Each input is Python source; pi_star is code-py-ast@v1 (the graduated stub). Tests the cross-modality discipline end-to-end through the bench harness."}}
|
||||
{"id":"5s-syn-code-001","battery":"5s","sub_battery":"syntax","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input":"x = 1","expected":"pass"}
|
||||
{"id":"5s-syn-code-002","battery":"5s","sub_battery":"syntax","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input":"def foo(): pass","expected":"pass"}
|
||||
{"id":"5s-syn-code-003","battery":"5s","sub_battery":"syntax","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input":"class Foo:\n def bar(self):\n return 42","expected":"pass"}
|
||||
{"id":"5s-syn-code-004","battery":"5s","sub_battery":"syntax","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input":"y = a + b * c","expected":"pass"}
|
||||
{"id":"5s-syn-code-005","battery":"5s","sub_battery":"syntax","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input":"for i in range(10):\n print(i)","expected":"pass"}
|
||||
{"id":"5s-syn-code-006","battery":"5s","sub_battery":"syntax","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input":"import sys\nimport os","expected":"pass"}
|
||||
{"id":"5s-syn-code-007","battery":"5s","sub_battery":"syntax","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input":"if x > 0:\n y = 1\nelse:\n y = -1","expected":"pass"}
|
||||
{"id":"5s-syn-code-008","battery":"5s","sub_battery":"syntax","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input":"data = [1, 2, 3, 4, 5]","expected":"pass"}
|
||||
{"id":"5s-syn-code-009","battery":"5s","sub_battery":"syntax","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input":"d = {'a': 1, 'b': 2}","expected":"pass"}
|
||||
{"id":"5s-syn-code-010","battery":"5s","sub_battery":"syntax","version":"v1","carrier":"code","domain":"python_ast","pi_star_ref":"code-py-ast@v1","input":"","expected":"pass"}
|
||||
|
|
@ -127,7 +127,8 @@ def test_claim_lattice_empty_input_returns_empty_array():
|
|||
@pytest.mark.parametrize(
|
||||
"key",
|
||||
[
|
||||
"code-py-ast@v1",
|
||||
# code-py-ast@v1 graduated from stub to real implementation —
|
||||
# see test_code_py_ast_* below.
|
||||
"logic-kernel@v1",
|
||||
"time-series-quantized@v1",
|
||||
"tabular-pinned@v1",
|
||||
|
|
@ -139,6 +140,100 @@ def test_stubs_raise_not_implemented(key):
|
|||
pi_star.canonicalize(b"anything")
|
||||
|
||||
|
||||
# --- code-py-ast@v1 (graduated from stub) ----------------------------
|
||||
|
||||
|
||||
def test_code_py_ast_canonicalizes_simple_assign():
|
||||
pi_star = get("code-py-ast@v1")
|
||||
out = pi_star.canonicalize(b"x = 1")
|
||||
assert isinstance(out, bytes)
|
||||
assert b"Assign" in out
|
||||
assert b"Constant" in out
|
||||
|
||||
|
||||
def test_code_py_ast_collapses_whitespace_and_comments():
|
||||
pi_star = get("code-py-ast@v1")
|
||||
a = pi_star.canonicalize(b"def foo(): pass")
|
||||
b = pi_star.canonicalize(b"def foo():\n pass # comment")
|
||||
assert a == b
|
||||
|
||||
|
||||
def test_code_py_ast_collapses_quote_style():
|
||||
pi_star = get("code-py-ast@v1")
|
||||
a = pi_star.canonicalize(b"s = 'hello'")
|
||||
b = pi_star.canonicalize(b's = "hello"')
|
||||
assert a == b
|
||||
|
||||
|
||||
def test_code_py_ast_collapses_operator_spacing():
|
||||
pi_star = get("code-py-ast@v1")
|
||||
a = pi_star.canonicalize(b"y = 1+2")
|
||||
b = pi_star.canonicalize(b"y = 1 + 2")
|
||||
assert a == b
|
||||
|
||||
|
||||
def test_code_py_ast_distinguishes_identifiers():
|
||||
pi_star = get("code-py-ast@v1")
|
||||
a = pi_star.canonicalize(b"def foo(): pass")
|
||||
b = pi_star.canonicalize(b"def bar(): pass")
|
||||
assert a != b
|
||||
|
||||
|
||||
def test_code_py_ast_distinguishes_operators():
|
||||
pi_star = get("code-py-ast@v1")
|
||||
a = pi_star.canonicalize(b"y = a + b")
|
||||
b = pi_star.canonicalize(b"y = a - b")
|
||||
assert a != b
|
||||
|
||||
|
||||
def test_code_py_ast_distinguishes_argument_order():
|
||||
pi_star = get("code-py-ast@v1")
|
||||
a = pi_star.canonicalize(b"f(x, y)")
|
||||
b = pi_star.canonicalize(b"f(y, x)")
|
||||
assert a != b
|
||||
|
||||
|
||||
def test_code_py_ast_rejects_non_bytes():
|
||||
from arborist.pi_star.protocol import PiStarError
|
||||
|
||||
pi_star = get("code-py-ast@v1")
|
||||
with pytest.raises(PiStarError):
|
||||
pi_star.canonicalize("not bytes") # type: ignore[arg-type]
|
||||
|
||||
|
||||
def test_code_py_ast_rejects_invalid_python():
|
||||
from arborist.pi_star.protocol import PiStarError
|
||||
|
||||
pi_star = get("code-py-ast@v1")
|
||||
with pytest.raises(PiStarError):
|
||||
pi_star.canonicalize(b"def foo( malformed")
|
||||
|
||||
|
||||
def test_code_py_ast_deterministic_across_repeated_calls():
|
||||
pi_star = get("code-py-ast@v1")
|
||||
src = b"class Foo:\n def bar(self, x):\n return x + 1\n"
|
||||
a = pi_star.canonicalize(src)
|
||||
b = pi_star.canonicalize(src)
|
||||
c = pi_star.canonicalize(src)
|
||||
assert a == b == c
|
||||
|
||||
|
||||
def test_code_py_ast_handles_empty_source():
|
||||
pi_star = get("code-py-ast@v1")
|
||||
out = pi_star.canonicalize(b"")
|
||||
# Empty source parses to ast.Module(body=[], type_ignores=[]).
|
||||
assert b"Module" in out
|
||||
assert b"body=[]" in out
|
||||
|
||||
|
||||
def test_code_py_ast_equivalence_class_id_format():
|
||||
pi_star = get("code-py-ast@v1")
|
||||
eid = equivalence_class_id(pi_star, b"x = 1")
|
||||
assert len(eid) == 64
|
||||
# Same source twice → same id.
|
||||
assert eid == equivalence_class_id(pi_star, b"x = 1")
|
||||
|
||||
|
||||
# --- composition --------------------------------------------------
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue