Closes the three open Phase-1b items of #000025; every §10 closure
criterion is now met, so the ticket flips to closed.
§10.14 — ForkScore threshold-calibration handoff to #000012.
bench/scripts/fivef_threshold_calibration.py (make bench-5f-threshold-
calibration) runs the canonical 5S/5T/5F packs + the 5F live packs and
reports baseline rates, observability granularity (1/n), and fork_score
verdicts on the parent vs synthetic child perturbations →
bench/results/5f-threshold-calibration-2026-05-11.md. Findings written
into ticket-000012 §8: keep SIGNAL_FLOOR / HARD_REGRESSION_FLOOR at
0.05; the small 5S packs (syntax n=10, semantics n=8) are coarser than
the floors so any regression there trips hard-reject (intended zero-
tolerance); the 5x averaging dilution in _delta_*; ceiling saturation
(every pack at 1.0 -> delta-rate terms <= 0). No constant change
shipped. 6 tests in tests/test_fivef_threshold_calibration.py.
§10.13 — feedback latency / efficiency on real workload.
run_feedback_loop now computes feedback_latency (listed in §5.5 since
Phase 1a, never implemented) — wall-clock seconds to apply a live
chain against its temp shard, surfaced per-task
(feedback_latency_seconds) + battery (feedback_latency_mean_seconds,
feedback_live_task_count). For live chains feedback_efficiency's cost
denominator switched from len(chain) (count of requested ops) to the
persisted footprint _persisted_cost = audit-event rows the chain
actually wrote + their body bytes / 1e6. Embedded chains keep
len(chain) and report feedback_latency_seconds = None. Latency is a
wall-clock field (run-to-run variable, like BatteryResult.timestamp)
and is not a fork_score input. 3 tests in tests/test_bench_batteries.py.
§10.11 — real selfmodel finetuning chains.
bench/scripts/selfmodel_chain_snapshot.py (make bench-5f-selfmodel-
snapshot) appends one chained SelfModel snapshot per run to a
persistent shard (~/.arborist/shards/selfmodel-chain.db, override via
ARBORIST_SELFMODEL_CHAIN_DB) with one CapabilityClaim per sub-battery
(metric = "5S-syntax" etc., measured_value = that pack's rate,
eval_digest = the pack's fixture digest, threshold = SIGNAL_FLOOR).
snapshot() auto-parents, so each snapshot is a distinct root and the
lineage grows by one per run. run_finetuning gains a third dispatch
mode — shard-chain (gated on a task's selfmodel_shard key) — via
_chain_finetuning_measure: reads the two most-recent snapshots
(latest() = child, its parent_selfmodel_root = parent) and measures
improvement on target_capability between them. This is the real
lineage replacing Phase-1a's synthetic parent->child pairs; the
chained delta reflects genuine cross-run drift (0.0 today — the
embedded packs are at ceiling). Operator pack
bench/fixtures/5f/finetuning-shardchain-v1.jsonl (6 tasks) + make
bench-5f-finetuning-shardchain; not in `make bench-5f`, `make test`,
or a fresh checkout (a missing/too-short chain fails honestly). The
real chain shard was bootstrapped 2-deep on 2026-05-11; make
chain-check-shards reports 0 breaks on it (and all other shards).
10 tests in tests/test_selfmodel_chain.py.
Full suite: 2311 passed, 28 skipped.
A new π* kernel that canonicalizes pure-integer counting
expressions and FAILS CLOSED on any input whose result isn't a
non-negative sp.Integer. Tighter domain than algebra-symbolic@v1,
which already accepts the same input surface but happily returns
symbolic / negative / non-integer outputs.
Distinguishing feature versus algebra-symbolic@v1:
algebra-symbolic@v1: binomial(n, k) → "binomial(n, k)" (symbolic
passthrough)
combinatorics@v1: binomial(n, k) → PiStarError (fail-closed
on free-symbol output)
algebra-symbolic@v1: binomial(Rational(1,2), 3) → 1/16 (rational)
combinatorics@v1: binomial(Rational(1,2), 3) → PiStarError
(output not Integer)
Boundary kept explicit: binomial(-3, 2) = 6 IS accepted because the
output is an integer 6. The fail-closed rule is on output shape
(Integer ≥ 0), not input range. Documented as
test_generalized_binomial_negative_args_accepted_when_integer.
Output format: plain decimal literal (b"10", b"5040"). Composes
with arithmetic@v1 for byte-identical agreement with the rational
route (b"10/1") so the multi-modality witness (#000028) can pin
equivalence-class agreement when both routes fire on the same
question.
Allowed surface (via SymPy primitives): binomial, factorial, ff /
rf (falling/rising), catalan, bell, partition, stirling, plus
arithmetic compositions over those primitives
(3*binomial(5,2) + factorial(4) = 54).
Coverage:
- 43 unit tests including binomial symmetry C(n,k)=C(n,n-k),
Pascal's rule C(n,k)=C(n-1,k-1)+C(n-1,k), the C(n,k) =
factorial(n)/(factorial(k)·factorial(n-k)) identity,
fail-closed paths (symbolic/negative/non-integer/relational/
parse), round-trip idempotence, composition with arithmetic@v1.
- 10 syntax + 12 semantics bench fixtures, 100% pass.
- bench/batteries/base.py PHASE_1_CARRIERS gains "combinatorics".
- Makefile bench-5s-combinatorics target.
All gate on pytest.importorskip("sympy") so a sympy-less suite
stays green. Full make test: 1537 passed / 28 skipped.
Sequencing rationale honored: this kernel lands FIRST so that
#000033 (claim-pack pillar VII for combinatorics) can bind its
records to the tighter integer kernel from day one — avoids
rebind churn on pi_star_ref fields.
tabular-pinned@v1 + calculus-limit@v1 + calculus-series@v1 +
linear-algebra@v1 + function-sampled@v1 — all reserved stubs
graduated; the π* registry is now 15 concrete kernels with no
remaining reserved-stub entries.
#000030 Phase 4 — calculus-limit@v1
====================================
sp.limit with thread-timeout. One-sided dir support (+/-/+-).
Pinned spelling for infinity cases: b"+oo" / b"-oo" / b"zoo"
(complex infinity) — bypasses sp.expand since Infinity isn't
algebraic. Finite results re-canonicalize through algebra-symbolic
recipe (sp.expand + sp.srepr). Unevaluated cases / timeouts emit
b"unevaluated:" + sp.srepr(<Limit>) sentinel, mirroring
calculus-integral's pattern.
#000030 Phase 5 — calculus-series@v1
=====================================
sp.series(f, x, x0, n).removeO() → sp.expand → sp.srepr. Drops
O(x**n) remainder explicitly so the canonical form is finite-byte.
Sentinel format mirrors limit/integral: b"unevaluated:Series(...)"
on timeout. n must be a positive int; 0 / float / negative rejected.
#000030 Phase 6 — linear-algebra@v1
====================================
Single π* covers the whole linear-algebra surface via {op, matrix}
JSON. Ops: rref / det / eigenvalues / inverse. Matrix cells go
through Fraction(Decimal(str(...))) for floats so 1, 1.0, "1.0"
all collapse to Rational(1, 1) — matching arithmetic@v1's
discipline. Without this fold, sp.sympify keeps floats as Float
(separate type) and downstream det/inverse return Float-shaped
bytes. Eigenvalues are sorted by srepr for determinism.
Output formats:
rref / inverse: rows/cols header + cells joined by | (rows by ||)
det: det:<num/den-or-srepr>
eigenvalues: eigenvalues:<value-1>x<mult-1>|...
#000030 Phase 7 — function-sampled@v1
======================================
Bridge to time-series-quantized@v1. SymPy expression + linspace
grid → quantized integer-vector signature in time-series's exact
output format (dt=...;dv=...;n=...;t0=0:v0|v1|...). Two functions
that render identically (within sample-grid tolerance) collapse
to the same canonical bytes. This is what plotting CAN become
in π* terms — the PNG render is a downstream view of the same
canonical evidence.
Math-only sampler (no numpy in the dep surface); Python's round()
is banker's-rounding so the bytes are interchangeable with
time-series-quantized@v1's output. Complex / non-finite samples
raise PiStarError rather than silently dropping imaginary parts.
tabular-pinned@v1 — last reserved stub graduates
=================================================
JSON-rows input ({schema, key_columns, rows}); declared
key_columns sort policy (stable sort by primary-key tuple);
type-fold per column (int/rational/bool through arithmetic@v1
discipline; str verbatim; bool normalized). Header case is
PINNED EXACT — Excel and PostgreSQL both care about case;
defaulting to lowercase-fold would break operator expectations.
Output: header (schema + key + n) + rows joined by \n + cells by |.
The π* registry has no remaining reserved stubs. Every modality
the substrate paper reserved is now real.
Test suite: 1568 passed (was 1467; +101). New closure-criterion
test (test_no_stub_pi_stars_remain) replaces the old reserved-stub
parametrize — adding a future stub re-opens this list.
110/110 fixtures pass across the 5 new bench-5s-* targets.
PHASE_1_CARRIERS gained calculus / linear-algebra / function-sampled
/ tabular.
Two new π* canonicalizers extend the math substrate above
arithmetic@v1 (closed-form rationals) and logic-kernel@v1
(propositional Boolean → CNF):
algebra-symbolic@v1 (Phase 1) — symbolic-algebra domain.
sp.expand → sp.srepr canonical bytes. Polynomial identity collapses
((x+1)**2 ≡ x**2 + 2*x + 1); exponential identity collapses
(exp(a+b) ≡ exp(a)*exp(b), inherited from sp.expand's default
behavior); trigonometric identity does NOT collapse
(sin²+cos² ≢ 1). The trig surface is reserved for a future
algebra-symbolic-simplified@v1 variant that wraps sp.simplify at
unbounded CPU cost. Rejects relationals (`x > 0`) and
BooleanFunction shapes (`x & y`) via `isinstance(expr, sp.Expr)` —
sp.Symbol confusingly inherits from Boolean so the right rejection
filter is "not Expr" rather than "Boolean".
calculus-derivative@v1 (Phase 2) — calculus domain. JSON-shaped
{f, x, n} input → sp.diff → sp.expand → srepr bytes. Output is
itself a valid algebra-symbolic@v1 input so the two compose
naturally under arborist.pi_star.compose. n defaults to 1; bools
explicitly rejected (Python isinstance(True, int) is True so we
filter that explicitly).
Optional dependency: sympy ships in the new [math] extra
(pyproject.toml). Folded into [dev] so make bootstrap pulls it
transitively. An explicit `bootstrap-math` Makefile target documents
the opt-in for minimal-install users. Both modules self-guard
via `try: import sympy as sp / except ImportError: sp = None` and
only register(...) when sympy is present, so a fresh checkout
without [math] still loads arborist.pi_star without raising.
Preflight algebra route lands in
arborist.qa.query._canonical_projection_preflight between the
arithmetic and logic routes. Charset regex (_CANONICAL_ALGEBRA_RE)
allows lowercase letters + math chars; requires at least one
letter (else arithmetic wins); rejects natural-language leading
verbs via _CANONICAL_ALGEBRA_NL_LEAD_RE (4-letter minimum so
single-/two-/three-char identifiers like x, xy, sin, cos, pi
survive while "simplify (...)", "factor x...", "expand (a+b)..."
fall through). PiStarError + KeyError both fall through cleanly
so a sympy-less install just routes everything past algebra.
Bench substrate:
- bench/batteries/base.py PHASE_1_CARRIERS gains "symbolic_algebra"
- bench/fixtures/5s/syntax-algebra-symbolic-v1.jsonl (10 fixtures)
- bench/fixtures/5s/semantics-algebra-symbolic-v1.jsonl (13 fixtures
including the documented trig non-collapse + exp collapse)
- Makefile bench-5s-algebra target → 100% pass
Tests: 18 algebra-symbolic + 38 calculus-derivative unit tests +
~10 new preflight-route tests in test_canonical_projection.py. All
gate on pytest.importorskip("sympy") so a sympy-less suite stays
green. Full suite: 1369 passed / 27 skipped.
Phases 3-7 (integral, limit, series, linear-algebra,
function-sampled) remain open as future work; each lands as its
own ticket when an actual consumer surfaces.
Closes the math half of fox's 2026-05-08 roadmap question ("did we
implement the math and logic stuff?"). Until now arborist had logic
KERNELS (Syllogism, Truthtables, Transitivity sub-batteries with
deterministic evaluators) but no math π*'s. SQD whitepaper §14
framed math as "operations over π*-canonical invariant objects:
integers, algebraic expressions, proof states, constraint graphs"
— that framing is now actually implementable.
arithmetic@v1 — exact rational arithmetic (SQD §14.1)
-----------------------------------------------------
Parse arithmetic expression → fractions.Fraction → canonical
"<num>/<den>" in lowest terms. Decimals via Decimal(str(...)) for
exact rational interpretation:
"0.1" → "1/10" (not 0.1±ε)
"0.1+0.2" → "3/10" (the SQD-canonical floating-point question)
"1+2" → "3/1"
"(1+2)*3" → "9/1"
"6/4" → "3/2" (lowest terms via Fraction invariant)
"2**3" → "8/1"
"-(1+2)" → "-3/1"
Rejects: identifiers, function calls, division by zero, non-integer
exponents, boolean literals. PiStarError with explanatory message.
logic-kernel@v1 — propositional CNF canonicalizer (SQD §14.3)
-------------------------------------------------------------
Parse Boolean expression (AND/OR/NOT/XOR/IMPL/IFF + named atoms +
parens) → eliminate IMPL/IFF/XOR → push NOT inward (NNF) →
distribute OR over AND (CNF) → dedupe + sort literals + sort
clauses + drop tautological clauses → serialize.
Equivalence classes preserved:
A AND B ≡ B AND A (commutativity)
(A AND B) AND C ≡ A AND (B AND C) (associativity)
A IMPL B ≡ NOT A OR B (IMPL rewrite)
A IMPL B ≡ NOT B IMPL NOT A (contrapositive)
NOT (A AND B) ≡ NOT A OR NOT B (De Morgan)
NOT NOT A ≡ A (double negation)
A OR (B AND C) ≡ (A OR B) AND (A OR C) (distribution)
A OR NOT A → TRUE (tautology)
A AND A ≡ A (idempotence)
Atom cap: N=8 (256 max clauses). Larger inputs raise PiStarError;
CNF blow-up is exponential in atom count, the cap keeps
canonicalization deterministic in bounded time.
Surface
-------
- arborist/pi_star/arithmetic.py — full implementation (new file)
- arborist/pi_star/logic.py — full implementation (graduates from
stub; preserves the LogicKernelV1 dataclass shape so the registry
key arithmetic@v1 / logic-kernel@v1 stays stable)
- arborist/pi_star/__init__.py — imports arithmetic to auto-register
- bench/batteries/base.py — PHASE_1_CARRIERS adds "arithmetic" + "logic"
- bench/fixtures/5s/syntax-arithmetic-v1.jsonl (12 fixtures)
- bench/fixtures/5s/semantics-arithmetic-v1.jsonl (15 fixtures)
- bench/fixtures/5s/syntax-logic-v1.jsonl (12 fixtures)
- bench/fixtures/5s/semantics-logic-v1.jsonl (18 fixtures)
- Makefile: bench-5s-arithmetic, bench-5s-logic-kernel,
bench-5s-math aggregate
- tests/test_pi_star.py: 30 new tests
- 13 for arithmetic@v1 (decimals, equivalence, rejects, idempotency,
integer exponents, negative results, the SQD 0.1+0.2 case)
- 17 for logic-kernel@v1 (commutativity, associativity, all rewrite
rules, De Morgan, double-neg, distribution, tautology, idempotence,
contrapositive, atom cap, syntax errors, idempotency)
Cross-modality discipline now spans:
text + claim_lattice + memory + code + arithmetic + logic
— six carrier domains, all with real π* implementations.
Two stubs remain (time-series-quantized@v1, tabular-pinned@v1);
neither is needed for math/logic coverage.
Full suite: 1256 passed, 36 skipped (+30 tests, +57 fixtures).
Completes the bridge: every 5F sub-battery now has an embedded
(Phase 1a) AND a live (Phase 1b.2) path. Per-task detail.source
distinguishes synthetic from live signal.
| Sub-battery | Live surface | Live fixtures |
|---|---|---|
| Formulate | qa.parse_claims.parse_pointer_claims | 15 (prior) |
| Feedback Loop | store.append_audit + memory.snapshot (temp shard) | 12 (prior) |
| Function | parse_pointer_claims + shape evaluators | 12 |
| Finetuning | selfmodel.store_snapshot + claims_for round-trip | 10 |
| Falsification | qa.verify.verify_quotes (real verifier) | 12 |
Function live: input_text routes through real parse_pointer_claims;
the same Phase-1a shape evaluators (shape_match / pointer_set_match
/ threshold_on_metric) run on the live-parsed output. Tests
parser→shape pipeline against actual organism behavior.
Finetuning live: gated via fixture's "live": true flag. Runner
writes parent + child SelfModel + capability_claim to a fresh temp
shard via real arborist.selfmodel.store_snapshot, reads back via
claims_for, runs the same improvement check. Tests the SelfModel
persistence + claim-attach surface, not just static fixture data.
Falsification live: answer_text + context routes through real
arborist.qa.verify.verify_quotes. Live signals translate the
verifier's audit_mode + verifier_method + unverified_quotes into
flat tags (UNGROUNDED, STRICT_<method>, HYBRID_<method>,
UNVERIFIED_QUOTE) the runner matches against expected_reason. One
fixture (5f-fal-live-003) deliberately documents a known soft-
signal gap — the entity strategy treats Insulin/Penicillin claims
as equivalent because both share "Alexander Fleming". Production
catches it via title-relevance + claim-lattice verifier; the
fixture pins the soft-path limit so future verifier changes
re-trigger review.
Surface:
- bench/batteries/b_5f.py — _live_function_produced,
_live_finetuning_measure, _live_falsification_violations
helpers + two-mode dispatch in run_function / run_finetuning /
run_falsification.
- bench/fixtures/5f/{function,finetuning,falsification}-live-v1.jsonl
(12 + 10 + 12 fixtures).
- Makefile: bench-5f-{function,finetuning,falsification}-live
targets; bench-5f-live aggregate now covers all five sub-batteries.
- Tests: 9 new in tests/test_bench_batteries.py
- live + embedded paths for each sub-battery (6 tests)
- direct helper tests verifying the real arborist surfaces are
invoked, not stubs (3 tests)
Phase 1a fixture digests unchanged. _DEFAULT_FIXTURES still points
at synthetic Phase-1a fixtures so `runner --all` behavior is
identical and deterministic; live mode invoked via explicit
--fixtures path or make targets.
Full suite: 1210 passed, 36 skipped.
5F battery is now the first to bridge synthetic→live across every
sub-battery. The pattern + live fixture format are reusable for
5R Phase 1b.2 (when SelfModel-backed workspace ops want live
verification).
Second live wire-up (after Formulate). run_feedback_loop now supports
both modes:
- Embedded (Phase 1a): chain of (operation, observation) string pairs;
runner aggregates observations and string-matches expected_delta.
- Live (Phase 1b.2): live_chain of typed ops applied to a fresh temp
arborist shard via real append_audit + memory.snapshot +
selfmodel.snapshot. expected_delta is a dict of predicates against
the resulting audit_events / memory_branch_summaries.
The live helper _live_feedback_chain creates a tempfile-backed shard,
runs the chain through real arborist surfaces, and queries the final
state. Every audit event chains via the production append_audit, so
the audit chain is re-verifiable after live execution (see new test
test_5f_live_feedback_chain_audit_chain_intact).
Three predicate types in expected_delta:
- audit_event_type_present: named event_type appears in audit chain
- memory_branch_present: named branch_id in memory_branch_summaries
- body_substring_present: substring appears in any audit body JSON
Surface:
- bench/batteries/b_5f.py — _live_feedback_chain helper +
_live_delta_satisfied predicate checker; two-mode dispatch in
run_feedback_loop
- bench/fixtures/5f/feedback-loop-live-v1.jsonl — 12 live fixtures
exercising providence_write, providence_repair, memory_snapshot,
selfmodel_snapshot ops. Includes 2 negative fixtures testing the
predicate checker (expected_delta absent → expected:fail).
- Makefile: bench-5f-feedback-loop-live + bench-5f-live aggregate
for all 5F Phase-1b.2 live wire-ups.
- Tests: 5 new in tests/test_bench_batteries.py
- live path runs all 12 fixtures
- embedded path still works (10 Phase-1a fixtures)
- helper directly tests audit-event write
- rejects unknown live op type
- audit chain re-verifies after live ops
Two of the five 5F sub-batteries now bridge synthetic → live
(Formulate + Feedback Loop). Function/Finetuning/Falsification
follow in subsequent commits.
Full suite: 1201 passed, 36 skipped.
First sub-battery to bridge from synthetic gold output to actual
organism behavior. run_formulate now supports two fixture modes
selected per-task:
- Embedded (Phase 1a): produced_lattice in the fixture. 10 seed
fixtures continue to pass via this path.
- Live (Phase 1b.2): only input_text in the fixture; runner calls
arborist.qa.parse_claims.parse_pointer_claims(input_text) and
matches the live output against expected_lattice.
Embedded takes precedence if both fields are present. Per-task
detail.source ("embedded" | "live") surfaces in bench output so
synthetic vs live signal is distinguishable.
Surface:
- bench/batteries/b_5f.py — _live_produced_lattice helper +
two-mode dispatch in run_formulate
- bench/fixtures/5f/formulate-live-v1.jsonl — 15 live-mode fixtures
with input_text + expected_lattice (no produced_lattice)
- Makefile: bench-5f-formulate-live target
- Tests: 4 new in tests/test_bench_batteries.py
- live path routes through real parser, all 15 pass
- embedded path still works (10 Phase-1a fixtures)
- _live_produced_lattice helper directly verifies parse output
- fixture missing both fields fails cleanly with explanatory reason
Phase 1a fixture digests unchanged. _DEFAULT_FIXTURES still points
at formulate-v1.jsonl so `runner --all` behavior is identical;
live-mode fixtures invoked via explicit --fixtures path.
Full suite: 1196 passed, 36 skipped.
Pattern set. Function/Finetuning/Falsification/Feedback Loop
follow in subsequent commits.
Phase 2 of #000021. React/Rearrange/Restore/Replicate/Resonate over
the workspace surface — selfmodel_records (#000014) + memory_records
(#000017), both landed earlier today. Closes the gap that gated 5R
since the substrate work shipped.
Sub-battery semantics (per SQD whitepaper §9.3 + ticket #000021 §4.2):
- React: incorporate new fact/constraint. Workspace = (snapshot_t0,
snapshot_t1, expected_delta). Pass = added_facts present + removed_facts
absent in t+1.
- Rearrange: restructure without semantic shift. Re-canonicalize
different surface forms through a named π*; pass = bytes match
expected_equivalent flag. Tests the order-invariance contracts in
SelfModel (capability_claim_hashes sorted) and Memory (branches
sorted by branch_id).
- Restore: retrieve prior fact. Workspace = (history[], current_facts[]).
Pass = fact in current OR any historical snapshot.
- Replicate: independent canonical encodings via π*. Same input run
N times must yield byte-equal output. Tests determinism contract.
- Resonate: variance across N runs. Deterministic π*'s yield
distinct=1; expected_max_distinct=1 enforces zero-variance contract.
Surface:
- bench/batteries/b_5r.py (5 deterministic runners; no LLM-as-judge)
- bench/fixtures/5r/{react,rearrange,restore,replicate,resonate}-v1.jsonl
(30 each = 150 new fixtures)
- runner.py registers 5r in _BATTERIES + _DEFAULT_FIXTURES
- Makefile: bench-5r + bench-suite (5S+5T+5F+5R aggregate)
Final tally:
5S syntax/semantics/syllogism/synthesis/semiotics 108
5T transfer/transfer-learning/triangulation/... 154
5F function/finetuning/falsification/... 50
5R react/rearrange/restore/replicate/resonate 150
TOTAL: 462 fixtures across 21 sub-batteries — 100% pass.
Tests: 6 new in tests/test_bench_batteries.py + adjustment to
test_session_integration.py for the 312→462 count + 5R sub-battery
presence assertion. Full suite: 1192 passed, 36 skipped.
Closes#000021. Phase 3 (external-corpus expansion) remains open
under the ticket but does not gate closure — the complete
Dav1DPrometheus surface is now executable infrastructure.
Per fox's 2026-05-08 review of fbd99a8: implement adaptation_efficiency
and feedback_efficiency in run_finetuning + run_feedback_loop with
explicit sentinels — not Python floating-point accidents.
_efficiency(gain, cost) helper:
- cost > 0: standard ratio
- cost == 0, gain > 0: EFFICIENCY_INFINITE (free improvement)
- cost == 0, gain == 0: EFFICIENCY_UNDEFINED (= 0.0; no signal)
- cost == 0, gain < 0: -EFFICIENCY_INFINITE (free regression)
Battery-level metrics report mean_finite (computed over finite
values only) + infinite_count + neg_infinite_count so the mean
stays dimensionally truthful and consumers can pivot on the special
buckets separately.
Phase 1a cost proxies:
- run_finetuning: _capital_cost_delta sums resource_budget
(max_compute_ms_delta * 1e-3 + max_storage_delta_bytes / 1e6).
Phase 1b.2 will replace with real capital_ledger reads.
- run_feedback_loop: chain length = cost. Phase 1b.2 capital_ledger
integration replaces it.
Tests added (7):
- _efficiency over four boundary cases
- 5F finetuning + feedback_loop emit the new metrics keys
- Synthetic zero-cost finetuning fixture verifies +inf path
Full suite: 1110 passed, 36 skipped.
Closes the one actionable from the fbd99a8 review.
Per fox's "partial punt on larger ones" — ships the bench/ skeleton +
small seed fixture sets so future v8/v7-W/SelfModel work can cite a
real fitness target. Full Phase 1 (50-200 fixtures per sub-battery)
and Phases 2-3 stay open in the ticket.
Phase 1a delivers:
- bench/batteries/{base,b_5s,b_5t,runner}.py — Battery protocol,
BatteryResult, fixture-digest helpers, CLI runner.
- Seed fixtures:
- bench/fixtures/5s/syntax-v1.jsonl — 10 tasks against
wikitext-base@v1 and claim-lattice@v1
- bench/fixtures/5s/semantics-v1.jsonl — 8 equivalence tasks
- bench/fixtures/5t/transfer-v1.jsonl — 4 paraphrase-invariance
tasks
- Runners for 5S Syntax, 5S Semantics, 5T Transfer. Other 5S/5T
sub-batteries are stubs returning zero-task results.
- Makefile targets: bench-5s, bench-5t, bench-5s5t.
- runtime_digest field captures the active π* registry fingerprint
so a registry change surfaces in bench results.
Tests: tests/test_bench_batteries.py (17 cases). Full suite:
1076 passed, 36 skipped. `make bench-5s5t` runs end-to-end and
emits JSON results.
Ticket #000021 status: in progress · Phase 1a landed; Phase 1b/2/3
remain open.
