bench: land #000023 + #000024 + #000025 (Phase 1a/1b — Dav1DPrometheus suite)
Implements three coupled tickets in one push: complete the 5S battery,
align 5T to Dav1DPrometheus vocabulary + complete it, open 5F battery.
All runners deterministic, no LLM-as-judge anywhere.
#000023 — 5S Phase 1b (closed)
run_syllogism + run_synthesis + run_semiotics replace stubs.
30 deterministic fixtures each (90 total, all passing).
Carrier metadata mandatory; unsupported carriers fail explicitly.
Syllogism kernel handles categorical_transitivity, chain_3,
invalid_converse, missing_premise. Order-agnostic over premise
permutations.
Synthesis uses content-token subset check (stopwords removed) —
catches single-token entity swaps that lax overlap missed.
Semiotics validates synonym-swap invariance via π* canonicalize +
re-substitution. Hidden-channel work stays defensive only.
#000024 — 5T Phase 1b (closed)
Vocabulary aligned: Transfer→transfer-learning, Truth→truthtables,
Timing→time. Legacy transfer-v1.jsonl + run_transfer kept intact.
run_transfer_learning, run_triangulation, run_truthtables,
run_transitivity, run_time replace stubs.
30 fixtures each × 5 = 150 total (155 with legacy transfer).
Triangulation runs 4 strategies (substring, token_subset,
token_overlap, entity_match) and gates on agreement threshold.
Truthtables: deterministic propositional evaluator with
recursive-descent parser; supports AND/OR/NOT/XOR/IMPL/IFF;
capped at N=4 variables.
Transitivity: typed-relation whitelist
(implies, subset_of, ancestor_of, before, less_than). BFS path
walk; mixed/unknown relations fail by construction.
Time: synthetic memory-snapshot chains test preservation,
stale-marking, current_root tracking. Phase 1b.2 will read real
memory_records.
#000025 — 5F Phase 1a (in progress; Phase 1b expansion still open)
New bench/batteries/b_5f.py with five sub-batteries:
- Function (shape_match / pointer_set_match / threshold_on_metric)
- Finetuning (parent→child SelfModel improvement check)
- Falsification (planted-error detection rate, verifier_method_root pinned)
- Formulate (structural lattice match: claim count + sorted-approx
text + exact pointer-ID-set; not exact-string)
- Feedback Loop (operation/observation chain → expected_delta in
aggregated observation feed)
10 deterministic seed fixtures per sub-battery (50 total).
feedback_efficiency / adaptation_efficiency hooks stub the capital-
ledger integration for v8 fork-choice.
Cross-cutting:
- bench/batteries/base.py: PHASE_1_CARRIERS whitelist +
validate_carrier helper. Backward-compatible with Phase 1a
fixtures that lack carrier field (defaults to "text").
- runner.py registers all 16 sub-batteries in _DEFAULT_FIXTURES
so `--all` runs the entire Dav1DPrometheus suite.
- Makefile: bench-5s / bench-5t / bench-5f / bench-5s5t5f
targets. bench-5t-legacy preserves Phase 1a access.
- tests/test_bench_batteries.py: 29 tests (was 17). Phase 1a
digest stability checks, sub-battery smoke tests, carrier
rejection tests, transitivity-whitelist test, truthtables
N>4 cap test.
Bench summary across the full suite:
5s syntax/semantics/syllogism/synthesis/semiotics 108/108 pass
5t transfer/transfer-learning/triangulation/truthtables/transitivity/time 154/154 pass
5f function/finetuning/falsification/formulate/feedback-loop 50/50 pass
TOTAL: 312 fixtures across 16 sub-batteries — 100% pass.
Full test suite: 1103 passed, 36 skipped.
Source: Legally Unprecedented Dav1DPrometheus (BasementAGI host).
Honoring his framework. The complete state-space synthesis (SQD +
v7 + 5S/5T/5F) is now executable infrastructure, not metaphor.
russell@unturf.com
2af28e66af