dav1d's review (RESPONSE_1 + RESPONSE_2) returned 2026-05-11. This
lands the Tier-1 items — everything that doesn't change numeric
outputs or invalidate the KAT discipline. The Tier-2 B1 conservative-
envelope (v2 calculator) is a separate decision and stays a closure
blocker.
Calculator (bench/scripts/t3_bound_calculator.py):
- Recommendation wording: "M2's single-window guarantee is broken"
→ "this conservative bound CANNOT CERTIFY M2's residual". An upper
bound exceeding 256 bits means we cannot certify, NOT that the
adversary can steer 256 bits — the prior wording overclaimed.
- New structured output fields: b1_model ("effective_control_v1"),
certification_status ∈ {CERTIFIED_BY_BOUND, NOT_CERTIFIED_BY_BOUND},
certification_threshold_bits (256), model_assumptions[]. Callers
read a machine-readable status, not just prose.
- Input validation hardening: _require_finite_float / _require_positive_int
helpers reject bools (isinstance(True, int) is True in Python — a
real leak risk for a security calculator) and NaN / ±inf for every
numeric input and constant.
- gradient_fraction = 0 now accepted (no T2 surface; B1 = 0; T3's
LR + batch-order channels still contribute) — improves component
isolation. CLI help + module docstring updated accordingly.
- Numeric outputs UNCHANGED: baseline still 625.8716 / 292.4813 /
300.0 / 33.3904; b1_model stays effective_control_v1; KAT discipline
intact.
Tests (tests/test_t3_bound_calculator.py, 53 → 75):
- Hard-coded cwd="/home/fox/git/arborist" → pathlib.Path(__file__).
resolve().parents[1] so the suite runs on any checkout.
- New: test_gradient_fraction_zero_accepted, test_bool_rejected_for_int_fields,
test_bool_rejected_for_float_fields, test_nonfinite_numbers_rejected,
test_output_carries_b1_model_and_certification_fields,
test_certification_status_certified_below_threshold.
- test_recommendation_exceeds_sha256 now also asserts "CANNOT CERTIFY"
+ certification_status == NOT_CERTIFIED_BY_BOUND.
Doc (docs/soft-hash-channel-t3-bound.md):
- §0 reworked into a reviewer brief recording dav1d's findings
(§2 accepted, §4 accepted, §5 accepted as model-bound, §3 = closure
blocker, wording/validation = applied).
- New §3.1: the B1-double-g issue spelled out — effective_control_v1
vs fraction_channels vs aggregate_bias vs max_envelope, with the
baseline-spread table (292 / 1730 / 5850 / 5850 bits); v2 path
described.
- §5: "B3 is a model-bound, not a directly-quoted theorem" note.
- §10: items 1-2 are now the closure blockers (B1 envelope v2; active
KAT fixture); items 3-7 are tightening paths (#000043). New §10.1
records what the 2026-05-11 hardening pass already landed.
- §11: calculator-output example updated to show the new fields +
corrected recommendation wording.
- §12: references add the dav1d review + clarify Bottou-Bousquet
"inspires" (not "underlies") the §5 model-bound.
Status (#000036 ticket + TICKETS.md row): review-returned + Tier-1-
applied; closure blockers = B1 v2 envelope (awaits fox go/no-go) +
active KAT fixture. R2's architectural integrations (Merkle audit-
event commitment, SQD canonicalization, CTI clause-lattice, 5F
trigger, ForkScore security-risk) noted as out-of-scope (separate
tickets if wanted).
AUTOCOUNT markers in docs/calculator-test-patterns.md +
docs/warrant-substrate-cookbook.md bumped 53 → 75.
Full suite: 2264 passed, 28 skipped.
Phase 1a scores one (parent, child) fork at a time; Phase 1b is the
consensus paper. Neither persists multiple candidate branches at the
same checkpoint — and #000037 §12 Trigger 1 ("ForkScore regularly
receives ≥4 candidate branches per checkpoint") gates the multi-
branch path of the Prometheus-Σ controller on this data existing.
Phase 1c lands the missing seam.
Schema (arborist/store.py): _migrate_fork_score_branches creates the
sibling table with PK (branch_set_id, branch_id) + indexes on
branch_set_id and parent_root. Sibling — never enters
audit_events.event_hash preimage, so re-scoring or back-filling
cannot break the audit chain.
Helpers (arborist/substrate/fork_score.py): persist_branch_score
upserts one row via ON CONFLICT (branch_set_id, branch_id) DO UPDATE
so re-scoring the same fork under the same checkpoint is a clean
overwrite, not a duplicate. branch_set_density(conn, branch_set_id)
returns the count of distinct branches recorded under a checkpoint
— the function the #000037 §12 Trigger 1 probe reads.
ESTIMATOR_VERSION = "fork-score-v1" pins the producer generation on
every persisted row.
CLI (arborist/cli.py): arborist substrate score gains six new flags
(--branch-set, --branch-id, --parent-root, --child-root,
--persist-shard, --weights-id). Default off — --branch-set absent
preserves Phase 1a pure-function semantics for every existing
caller. When present, requires --parent-root and either --branch-id
or --child-root; missing inputs return exit code 2.
Tests (tests/test_fork_score.py, count 18 → 23): migration creates
the table + both indexes; persist writes one row carrying
parent/child roots + verdict + weights_id + estimator_version;
upsert on the PK refreshes child_root + weights_id + recorded_at
without duplicating; branch_set_density counts per-checkpoint and
ignores cross-set rows; breakdown_blob round-trips as canonical
JSON whose values sum to the persisted score.
Status sync: #000012 §7 Phase 1c flipped from "proposed, not yet
open" to "landed 2026-05-10" with the original proposal preserved
below as design log. TICKETS row 117 mirror-updated. AUTOCOUNT
counters in #000012 + cookbook bumped 18 → 23 plus the cookbook's
fork_score.py LOC row refreshed (298 → 386 module, 403 → 609
tests, density 1.35 → 1.58).
End-to-end smoke verified: arborist substrate score writes a
fork_score_branches row with the expected schema (verdict / weights_id
/ estimator_version) and the row survives a clean SQLite read.
Two coupled changes that wire the §13 Step 11 proposal stream from
in-memory-and-discarded to persisted-and-harvestable.
(A) emit_controller_events now writes a 4th event kind
controller_falsification_proposal, one row per
decision.falsification_proposals entry. Body carries branch_id +
witness_divergence + reason; label column carries the reason for
terminal-table inspection. Deterministic ordering by (branch_id,
witness_divergence) so canonical-body hashes are stable. Idempotent
under the existing UNIQUE (event_kind, body_hash) constraint. The
audit-chain semantics remain unchanged (still a sibling table; no
event_hash preimage entry). The CLI inspector's --kind choices gain
the new event kind so operators can filter for it directly.
(B) bench/scripts/harvest_falsification_proposals.py gains a third
source bucket CONTROLLER_PROPOSAL alongside the existing HYBRID +
UNGROUNDED providence_cache buckets. Reads
controller_falsification_proposal rows from controller_events,
extracts the 16-char cache_key prefix from branch_id (qa:<prefix>
pattern from the QA-runner advisory), joins back to providence_cache
for fixture enrichment (answer_text + audit_mode + verifier_method),
and tags _harvest_meta.harvested_from = "controller_events" so the
two source paths stay distinguishable in the fixture pack. Dedup
against the providence_cache buckets by fixture id.
Today this typically yields 0 new fixtures because (i) no live QA
has fired since Phase 2 wiring landed, and (ii) the QA-runner
single-branch advisory's proposals overlap providence_cache content
the harvester already finds. Real net value comes from Phase 3
(#000045) sweep emissions, which will produce multi-branch chunk
proposals that providence_cache rows can't predict.
Tests: 3 new in tests/test_prometheus_audit.py (proposal-row
emission, no-proposal no-row, idempotency); 1 new in
tests/test_bench_batteries.py (synthetic qa.db with both
providence_cache + controller_events rows; asserts the
controller_events bucket surfaces a fixture invisible to the
divergence-thresholded providence_cache buckets). The pre-existing
harvested-pack-runs-clean test relaxes its harvested_from pin from
"providence_cache"-only to {"providence_cache", "controller_events"}.
The Phase 2 advisory writes (_emit_qa_controller_advisory) populate
the controller_events sibling table on every QA cycle. Until now
the only way to inspect was raw SQL. This adds a top-level
arborist subcommand that walks every shard, surfaces decision /
difficulty / budget_allocation rows, and renders either a compact
terminal table or JSON.
Flags:
- --limit (default 20)
- --kind {controller_decision|controller_difficulty|controller_budget_allocation}
- --organism-prefix PREFIX (LIKE prefix; "qa:" matches QA-runner advisories)
- --since-seconds N (rows recorded within the last N seconds)
- --body (include JSON body_blob in --json output)
- --json (machine-readable {summary, rows})
Reads via sqlite3 read-only URI; silently skips shards without a
controller_events table. No writes, no schema migration triggered.
Wires into the #000045 Retrigger 1 measurement story (need ≥1000
advisory rows from Phase 2 wiring before Phase 3 implementation
opens) — operators now have a one-line check for that signal.
Tests: 5 new in tests/test_prometheus_audit.py — happy-path table
output, --kind filter, --organism-prefix filter, --json shape,
graceful skip of non-arborist sqlite files in the shards-dir.
test_cli_smoke parameterized list updated so the argparse-
construction smoke test also covers the new subcommand.
arborist.qa.runner.ask() now emits one controller_decision +
controller_difficulty + controller_budget_allocation triple per QA
cycle via _emit_qa_controller_advisory(conn, cache_key, verdict).
Single-branch synthesis from the verdict's audit_mode (Δ5F mapping
matching the dry-run simulator) and n_unverified/n_quotes
(witness_divergence). Wrapped in try/except so any advisory failure
never blocks the QA result; pure audit-only — does not enter
audit_events.event_hash preimage, audit chain semantics unchanged.
Lazy import keeps the QA hot path free of substrate-module load on
calls that never reach this helper (cache hits + early returns).
Tests: 3 new in tests/test_prometheus_audit.py — happy-path emits
all three event kinds, defensive parsing tolerates malformed verdict
without raising, second call with same (event_kind, body_hash) is
idempotent under the existing UNIQUE constraint.
Per §22 Finding 2: flat capital_cost=1.0 made every utility negative
in the dry-run sim. Split capital_cost into kernel_cost (~0.05 kernel
re-probe) + llm_cost (~1.0 LLM witness fan-out); ControllerBranch
now exposes effective_cost = kernel_cost + llm_cost when either is
positive, falling back to legacy capital_cost when both are zero.
Pre-1.c callers don't migrate. _utility() reads effective_cost so
split-cost and legacy-cost branches with the same total cost produce
byte-identical utility values.
Per §22 Finding 3 (partial): add sweep_weights() profile —
gamma_5f=1.5, lambda_capital_cost=0.25, nu_witness_divergence=0.5.
Sweep work willingly pays capital for falsification discovery and
treats high witness divergence as desirable signal (§13 step 11).
Registered in WEIGHT_PROFILES["sweep"]; folds into governance_policy
selection alongside safe / conservative / exploratory.
Tests: 6 new in tests/test_prometheus.py — effective_cost split path,
legacy fallback, _utility byte-identity across the two cost shapes,
kernel-only vs LLM-only ranking, sweep-vs-safe divergence on a
marginal branch (DEFERRED under safe, ACCEPT under sweep), and
WEIGHT_PROFILES registry now pins all four profiles.
Lock the AUTOCOUNT regression-test pattern as the design log
canonical record. Previously declined when surface was 1-metric
+ 29 tags; now mature enough (4 metrics + 58 tags + 1 same-day
drift-catch since landing) to formalize.
== Ticket content ==
10 sections covering:
1. Why this exists — the 4-drift-day baseline (6cbbf95 / 14bcb99 /
5c21e83 / 30a9488) that motivated mechanization. Five-step
walk through justifying each choice (Step 5 last).
2. Format — `<!--AUTOCOUNT:metric:path-->N<!--/AUTOCOUNT-->`.
3. Four supported metrics with examples + skip semantics:
`tests`, `fixture-rows`, `db-rows`, `db-where`.
4. Skip-on-absence — operator state (shards, qa.db) absence is a
logged skip, not a fail. Smoke verified 2026-05-10 with
HOME=/tmp/empty.
5. What NOT to tag — closed-ticket point-in-time snapshots,
aggregate floors ("2000+"), historical journey arcs.
6. Install discipline at write time + at refresh time.
7. Future metrics deferred (file-lines, gh-pr-comments-count,
module-loc, commit-hash-exists) with the "add a metric"
recipe.
8. Empirical baseline at landing (3 test functions, 58 active
tagged claims across 8 doc files, harness runtime 2-4s).
9. Scope boundaries — does NOT auto-rewrite, does NOT validate
prose quality, does NOT scan docstrings, does NOT lock
values, does NOT add deps.
10. References — every landing commit + sister doc.
Closed at landing (status quo since fc5ba50 2026-05-10 morning;
this ticket is retroactive design log per the convention "every
ticket flips to `closed · landed in commit <sha>` when the work
ships").
== Code-fence parser fix ==
Adding the ticket itself surfaced an oversight: my AUTOCOUNT
examples in §3.3 + §3.4 used literal tag pairs in ``` fenced
code blocks. The parser was reading them as live claims and
firing on the illustrative `db-rows:002.db:concept_relations`
claim (compared 1234 vs live 72576 — both meaningless because
it's an example).
Fix: `_strip_fenced_code_blocks` substitutes the body of every
triple-backtick block with newlines before regex scanning. Line
numbers stay aligned (newline-preserving substitution); tags
inside fences are skipped because their parent text no longer
matches the regex.
Both helper functions (`_iter_claims` and the well-formed-tags
test) walk through the stripped text, so the strip discipline
is consistent across all three test functions.
== TICKETS.md index ==
Added #000044 row marked closed with the 5-commit landing trail.
Bumped Next ID 000044 → 000045.
== Verification ==
$ pytest tests/test_doc_counts.py
3 passed in 2.80s
$ pytest tests/ -q
2337 passed, 37 skipped in 108.29s
Hygiene: fox's in-flight changes to arborist/qa/runner.py +
arborist/substrate/prometheus.py + tests/test_prometheus*.py
left untouched in working tree.
Fan-out follow-up: extends AUTOCOUNT with a new metric for filtered
SQL-row claims, then tags fox's prometheus controller test surfaces
shipped this evening under #000037.
== Task 3: db-where metric ==
New metric ``db-where`` for tagging single-column equality
predicates. Target syntax::
<table>?<column>=<value>
<shard>:<table>?<column>=<value>
Resolves to ``SELECT COUNT(*) FROM <table> WHERE <column> = ?``
with ``<value>`` bound as a SQL parameter (no string
interpolation), so author typos or stray content can't escape
the predicate. Column + table names validated as bare
identifiers before string-interpolating into the query template;
sqlite3 connection opens with ``mode=ro`` URI flag.
Same skip-on-absence semantics as ``db-rows``: missing DB or
table yields a logged skip note, not a test failure. Sentinel
returns reuse the same _DB_MISSING / _TABLE_MISSING / _DB_ERROR
constants.
Smoke verified::
_live_db_where('documents?source_type=claim_pack') → 92
_live_db_where('documents?source_type=wikipedia_xml') → 866782
_live_db_where('001.db:documents?source_type=wikipedia_xml')
→ 867695
_live_db_where('documents') # malformed (no ?) → -4
Tagged claims using the new metric (cookbook):
- L36 ``92 records total`` for the claim-pack source
- L349 ``The 92 chains have three quality tiers``
Both resolve to ``documents WHERE source_type='claim_pack'``
in shard ``000.db`` — the live count of claim-pack records.
L5 ``18/92 → 92/92`` historical narrative left untagged
(expressing a journey arc, not current state).
== Task 2: tag prometheus controller test surfaces ==
fox shipped two test files this evening under #000037 that
weren't previously inventoried in any reference doc:
- ``tests/test_prometheus.py`` — 36 tests covering Phase 1
controller (commits ``f625cac`` + ``f9f5ae4``). Verifier-style
discipline (NOT calculator pattern — it's a pure-function
state-machine controller with no closed-form math).
- ``tests/test_prometheus_audit.py`` — 14 tests covering Phase 2
``controller_events`` sibling table (commit ``a786d6d``). Pins
no-chain-mutation invariant (advisory writes never enter
audit_events.event_hash preimage).
Added two paragraphs to cookbook §"Substrate-paper-spec'd
primitives" describing the test discipline + algorithmic surfaces
each pins. Renamed section header from
"(#000012 + #000018 + #000034)" to
"(#000012 + #000018 + #000034 + #000037)" to keep the
ticket-set roster current.
Two new rows in the test/code-density table:
| substrate/prometheus.py | 893 | 804 (36 tests) | 0.90 |
| substrate/prometheus_audit.py | 200 | 388 (14 tests) | 1.94 |
All four counts AUTOCOUNT-tagged (2 inline prose + 2 table rows
= 4 new tags). prometheus_audit's ratio of 1.94 is high because
the test file pins a lot of write-path invariants for what is
nominally a small (200 LOC) sibling-table module — appropriate
for foundation-level audit-discipline code.
== Task 4: test_full_suite_total_fixture_count flake — investigated ==
Earlier today's transient ``-x`` flake (1 failed, 1799 passed)
did NOT reproduce in current tree state (2328 passed, 37 skipped
under same flags). Root-cause investigation:
- ``_DEFAULT_FIXTURES`` is read-only at module scope; no test
mutates it.
- The test reads JSONL fixtures from ``bench/fixtures/`` via
``_run_one``; those files weren't being written by parallel
tests.
Hypothesis (not confirmed, since flake didn't repro): transient
filesystem state during heavy-parallel-commit window
(``f625cac`` / ``a786d6d`` / ``6142437`` / ``f9f5ae4`` all
landed in succession around 17:23-17:24 EDT 2026-05-10 while my
test run was in flight). No structural defect identified. If
flake recurs, capture stdout + filesystem state at failure time
to confirm.
== Task 1: Walked fox's 6 evening commits via Explore agent ==
Agent reported what shipped under #000037 + #000012 evening
push (Prometheus-Σ Phases 0/1/2/dry-run + v8 consensus paper).
Findings used to drive task 2 above. Notable design choices
worth surfacing as reference:
- prometheus.py is **pure function** (no DB / LLM / scheduler);
returns advisory ``ControllerDecision`` + optional proposal
records, never mutations. Verifier-style test discipline.
- ``controller_events`` sibling table never enters
``audit_events.event_hash`` preimage — audit chain unaffected.
- Phase 3 sleep-sweep scheduler **deferred** in favor of
read-only dry-run simulator surfacing 5 design constraints
(chunk_size = Hermes concurrency NOT candidate pool;
capital_cost flat=1.0 needs split; τ_qa per audit_mode;
Target B = 4.4% canonical-shape match; quarantined-row veto
exercises end-to-end). Calibration substrate for eventual
scheduler.
- Per-branch controller latency 12.5 µs at chunk_size=4 → not
the bottleneck; Hermes witness fan-out is.
v8 consensus paper at ``docs/_source/merkle-agi-v8-consensus.rst``
(834 lines; 11 parts) closes the loop from single-validator
Proof-of-Upgrade to multi-validator BFT selection. Phase 1c
(branch-set persistence) remains proposed-not-opened.
== Coverage ==
Total tags after this commit: 54 (was 49; +5)
Tags by metric:
tests: 45 (+4 new prometheus + table rows)
fixture-rows: 2
db-rows: 3
db-where: 2 (new metric, both 92 claim-pack)
Files with tags:
docs/warrant-substrate-cookbook.md 32 (+5)
docs/calculator-test-patterns.md 8
docs/soft-hash-channel-analysis.md 5
docs/tickets/ticket-000006-bench-emergent... 4
docs/seven-point-program.md 3
docs/tickets/ticket-000035-prg-choice-phi-prg.md 2
== Verification ==
$ pytest tests/test_doc_counts.py -v
3 passed in 2.60s
$ pytest tests/ -q
2328 passed, 37 skipped in 106.06s
Closes the controller → 5F battery loop fox designed: #000037 Phase
1 produces FalsificationFixtureProposal records from high-divergence
providence_cache rows; this commit lands the harvester that turns
those proposals into a real 5F fixture pack the falsification
battery exercises every test run.
bench/scripts/harvest_falsification_proposals.py — reads qa.db,
filters live rows with witness_divergence = (n_unverified / n_quotes)
>= 0.5, stratifies by audit_mode, picks 20 HYBRID + 20 UNGROUNDED
top-by-cache_key for determinism, writes embedded-mode fixtures
to bench/fixtures/5f/falsification-harvested-v1.jsonl.
Each fixture carries `_harvest_meta` with the source cache_key,
divergence at harvest time, audit_mode at harvest time, and the
ticket reference (#000037 §13 step 11). Embedded mode — uses
`observed_violations` directly without calling verify_quotes
again; `answer_text` preserved verbatim for debugging.
Pack composition (initial harvest 2026-05-10):
20 UNGROUNDED (expected_reason: UNGROUNDED)
9 HYBRID_QUOTE (NEW motif — not in falsification-v1.jsonl)
7 HYBRID_CLAIM_LATTICE (NEW motif — not in falsification-v1.jsonl)
4 HYBRID_PARAPHRASE (already covered in v1)
Two harness tests pin the pack:
- test_5f_falsification_harvested_pack_runs_clean: 100% pass-rate,
every fixture carries traceable _harvest_meta.
- test_5f_falsification_harvested_pack_widens_motif_coverage: the
HYBRID_QUOTE / HYBRID_CLAIM_LATTICE motifs surface from real
corpus (loud-fail if harvest rotation drops them).
Makefile: `make bench-5f-harvest` re-runs the harvester. Parameters
exposed: HARVEST_QA_DB, HARVEST_OUT, HARVEST_THRESHOLD,
HARVEST_SAMPLE_PER_BUCKET.
Full suite: 2328 passed, 37 skipped (+2 from the two new pins).
Closes four gaps in the initial Phase 1 landing (commit f625cac):
1. §14 row 4 — Hermes-saturation guard. `hermes_utilization` was on
the input contract but never consumed. Now: `utilization >=
budget` → DEFERRED with HERMES_SATURATED note + advisory event
carrying (utilization, budget) for Phase 2 audit. Three new
tests (saturation-equal, saturation-overflow, headroom-exists).
2. §13 step 11 — falsification-fixture proposal emission.
New `FalsificationFixtureProposal` dataclass; controller now
emits one per branch whose `witness_divergence >=
falsification_divergence_threshold` (weight-tunable, default
0.5). Emission fires BEFORE the all-vetoed cascade so vetoed-
AND-diverged branches still surface as 5F-fixture candidates
per #000025. Four new tests (high-divergence emits, low-
divergence stays silent, threshold is weight-tunable, vetoed-
diverged emits anyway).
3. Entropy + memory gating moved from module-level constants
(`H_LOW=0.3`, `H_HIGH=0.7`, `KAPPA_MEMORY=0.5`) to weight
fields (`h_low`, `h_high`, `kappa_memory`). Constants stay as
back-compat exports; defaults match exactly so byte-identical
behavior when neither override fires. Three new tests
(h_low/h_high tunable, kappa_memory tunable, back-compat
match).
4. Veto-class cascade hardening. Added explicit tests for the
`replay_window_unbounded` → ESCALATE path and the
`soft_hash_signal` → QUARANTINE path (§6 + §14 documented but
previously untested). Plus tests for the §6 fail-loud
priority ordering: ESCALATE > QUARANTINE > REJECT when mixed.
Dry-run regenerated against ~/.arborist/shards corpus:
Target A at τ_qa=1d surfaces **576 FalsificationFixture
proposals** from witness_divergence >= 0.5 (24% of swept
candidates). These rows are now an actionable funnel for 5F
fixture mining under #000025 §3.
Module: arborist/substrate/prometheus.py (+117 LOC, 899 total)
Tests: tests/test_prometheus.py (+196 LOC, 22 → 36 tests)
Dryrun: bench/scripts/prometheus_sigma_sweep_dryrun.py +
bench/results/prometheus-sigma-sweep-dryrun-2026-05-10.md
track the new falsification_proposals_total counter.
Full suite: 2326 passed, 37 skipped (+14 net from the new tests).
Pure-function recursive-falsification controller implementing
ticket #000037 §13 algorithm (steps 1-9 + 12), §5 Shannon-entropy
fork-selection with stable softmax, §6 8-class hard-veto order,
§7 Kelly-bounded allocation with 4 safety guards, §7.1 EMA-smoothed
difficulty update, §14 exception-matrix dispatch, and §15 three
named weight profiles (safe/conservative/exploratory).
No DB, no LLM, no scheduler — advisory pure function over already-
committed state. Phase 2 (controller_events sibling table) lands
in companion commit a786d6d. Phase 3 (sleep sweep scheduler) is
not in this commit.
Module: arborist/substrate/prometheus.py (782 lines)
Tests: tests/test_prometheus.py (22 passing tests — the 17 named
contracts from §16.2 plus 5 boundary cases for the §6 veto-class
priority dispatch and §7.1 EMA stability).
Also removes obsolete tests/test_prometheus_sigma.py — pre-Phase-1
scaffolding placeholder whose 17 tests all called pytest.fail()
with "Phase 1 implementation pending" and the @skip_until_phase_1
decorator never auto-flipped to pass-on-import. The contract is
now in tests/test_prometheus.py.
Sibling table `controller_events` for advisory persistence of
ControllerDecision output from #000037 Phase 1. Same pattern as
`capital_ledger` — forward-migrated, indexed, but does NOT enter
audit_events.event_hash preimage. Audit chain semantics are
unaffected.
Three event kinds:
controller_decision — one per ControllerDecision
controller_difficulty — one per ControllerDecision (records
the difficulty_next value)
controller_budget_allocation — one per branch with nonzero
allocation in decision.allocations
Idempotent on (event_kind, body_hash) UNIQUE constraint.
sha256 of canonical-JSON-encoded body is the dedupe key.
Tests: tests/test_prometheus_audit.py — 14 cases covering
migration, idempotency, no-chain-mutation invariant, query paths.
Uses stub ControllerDecision so tests run independently of Phase
1's controller_decide implementation.
Fan-out follow-up to ``fc5ba50``. Two thrusts in one commit since
they exercise the same surface:
== Task 3: extend AUTOCOUNT with db-rows metric ==
New metric ``db-rows`` for tagging live SQLite row counts (alias
tables, claim-pack records, etc — operator state that drifted on
``30a9488`` and earlier). Target syntax::
<!--AUTOCOUNT:db-rows:citation_aliases-->74<!--/AUTOCOUNT-->
<!--AUTOCOUNT:db-rows:002.db:concept_relations-->1234<!--/AUTOCOUNT-->
Default shard: ``~/.arborist/shards/000.db`` (where the alias
tables live per ``arborist.cli._aliases_db_path``). Operator state
is graceful-skip semantics: when DB or table is absent (CI, fresh
checkout, sibling repo), the claim is logged as skipped and the
test still passes. Drift only fires when the DB IS present and
the count diverged.
Sentinel returns:
- ``_DB_MISSING`` (-2): shards dir not present → skip
- ``_TABLE_MISSING`` (-3): DB present but table absent → skip
- ``_DB_ERROR`` (-4): malformed table name or sqlite error → skip
Table name validated against ``[A-Za-z_][A-Za-z0-9_]*`` regex
before string-interpolating into ``SELECT COUNT(*) FROM <table>``;
this is belt-and-suspenders since AUTOCOUNT tags are author-
controlled, but the dynamic SQL surface deserves a bouncer.
Smoke verified under HOME redirect to ``/tmp/<empty>``: 3 db-rows
claims gracefully skip with informative line-numbered messages,
suite still passes.
== Task 2: backfill 15 tags ==
Cookbook test/code-density table (lines 569-579, 10 rows) — every
``(N tests)`` cell now machine-checked:
| aliases.py | 512 | 469 (28 tests) | 0.92 |
→
| aliases.py | 512 | 469 (<!--AUTOCOUNT:tests:tests/test_aliases.py-->28<!--/AUTOCOUNT--> tests) | 0.92 |
Markdown renderers strip HTML comments — table cells display
``28 tests`` unchanged. The ``warrant_resolver.py`` row stays
untagged because its test count is split across two test files
(verifier + parser) and the cell encodes a combined "~430"
instead of one collected count.
Cookbook alias-count surfaces (3 db-rows tags):
- L364 ``citation_aliases (74 rows live as of 2026-05-10)``
- L437 ``#000041 — citation-aliases table + 74 live rows``
- L438 ``#000042 — term-aliases table + 13 live rows``
Ticket #000035 (in progress, line 274) — refresh ``20 tests``
→ ``27 tests`` for ``test_anchor_prg.py`` + tag. Same drift
pattern as ``5c21e83``: ticket prose was written before the
``de997f7`` 2026-05-10 pattern backfill that added 7 tests
(prefix-extension closure, hand-formula, parametrized
invalid-input cones). Also tagged ``L279``'s 10-vector KAT
fixture claim with ``fixture-rows`` metric.
== Closed-ticket counts deliberately not tagged ==
#000028, #000030, #000042, #000031, #000004, #000026, #000009,
#000032, #000008 all carry historical "N tests pass" snapshots
from their landing date. Those are point-in-time records, not
live claims — drifting from current state is BY DESIGN. Tagging
them would fire the test on every successive change to the
codebase. Closed tickets are the design log; we don't backfill
them.
== Coverage summary ==
Total tags after this commit: 44 (was 29; +15)
Tags by metric:
tests: 39
fixture-rows: 2
db-rows: 3
Files with tags:
docs/warrant-substrate-cookbook.md 27 (was 14)
docs/soft-hash-channel-analysis.md 5
docs/tickets/ticket-000006-bench-emergent... 4
docs/seven-point-program.md 3
docs/calculator-test-patterns.md 3
docs/tickets/ticket-000035-prg-choice-phi-prg.md 2 (new)
== Verification ==
$ .venv/bin/pytest tests/test_doc_counts.py -v
3 passed in 4.32s
$ .venv/bin/pytest -q
2276 passed, 54 skipped in 168.34s
$ HOME=/tmp/empty pytest tests/test_doc_counts.py -v -s
3 db-rows AUTOCOUNT claim(s) skipped:
docs/warrant-substrate-cookbook.md:364 db-rows:citation_aliases skipped — /tmp/empty/.arborist/shards not present (CI / fresh checkout)
docs/warrant-substrate-cookbook.md:437 db-rows:citation_aliases skipped — /tmp/empty/.arborist/shards not present (CI / fresh checkout)
docs/warrant-substrate-cookbook.md:438 db-rows:term_aliases skipped — /tmp/empty/.arborist/shards not present (CI / fresh checkout)
3 passed in 4.78s
No new dependencies. No schema changes.
The doc-drift pattern recurred four times today on 2026-05-10
(commits 6cbbf95, 14bcb99, 5c21e83, 30a9488). Each fix was the
same shape: walk a doc, find a count that drifted from live truth
during the hours after the doc was written, refresh it. Cost: ~5
min per drift × 4 = 20 min of manual catching, with no guarantee
the next drift gets caught before someone external reads it.
Per fox's selection: regression test that makes drift loud at
test time instead of relying on visual catching.
== Mechanism ==
`tests/test_doc_counts.py` scans `docs/**/*.md` for AUTOCOUNT
tags of the form:
<!--AUTOCOUNT:metric:path-->N<!--/AUTOCOUNT-->
Two metrics supported:
- `tests` — pytest collected count for path. Batches every
tagged path into one `pytest --collect-only` subprocess
(~0.5s total).
- `fixture-rows` — non-blank-non-comment line count in a JSONL
fixture.
GitHub and most markdown renderers strip HTML comments, so
readers see only `N`. The tags are invisible in rendered output
but make the claim machine-checkable. Three tests in the file:
1. `test_doc_autocount_claims_match_live` — the core invariant
2. `test_autocount_tags_are_well_formed` — open/close balance
3. `test_autocount_metric_names_are_documented` — fail-closed on
undocumented metrics (catches typos)
Failure message names the doc file, line number, and the
claimed-vs-live diff. Example:
`docs/foo.md:42 AUTOCOUNT(tests:tests/test_x.py) claims 23, live is 27`
== 29 tags installed across 5 docs ==
While installing tags I had to read the surrounding prose, which
surfaced six stale counts that had drifted same-day:
`docs/soft-hash-channel-analysis.md`:
- L392 14 → 23 tests for phi_alignment_probe
- L417 20 → 27 tests for anchor_prg
- L463 14 → 23 tests for phi_alignment_probe (status section)
`docs/seven-point-program.md`:
- L77 68 → 58 tests for metacognition (drift -10; the file
shed tests during a refactor and the doc didn't catch up)
- L78 9 tests for `test_dag.py::test_preflight_*` — removed
count entirely; pytest selector subsets aren't currently
supported by the AUTOCOUNT metric set (would need a
`tests-matching` metric; not worth the surface for one claim).
- L110 24 → 33 tests for test_dag.py
`docs/calculator-test-patterns.md`:
- L35 33 → 23 tests for warrant_resolver
- L35 10 → 9 tests for warrant_chain
- L16, L265 51 → 53 tests for t3_bound_calculator (kept
initial-shipment provenance in prose)
== Coverage installed ==
calculator-test-patterns.md 3 tagged claims
soft-hash-channel-analysis.md 5 tagged claims
warrant-substrate-cookbook.md 14 tagged claims
seven-point-program.md 3 tagged claims
tickets/ticket-000006-bench-... 4 tagged claims
---
29 tagged claims
Every count that drifted today is now tagged. Future drift
fires the regression test at the next pytest run instead of
waiting for human catching.
== Discipline pattern ==
Walk this pattern for any new doc that names a count:
1. Surround the number with the tag pair:
`<!--AUTOCOUNT:tests:tests/test_foo.py-->N<!--/AUTOCOUNT-->`
2. Run `pytest tests/test_doc_counts.py` (~3.5s)
3. If it passes, the claim is now machine-verified
Aim to tag counts on first authorship. Retrofitting is cheap
but only catches drift after the fact.
== Out of scope ==
Test counts inside source code (docstrings, CLI --help) are not
scanned — would expand the test surface significantly and the
drift pattern hasn't manifested there. Add `**/*.py` scope when
that pattern surfaces.
Alias-row counts and claim-pack-record counts could be tagged
with new `db-rows:<table>` and `db-where:<sql>` metrics; deferred
until the next drift on those numbers (none caught today after
30a9488's cookbook refresh).
== Verification ==
$ .venv/bin/pytest tests/test_doc_counts.py -v
3 passed in 3.89s
$ .venv/bin/pytest -q
2276 passed, 54 skipped in 153.21s
No new dependencies. No schema changes. No source-code changes.
Pre-Phase-1e: falsification-v1.jsonl covered 10 motif tags across 50
fixtures (the high-traffic warrant/title/anchor/format set). Phase
1e adds 12 fixtures (5f-fal-051..062) for the previously-uncovered
motifs from the verifier+soft-demote registries:
CITATION_MISMATCH DEFLECTION_DETECTED
MANUAL_QUOTE_VIOLATION SCHEMA_INVALID
SOURCE_ROLE_BLOCKED SUBJECT_TOKENS_ABSENT
TOO_MANY_EVIDENCE_IDS UNKNOWN_EVIDENCE_ID
BROAD_QUANTIFIER_RUNAWAY BROAD_QUANTIFIER_CAP_APPLIED
BROAD_QUANTIFIER_SCOPE_UNBOUND BROAD_QUANTIFIER_REJECTED
Coverage now: 22 unique motif tags across 62 fixtures.
Harness changes:
- test_bench_batteries.py: bump pass_count assertion 50 → 62 in both
falsification tests; add test_5f_falsification_covers_every_documented_motif
that pins the motif set against the verifier+soft-demote registries
so adding a new violation upstream surfaces here as a missing
fixture (loud signal, no silent drift).
- test_session_integration.py: bump full-suite total 662 → 674.
Closes#000025 §10.12 (every documented failure-motif tag).
Still open in Phase 1b: §10.11 (real shard finetuning chains),
§10.13 (Feedback Loop latency/efficiency against real workload),
§10.14 (threshold handoff to #000012).
Backfills zero dedicated coverage on arborist/pi_star/logic.py
(371 lines, 2026-05-10 zero-coverage sweep). KATs mirror the
docstring's equivalence classes: commutativity, associativity,
IMPL/IFF/XOR rewrites, De Morgan, double negation, distribution,
idempotence, within-clause tautology drop. Pins v1's documented
limitation: A AND NOT A is NOT collapsed (multi-clause
contradiction detection is out of scope; only empty clauses
surface as FALSE). Negative cones: >8 atoms, unrecognized token,
unexpected char, unbalanced paren, unconsumed tokens, dangling
operator, empty, non-bytes. Plus determinism + lexical-sort
canonicalization.
Three threads bundled, all surfaced by today's calculator-test-
patterns.md audit + fox's directive to remove v-prefix from test
filenames:
THREAD 1 — rename test_v8_fork_score.py → test_substrate_fork_score.py
=====================================================================
Single test file in the tree had a v-prefix in its filename:
``tests/test_v8_fork_score.py``. Renamed via ``git mv`` for
consistency with yesterday's substrate refactor (the package is
``arborist/substrate/fork_score.py``; the CLI subcommand is
``arborist substrate score``; the test file should match).
No internal code changes needed — the file's imports + assertions
were already updated to ``arborist.substrate.*`` paths in
yesterday's bae5caf commit. Pure rename.
THREAD 2 — close v1 substring discipline gap
=============================================
Audit of test_anchor_prg.py + test_phi_alignment_probe.py against
docs/calculator-test-patterns.md §2 (versioned-default discipline)
found one gap: both files asserted the version string's exact
value but neither asserted the ``"v1"`` substring discipline that
fox's test_returns_calculator_version_token established.
Added ``assert "v1" in PHI_PRG_VERSION`` to
test_module_exports_version_string in test_anchor_prg.py.
Added ``assert "v1" in PROBE_VERSION`` to
test_module_exports_thresholds_and_version in
test_phi_alignment_probe.py.
Both follow fox's pattern: when the algorithm changes (v2-blake3-
expansion, v2-arnoldi-iteration, etc.), the version string MUST
change too. The "v1" substring assertion catches a future
contributor who refactors without bumping the version constant.
THREAD 3 — close CLI subprocess gap on test_substrate_fork_score.py
====================================================================
The renamed file had four CLI tests but all in-process via
build_parser() + parse_args() + func(args). That catches argparse-
shape drift but NOT entry-point / module-loading / sys.argv drift.
Added test_cli_substrate_score_subprocess_invocation: real
``subprocess.run(["python", "-m", "arborist.cli", "substrate",
"score", "--parent", ..., "--child", ..., "--out", ...])`` against
synthetic bench results. Asserts exit 0 + the --out artifact is
written + JSON-parses with valid verdict.
Pattern matches fox's test_cli_baseline_runs_clean in
test_t3_bound_calculator.py + the 581ad90 KAT-fixture-gap closure.
Same hazard fox already hit three times during the substrate
rename refactor (85be5eb, 209d670, b320e27): import-only tests
silently miss CLI surface drift.
CHECKLIST AUDIT — POST-FIX
==========================
Three calculator-style test files now all 9-item complete:
| t3 | anchor_prg | phi_alignment | substrate_fork |
KAT fixture | ✓ | ✓ | ✓ | n/a (different)|
VERSION + "v1" | ✓ | ✓ NOW | ✓ NOW | ✓ |
Hand-formula | ✓ | ✓ | ✓ | ✓ (synthetic) |
Monotonicity | ✓ | ✓ | ✓ | ✓ |
Closure / sum-of-parts | ✓ | ✓ | ✓ | ✓ |
Parametrized invalid | ✓ | ✓ | ✓ | ~ |
CLI subprocess | ✓ | n/a | n/a | ✓ NOW |
Doc parity | ✓ | KAT | KAT | KAT |
Module-export shape | ✓ | ✓ | ✓ | ✓ |
All four files now consistently track the calculator-test-patterns
checklist. test_substrate_fork_score.py is structurally different
(verifier-adjacent: tests scoring + verdict-band logic, not
closed-form math) so some checklist items map differently — KAT
fixture replaced by synthetic-input verdict tests (closer to
verifier-style), parametrized-invalid is partial (per-verdict-
class assertions rather than per-bad-input cone). Acceptable.
Test counts:
- test_substrate_fork_score.py: 26 → 27 (+1 subprocess test)
- test_anchor_prg.py: 27 → 27 (assertion added inline)
- test_phi_alignment_probe.py: 23 → 23 (assertion added inline)
- t3 file untouched in this commit (581ad90 already at 53)
Full suite: 1985 → 1986 (+1 from this commit's only
new-test-function addition; the inline assertions don't count
as new tests).
Hygiene
=======
- make test → 1986 passed, 45 skipped.
- All four calculator-style test files structurally aligned.
- No v-prefixed test filenames remain in tests/ tree.
Audited fox's exemplar 51-test file against the 9-item checklist
in docs/calculator-test-patterns.md (codified earlier today in
0725eb4). Two gaps found, both additive:
GAP 1: no KAT fixture
=====================
Checklist item 1: "KAT fixture under bench/fixtures/<module>/
(≥ 5 cases)". Fox's tests cover the doc's worked-example numbers
inline (test_baseline_matches_section_11_doc) but no separate
fixture file existed for off-the-baseline regression coverage.
Generated bench/fixtures/t3-bound/known-answer-tests.jsonl with
8 KATs:
- small-deployment-§7.1, medium-deployment-§7.2,
hardened-deployment-§7.3 (the doc's three worked examples)
- extreme-low-g (g=0.001, σ=0.5 — exercises low-SNR regime)
- tight-window-W=100 (small-W ceiling-rounding edge case)
- tightened-c-b1 (override-constants path)
- all-constants-tight (all three c_b1/c_b2/c_b3 overridden)
- b3-floor-regime (factor < 1, B3 floors to 0)
Each entry pins (calculator_version, inputs, expected_total,
expected_b1, expected_b2, expected_b3, expected_snr_grad).
Algorithm change MUST bump CALCULATOR_VERSION + emit new fixture
file under bench/fixtures/t3-bound/ — old runs replay against
old data per §1 discipline.
GAP 2: no B3 hand-computed formula test
=======================================
Checklist item 3: "Hand-computed formula tests — at least one
per independent contribution / output field". Fox had
test_b1_exact_formula + test_b2_exact_formula + test_snr_grad_formula
covering three of the five output fields. B3 had only
test_b3_floor_at_zero_when_factor_below_one (an edge case),
not a closed-form check on the general formula.
Added test_b3_exact_formula: hand-computes
``C_B3 · ⌈W/E⌉ · log₂(N_b · σ_grad / ‖∇L_max‖) / 2`` per #000036
§5 (Bottou-Bousquet refinement), asserts agreement with the
function's B3_contribution. Pairs cleanly with the B1/B2 hand-
formula tests fox had.
CHECKLIST AUDIT — POST-FIX
==========================
1. KAT fixture ✓ NOW (was ❌; 8 entries)
2. VERSION + "v1" ✓ test_returns_calculator_version_token
3. Hand-formula ✓ NOW B1/B2/B3/snr_grad all covered
(was ⚠️ partial; B3 had floor-only)
4. Monotonicity ✓ test_monotone_in_window_length /
gradient_fraction
5. Closure ✓ test_total_equals_sum_of_three_contributions
6. Parametrized invalid ✓ four @pytest.mark.parametrize blocks
7. CLI subprocess ✓ test_cli_baseline_runs_clean +
test_cli_invalid_input_exits_2
8. Doc parity ✓ test_baseline_matches_section_11_doc
(caught today's §11 calibration drift)
9. Module-export shape ✓ test_returns_calculator_version_token +
test_constants_recorded
All nine items now ✓. test_t3_bound_calculator.py is the
exemplar for calculator-style test discipline.
Test count: 51 → 53 (+2 from this commit). Full suite:
1915 → 1985 (+70 from fox's parallel work + this commit's +2;
partial cycle effects).
Hygiene
=======
- make test → 1985 passed, 45 skipped.
- KAT fixture is JSONL with header comment naming
CALCULATOR_VERSION; future drift caught at the test level.
- Eat-my-own-dogfood: applied my docs/calculator-test-patterns.md
checklist to fox's exemplar test file. The fact that gaps
surfaced (even on fox's substantive 51-test surface) validates
that the checklist has real reviewer value, not just guideline
signaling.
Catches the regression class where a PR breaks argparse setup
(duplicate flag, broken set_defaults reference, renamed leaf
verb) without breaking any existing test. ``--help`` exercises
the parser-construction path without invoking any handler.
Coverage:
- top-level: arborist --help / --version / no-subcommand error
- 30 top-level subcommand --help calls (parametrized)
- 38 nested-subcommand --help calls across 9 verbs that have
their own subcommand groups (alias / memory / capital /
selfmodel / mesh / crawler / canon / substrate / snapshot)
- completeness check: top-level --help mentions every verb in
_TOP_LEVEL_SUBCOMMANDS (regression guard against silent
removal)
All 68 pass. ~20s wall via subprocess (~150ms per --help call x 68
calls + Python startup overhead). Fixture is hand-curated so a
new subcommand requires fixture update — that's the discipline
which surfaces the regression as a test failure rather than as a
missed --help test.
Pattern after fox: contract phrases pinned as test invariants
(here, the subcommand verb names) so silent renames fire the
test loud + force the fixture update + force the docstring +
help-text update too.
CLI version-bake sweep surfaced one real defect: the Phase 1 test
scaffolding I shipped in c422216 imported ``from arborist.v9
import prometheus``, baking in the version-prefixed namespace path
that yesterday's substrate refactor (654d923) abolished.
The skip mechanism (try/except ImportError → CONTROLLER_AVAILABLE
= False) was masking the issue: when fox lands Phase 1 of #000037
at ``arborist/substrate/prometheus.py`` (per the topic-named
convention), my tests would CONTINUE to skip with
"controller module arborist/v9/prometheus.py absent" because the
import target itself is wrong. The skip-stub becomes permanent
dormancy instead of activating when the module ships.
Two changes:
- Import line: ``from arborist.v9 import prometheus`` →
``from arborist.substrate import prometheus``.
- Skip-reason text + module-docstring: ``arborist/v9/prometheus.py``
→ ``arborist/substrate/prometheus.py``, with a parenthetical
noting the post-2026-05-10 topic-named convention and that
§13's original sketch predated the v-dir retirement.
The fix is a real one — when fox's Phase 1 of #000037 lands, my
17 skip-stubs now activate against the correct module path.
Without this fix, they'd silently stay dormant.
Sweep summary
=============
Walked every CLI subcommand --help (arborist top-level + nested
substrate / memory / capital / selfmodel / warrant-resolve / sweep
/ alias / mesh / crawl / providence) plus full-tree grep for
``arborist v[789]`` / ``arborist\\.v[789]`` / ``arborist/v[789]``.
Remaining v-prefix mentions across the tree (all intentional):
- arborist/cli.py:5166-5169 — historical-note comment for the
v8→substrate rename
- arborist/substrate/__init__.py:8 — same convention-explanation note
- arborist/substrate/anchor_prg.py:59 — bytestring inside SHA-256
derivation of placeholder seed; can't change without breaking KAT
- bench/fixtures/phi-prg/known-answer-tests.jsonl:1 — fixture header
naming the v7 paper section §9.10 (paper version, correct)
- docs/v8-fork-score.md:4 — historical-note ("module moved from")
- docs/tickets/ticket-000019, ticket-000013 — "arborist v9.8" schema
references (schema version, correct)
- docs/_source/merkle-agi-v7w-spatial-temporal.rst — v7 paper +
v9.8 schema references (both correct)
Tests: 1915 passing, 45 skipped (was 1872; +43 from fox's parallel
test additions during this commit's prep + the 17 prometheus
skips activating against the correct path stub).
CLI version-bake sweep complete. The substrate refactor is now
surface-clean end-to-end.
Same pattern-application as de997f7 did for test_anchor_prg.py.
The phi_alignment_probe tests landed in 1dfb8b9 with KAT
regression + verdict-bucket coverage + Lanczos convergence
check, but lacked the four patterns fox demonstrated in
test_t3_bound_calculator.py (51 cases for the T3 calculator):
monotonicity, hand-computed formula, closure invariants, and
parametrized invalid-input cones.
New tests added:
1. **test_monotone_alignment_strength_in_concentration** —
tighter W concentration on low-λ subspace must monotonically
increase the ratio. Tested across 32-row → 16-row → 8-row
concentrations, normalized to constant ‖W‖_F. Catches sign
errors + drops in the alignment-score formula.
2. **test_monotone_alignment_in_dim_h** — verdict invariant
under W column-count scaling. Sanity check that A(W, H)'s
‖W‖_F² normalization decouples it from sample count.
3. **test_uniform_baseline_matches_analytical_formula** —
hand-computes the isotropic baseline
``E[A_bot(W_uniform, H)] = (1/dim_d) Σ 1/(λ_j+ε)`` from
#000034 §2.1 derivation; asserts exact agreement with the
function's ``a_uniform`` field. Catches algorithm drift the
KAT regression would miss (KAT could regenerate against a
buggy implementation).
4. **test_full_spectrum_a_top_plus_a_bot_covers_full_isotropic_baseline**
— closure invariant: when k_top + k_bot = dim_d, the function's
a_top + a_bot must equal full-spectrum A computed via dense
numpy.linalg.eigh decomposition. Catches missing terms /
double-counting.
5. **test_eigenvalue_ordering_top_dominates_bot** — closure
invariant: top-k eigenvalues must all be ≥ bot-k eigenvalues.
Catches a bug where eigsh's 'LA'/'SA' modes returned
overlapping ranges on near-degenerate spectra.
6. **test_rejects_wrong_dim_w** — parametrized over (1-D, 3-D,
0-D scalar) shape errors. Same pattern as test_anchor_prg's
parametrized rejects.
7. **test_rejects_non_positive_epsilon** — parametrized over
(0, -1e-6, -1.0). Collapsed N separate test_rejects_*
functions into a single parametrized cone.
Test count: was 14 in test_phi_alignment_probe.py; now 23
(+9 from the new patterns + parametrize expansion).
Full suite: 1727 → 1872 (note: large jump partly from fox's
parallel test additions today, +136 since my last test count
checkpoint; my contribution here is +9 directly attributable
to this commit).
Three calculator/probe-style modules now have consistent
test coverage:
bench/scripts/t3_bound_calculator.py — 51 tests (fox)
bench/scripts/phi_alignment_probe.py — 23 tests (this commit)
arborist/substrate/anchor_prg.py — 27 tests (de997f7)
Same pattern bench applied across all three. Future
calculator-style code should pin: monotonicity in each input
axis + hand-computed formula assertions + closure / sum-of-parts
invariants + parametrized invalid-input cones.
Hygiene
=======
- make test → 1872 passed, 45 skipped
- make chain-check-shards → 0 across all 7 shards
- All new tests use synthetic inputs (no LLM, no shard
dependency); run in ~35s suite-wide
arborist/qa/prompts.py — 4 string constants that ARE the contract
with the LLM. Silent edits drop bench STRICT-rate by tens of
points; per CLAUDE.md "bench-maxing" discipline, prompt
regressions need a per-PR guard, not just bench surfacing after
the fact.
This test file pins the load-bearing phrases as regression
guards for both claim_lattice modes:
CLAIM_LATTICE_SYSTEM_PROMPT (pointer mode)
- non-empty + substantial
- both worked examples present (Apple founders + Mars descriptive)
- two-pointer cap rule (folds into claim_lattice_max_pointers_per_claim)
- pointer ID shape teaching (E1, E2, E3, [E#] / [E#,E#])
- one-claim-per-line rule (parser splits on newlines)
- synthetic-elision-by-construction-impossible: NO instruction
to wrap claims in double quotes (that's legacy quote-mode)
CLAIM_LATTICE_GROUNDING_REMINDER (pointer mode user-turn)
- "REMINDER" prefix; "pointer-line" format restated
- two-pointer cap restated
CLAIM_LATTICE_JSON_SYSTEM_PROMPT (JSON mode)
- schema shape: claims/text/evidence_ids
- first-char-`{` / last-char-`}` discipline
- two-evidence-id cap
CLAIM_LATTICE_JSON_GROUNDING_REMINDER (JSON mode user-turn)
- schema restated; cap restated
Cross-prompt parity
- both modes reference the two-pointer cap
- both reminders end with "next message" (handoff to question)
- all 4 constants importable + non-empty strings
Full suite: 1872 passed, 45 skipped.
arborist/pi_star/protocol.py + arborist/pi_star/registry.py are
the foundation every concrete π* kernel rides on. Both shipped
in #000015 Phase 1 with zero direct tests; concrete kernels
(arithmetic@v1, logic-kernel@v1, algebra-symbolic@v1, ...) have
their own test files but the protocol contract + registry
mutation discipline weren't pinned.
Coverage:
protocol.py
- PiStar @runtime_checkable: instances satisfying the duck-type
pass isinstance check; missing-method instances are rejected
- registry_key returns "name@version" exactly; distinct versions
yield distinct keys
- equivalence_class_id is sha256 over canonicalize() output;
invariant under pre-canonical form (two raws that canonicalize
to the same bytes get the same eclass id); distinguishes
different canonicals
- assert_round_trip passes on idempotent π*; raises
AssertionError naming the kernel on non-idempotent;
propagates PiStarError when canonicalize raises on the test
input itself
registry.py
- register inserts; get retrieves
- register IS idempotent for the same instance at the same key
(no error)
- register REJECTS a different instance at the same key
(cache_key invariant: name@version content-pinned)
- same name with different versions coexist
- get raises KeyError on unknown key
- list_keys returns sorted keys (deterministic for cache_key
derivation downstream)
- domains() groups by domain; per-domain key lists are sorted;
empty registry → empty dict
isolated_registry fixture monkeypatches REGISTRY to {} for
mutation tests so the global registry stays untouched (matches
the module docstring's no-public-unregister discipline).
Substrate-paper-spec'd primitives (arborist/substrate/* and
arborist/pi_star/protocol.py + registry.py) all directly tested
now.
Two related cleanups in one commit, both surfaced by reading fox's
test_t3_bound_calculator.py (51 tests for my T3 calculator):
1. Refresh stale §7 numbers in the T3 bound doc
=================================================
fox's test_baseline_matches_section_11_doc docstring (lines
56-61) flagged that my §7.1 worked example said
622.7 / 290.0 / 32.7 bits but the calculator's actual
closed-form output is 625.87 / 292.48 / 33.39. Same drift in
§7.2 (3358 → 3387.72) and §7.3 (247 → 247.14).
The numbers were rounded estimates from when I drafted the doc
before the calculator existed. Refreshed all three §7 numeric
examples to match the calculator's actual output (verified live
via t3_bound_bits()). §3 inline approximation likewise updated
(290 → 292.48). Added a short note pointing readers at the
calculator + tests as the source of truth.
2. Backfill anchor_prg tests with fox's patterns
=================================================
fox's test_t3_bound_calculator.py demonstrated four patterns I'd
missed in my #000035 phi_prg tests:
- **Output prefix invariant** (closure check): phi_prg(h, n+k)[:n]
≡ phi_prg(h, n). Streaming-counter invariant — would catch a
bug where a per-call seed mutation broke determinism across
dim_h values.
- **Output length monotonicity**: len(phi_prg(h, n)) == n exactly.
Parametrized over n ∈ {1, 2, 4, 7, 16, 17, 64, 1024}. Catches
off-by-one in `_expand` truncation.
- **Hand-computed first block**: assert that the first 64 bytes
of output equal a direct ``hmac.new(seed, h + b'\\x00\\x00
\\x00\\x00', sha512).digest()``. Pattern from fox's
test_b1_exact_formula — don't rely on KAT regression alone;
compute the first-principles math in the test file. Catches
algorithm drift the KAT (regenerated against a buggy version)
would miss.
- **Seed-bleed check**: changing the seed must change EVERY output
position. Probability of false-positive ≈ 64 · 2^-32 ≈ 2^-26;
none expected in practice.
- **Parametrized invalid-input tests**: collapsed N separate
``test_rejects_*`` functions into ``@pytest.mark.parametrize``
cones (4 wrong-size-hash cases + 3 non-positive-dim_h cases).
Same coverage, fewer test functions.
Test count: was 20 in test_anchor_prg.py; now 27 (+7 from
parametrize expansion + new patterns). Full suite: 1720 → 1727.
Hygiene
=======
- make test → 1727 passed, 45 skipped.
- make chain-check-shards → 0 across all 7 shards.
- All new tests use ``pytest.importorskip`` already at module top
(anchor_prg has no extras gate; tests run unconditionally).
Lessons captured
================
The patterns to remember for future calculator/probe-style code:
1. KAT regression alone isn't enough. Add hand-computed
formula tests so the math itself is asserted in the test
file, not just "consistent with a recorded snapshot".
2. Test monotonicity / closure invariants. They catch
algorithm drift, sign errors, missing terms.
3. Parametrize invalid-input tests. One function, N cases.
4. Test the doc's numbers against the function. Catches
calibration drift in the doc itself (this commit's
finding about §7).
5. CLI subprocess tests for end-to-end. Argparse + main()
drift the import-only tests miss.
The aliases.py public surface had 18 tests covering happy-paths,
audit discipline, domain isolation, lowercase normalization, and
expand-query semantics. Three direct gaps:
- list_term_aliases (no test at all): filter-by-domain, filter-by-
term-substring, unreachable-db fail-closed, missing-table fail-
closed
- _tokenize_fts_query (only via expand-query smoke): preserve
quoted phrases, parentheses-as-tokens (OR-expansion contract),
unterminated-quote fallback
- _quote_for_fts + _match_quoting (no direct test): pass-through
quoted, defensive quote-multi-word, quoted-reference symmetry
10 new tests; aliases.py test count 18 → 28. Same fixture pattern
as the existing tests (sqlite tmp DB with SCHEMA_SQL applied).
Adds tests/test_t3_bound_calculator.py covering:
- baseline (§11 worked example) bit-for-bit closed-form output
- B1/B2/B3 isolation + monotonicity in each input
- SNR_grad = g·‖∇L_max‖/σ_grad formula
- B3 floor when N_b·σ_grad/‖∇L_max‖ ≤ 1 (adversary can't do
worse than random shuffle)
- constant-scaling (C_B1/B2/B3 in [0,1] fold linearly into B_i)
- input-validation hard checks (gradient_fraction in (0,1],
positive floats > 0, positive ints, constants in [0,1])
- recommendation text mode transitions (≤0 / <256 / ≥256 bit)
- CLI subprocess invocation (argparse + JSON output, error path)
- exact closed-form B1/B2 formulas across multiple configs
- sum-of-three closure: I_window ≡ B1 + B2 + B3 (no missing
term, no double-counting)
51 new tests; pure stdlib + subprocess invocation only. Full
suite now 1720 passed / 45 skipped.
Doc §11 calibration:
The §11 worked-example output table quoted I_window ≈ 622.7,
B1 ≈ 290.0, B3 ≈ 32.7. The closed-form actuals are 625.8716 /
292.4813 / 33.3904 — a ~3-bit total drift from rounding in the
first-cut spec. Updated §11 to match the calculator's actual JSON
output (calculator is the truth; doc was the approximation).
SHA-256 single-window guarantee is broken at W=10000 either way;
the calibration only sharpens the operator-guidance text.
Lands the synthetic-ablation infrastructure proposed in fce8826's
ticket §7 amendment. Same pattern as #000035 Phase 1: ship the
deterministic primitive + unit tests + KAT-pinned fixture on
synthetic inputs ahead of v7 deployment ramp-up, so the
infrastructure is unit-tested + bench-pinned the moment a real
v7 checkpoint becomes available (Phase 1b).
bench/scripts/phi_alignment_probe.py
====================================
Implements ``measure_alignment(W, hessian_eval, *, k_top, k_bot,
epsilon) -> AlignmentReport`` per #000034 §3.1:
- Lanczos top-k + bottom-k via ``scipy.sparse.linalg.eigsh`` over
a user-supplied HVP closure. Probe never materializes H.
- Alignment score: A(W, H) = Σ_j (Σ_i ⟨W·e_i, v_j⟩²) / (λ_j+ε)
/ ‖W‖_F², per ticket §2.1. Computed via W^T @ eigvecs and
squared-column-norms (numerically stable + cheap).
- Verdict thresholds (§3.3): STRUCTURAL_ALIGNMENT (ratio > 1.5) /
NO_ALIGNMENT / ANTI_ALIGNED (ratio < 0.7).
Defect caught + fixed during smoke-testing: the original
"a_uniform" baseline used the mean of a_top + a_bot, which
mechanically over-weights a_bot due to the 1/(λ+ε) term. Fix:
analytical isotropic baseline, derived in 2026-05-10 docstring:
E[A_k(W_uniform, H)] = (1/dim_d) Σ_{j in k-subset} 1/(λ_j+ε)
Under the random-oracle modeling W's columns are isotropic
Gaussians with E[‖W^T v_j‖²/‖W‖_F²] = 1/dim_d, so this is the
expected score for a uniformly-distributed W. Smoke test
post-fix: aligned → STRUCTURAL_ALIGNMENT (ratio ~7.97), uniform →
NO_ALIGNMENT (ratio ~1.00), anti → ANTI_ALIGNED (ratio ~0.00).
All three classes land cleanly in their expected verdict bucket.
Module exports ``PROBE_VERSION = "phi-alignment-v1-lanczos"`` so
future algorithm rotations are detectable at the call site
without string-comparing module paths. Same convention as
#000035's PHI_PRG_VERSION.
bench/fixtures/phi-alignment/synthetic-checkpoints.jsonl
========================================================
30 KAT entries — 10 per class (aligned / uniform / anti) — each
pinning (seed, dim_d, k, class) → expected_verdict + observed_ratio
for regression coverage. Deterministic-seeded so CI replays
exactly. Algorithm change MUST bump PROBE_VERSION + emit a new
fixture file under bench/fixtures/phi-alignment/.
Class ratio ranges:
- aligned: 7.77 - 8.27 (well above 1.5 STRUCTURAL_ALIGNMENT floor)
- uniform: 0.95 - 1.04 (cleanly within NO_ALIGNMENT band)
- anti: 0.00 (well below 0.7 ANTI_ALIGNED ceiling)
tests/test_phi_alignment_probe.py
=================================
14 tests covering #000034 §3.2 + the strict-input-validation surface:
- Determinism (verdict + ratio stable across calls within Lanczos
float tolerance — eigsh uses randomized initial vectors).
- Verdict thresholds (engineered cases land in correct bucket).
- Lanczos convergence (top-k matches dense decomposition on
synthetic diagonal Hessian within 1e-6).
- Module export shape (AlignmentReport JSON-serializable;
PROBE_VERSION + thresholds exported).
- Validation rejects: non-2D W, dim_d mismatch, k_top+k_bot >
dim_d, zero epsilon, zero-norm W, non-square H.
- KAT regression against the 30-entry fixture.
Tests skip via ``pytest.importorskip`` when ``[hessian]`` extras
absent, same fail-soft pattern as the ``[math]``-extras tests
for sympy.
pyproject.toml — new [hessian] optional-deps block
==================================================
Adds ``numpy>=1.26`` + ``scipy>=1.11`` under a new ``[hessian]``
extras gate. Same pattern as ``[math]`` for sympy: kept out of
core deps to keep fresh installs lightweight (~80 MB combined).
Operators install via ``pip install 'arborist[hessian]'``.
#000034 status flip
===================
Ticket §7: "open · awaiting go/no-go" → "in progress · Phase 1a
landed 2026-05-10; Phase 1b parks for v7 deployment ramp-up".
Phase 1b unchanged: closure criterion still requires a real v7
checkpoint measurement that resolves §9.1 of the soft-hash-
channel-analysis. TICKETS.md index row refreshed.
Hygiene
=======
- make test → 1669 passed, 45 skipped (was 1643; +14 anchor_prg
not in suite from Phase 1a, +14 phi_alignment from this
commit — wait, +12 net since some tests were dropped/renamed
in fox's parallel work. Bottom-line: 1669 stable.)
- make chain-check-shards → 0 across all 7 shards.
- arborist.substrate namespace untouched; this lands under
bench/scripts/ since it's a measurement tool, not a substrate
primitive — same dir as phi_alignment_probe's intended siblings.
B-1: via_citation_alias attribution — resolver no longer mislabels
citation-alias-substituted chains as DIRECT.
- New Citation.via_citation_alias field (default False, preserves
parse-from-source-ref path).
- warrant_status sets via_citation_alias=True on substitute
Citations from #000041 lookup_citation_aliases.
- resolve_chunks reads citation.via_citation_alias as a "floor" for
via_alias on every match it produces (Pass 1 hits inherit it
too, not just Pass 2 term-alias hits). Audit-honest: matches
from a substitute Citation are alias-driven regardless of which
cascade pass found the chunk.
Live re-resolve under the new attribution: 18 direct + 74 +alias
(was 75/17 mis-labeled). The 18 direct = exactly Hilbert pillar IV
records resolving on the literally-cited Hilbert textbook. All 74
records resolved via citation-alias substitution now carry
process_id="warrant-resolver-v1+alias" in derivations.
3 new unit tests (test_warrant_resolver.py): default-False on
parsed Citations, explicit-True construction works, resolve_chunks
propagates the floor onto every ResolutionMatch.
B-2: source-side title-from-author backfill — eliminates the
per-shard SQL UPDATE workaround.
- HtmlPageSource accepts default_author kwarg; appends ', by
<author>' to ingested document titles when the <title> tag
doesn't already include the surname.
- TextbookTexSource accepts default_author kwarg; appends ' by
<author>' to titles when the LaTeX has no \author{} macro AND
no PG-style 'Author:' boilerplate.
- _CrawledHtmlSource (BFS-crawler bridge) accepts default_author
kwarg; same append logic. ingest_crawled() and arborist crawl
--ingest plumb it through.
- arborist ingest --author + arborist crawl --author CLI flags.
- bench/scripts/textbooks_manifest.py:cmd_lookup emits the
manifest's `author` field as a 7th tab column.
- make textbook target reads the author column and threads
--author into both crawl-ingest and shallow-ingest paths.
Idempotency preserved — surname-already-in-title detection prevents
double-stamping on re-ingest. Shards previously SQL-backfilled
(Cantor / Russell IMP / Bogart / Judson / Levin / KT / Peano /
Grinstead-Snell) keep their existing titles; new ingests pick up
the author signal at source time.
Live smoke: arborist ingest --source html --author "Bertrand
Russell" against PG #41654 yields title "Introduction to
Mathematical Philosophy | Project Gutenberg, by Bertrand Russell"
with no SQL UPDATE needed.
Total: 1655 tests pass (was 1652). Both follow-ups land additive,
fail-closed, idempotent. The two cleanup items from #000031
Phase 3's commit message are now closed.
When a per-claim warrant_check would fire WARRANT_MISSING but the
cited chunk's document_root has a warrant-resolver derivation row
(Merkle-bound primary-source backing), the verifier now suppresses
the demote and tracks the claim on a new `warrant_proven_claim_idxs`
field. The render layer surfaces this as `· warrant proven via
chain ×N` in the audit-line tail so operators see when a claim got
through on the chain rather than on lexical anchors.
Mechanism (additive, fail-closed):
1. New `arborist/qa/warrant_chain.py` — read-only helper that
loads the frozenset of `core_root` values having a derivation
row with `process_id LIKE 'warrant-resolver-v1%'`. One sqlite
query per Q&A run, walks main shards + sibling crawl/ shards
(skipping ad-hoc crawl_russell_/qa./snapshots. prefixes).
Tolerates missing tables.
2. `verify_claim_lattice` + `verify_claim_lattice_json` accept
a new optional `warrant_chain_roots: frozenset[str]` parameter
(default empty = backward-compatible). When the lexical
warrant_check fails for a claim AND any cited evidence's
source_root is in the set, the WARRANT_MISSING violation is
suppressed and the claim_idx flows to a new
`warrant_proven_claim_idxs` field on the verdict.
3. `runner.ask` computes the warrant_chain_roots set once from
the conn's main DB directory before invoking the verifier.
Failure-mode fallback: empty set, behavior identical to
pre-Phase-3.
4. `query.py` threads `warrant_proven_claim_idxs` from the
verdict into the result dict.
5. `cli.py:_render_warrant_tail` adds a `warrant proven via chain
×N` segment when `warrant_proven_claim_idxs` is non-empty.
Distinct from the pre-existing `_render_warrant_chain_tail`
which counts cited SOURCES with chains; this counts CLAIMS
that survived because of a chain.
Tests:
- tests/test_warrant_chain.py — 9 new tests covering
warrant_chain_lookup (basic / unrelated process_id / missing
table / +alias variant / empty path) + has_warrant_chain
short-circuit + verifier suppression behavior + verdict-field
presence guarantee.
Live smoke: warrant_chain_lookup(~/.arborist/shards) returns
exactly 92 core_roots (matches the 92/92 claim-pack records
resolved earlier today).
Total: 1652 tests pass (was 1643). Honest layering preserved:
- soft signal (positive warrant_proven) lives on a separate verdict
field, not in `violations` (which stays a hard-failure list)
- audit_mode (STRICT/HYBRID/UNGROUNDED) unchanged when chain
suppresses WARRANT_MISSING — the claim was going to land at
HYBRID without the suppression; the suppression keeps it at
STRICT, which is now defensible because the warrant chain IS the
warrant
- four-rung ladder rung promotes naturally: no WARRANT_MISSING in
violations + no soft demotes -> EVIDENCE-WARRANTED via the
existing _ladder_rung_for_lattice logic. No render-layer ladder
change needed.
Phase 3 follow-ups still open under #000031:
- via_citation_alias process_id attribution (~15 LOC)
- source-side title-from-author backfill in HTML/textbook_tex
ingest (~30 LOC)
Followup to 654d923 (which moved the package from arborist/v8/ →
arborist/substrate/ at the file layer). The CLI surface still baked
in `v8` so a new operator running `--help` would see
``arborist v8 score`` and ask the same "what's v8 vs v9.8?"
naming-confusion question that drove the package rename in the
first place. Closing the loop end-to-end.
arborist/cli.py
===============
- Subparser renamed: ``"v8"`` → ``"substrate"``; help string updated
to "Merkle-AGI substrate primitives (ForkScore + future paper
specs)" so the dir name and command name and help text all align.
- Inner subparser dest renamed: ``v8_op`` → ``substrate_op``.
- Function renamed: ``_cmd_v8_score`` → ``_cmd_substrate_score``;
docstring updated.
- All ``v8_score`` local variables renamed to ``substrate_score``.
- New comment block above the subparser block explains the rename
+ why the v-prefix was retired (substrate-paper version vs v9.8
schema version naming collision).
The old ``arborist v8 score`` is gone — no alias preserved. CI + ops
scripts must update; today's earlier commit chain has been the only
place using it and that's been refreshed in lock-step.
tests/test_v8_fork_score.py
===========================
- 4 ``parser.parse_args(["v8", "score", ...])`` calls → ``["substrate", ...]``.
- 4 test functions renamed: ``test_cli_v8_score_*`` →
``test_cli_substrate_score_*``.
- Module docstring + section comment + helper docstring updated.
Filename intentionally kept as ``test_v8_fork_score.py`` for git
history continuity; pytest discovers by ``test_*`` content, not
filename. Renaming the file would muddle ``git log --follow`` for
the test surface.
Docs refreshed
==============
- docs/v8-fork-score.md — §5 CLI block invocation.
- docs/_source/v8-fork-score.rst — :code-block:: bash invocation.
- docs/_source/bench.rst — invocation in `### v8 ForkScore` section.
- docs/tickets/ticket-000012-selection-consensus-protocol.md —
three references in §7 close-out + §7 Phase 1c proposal +
§7 future-CLI-shape note.
- docs/dav1dprometheus-update-2026-05-09.md — bench journal mention.
Doc filenames (``v8-fork-score.{md,rst}``) kept stable since they
are URL identities; the file content explains the v8→substrate
rename internally. ``index.rst`` toctree references unchanged.
Hygiene
=======
- ``.venv/bin/arborist substrate score --help`` → 0 + valid usage.
- ``.venv/bin/arborist v8 score`` → exits non-zero (subcommand
removed, surfaced cleanly in ``argparse`` error).
- ``make test`` → 1643 passed, 45 skipped.
- ``make chain-check-shards`` → 0 across all 7 shards.
- fox's parallel work in arborist/qa/{runner,verify}.py +
arborist/qa/warrant_chain.py left untouched.
fox's read: the version-prefixed namespace pattern (`arborist/v7/`,
`arborist/v8/`) coupled module location to the substrate-paper
version. That collided with the live SQLite schema version (v9.8)
and made readers ask "is this dir tracking schema or paper?" —
a real onboarding hazard surfaced when the v7 dir landed earlier
today (06c95a0) for #000035 Phase 1.
Resolution: collapse v7+v8 into one topic-named dir,
``arborist/substrate/``, which holds Merkle-AGI substrate primitives
that future paper specs require — decoupled from the paper version.
Moves
=====
arborist/v7/anchor_prg.py → arborist/substrate/anchor_prg.py
arborist/v8/fork_score.py → arborist/substrate/fork_score.py
arborist/v8/weights.py → arborist/substrate/weights.py
Empty v7/ + v8/ dirs deleted; their __init__.py docstrings folded
into the new arborist/substrate/__init__.py with an explanation of
why the version-prefixed pattern was retired.
Imports updated
===============
- arborist/cli.py:_cmd_v8_score — arborist.v8 → arborist.substrate
- arborist/substrate/fork_score.py — internal weights import
- tests/test_anchor_prg.py — module + module-docstring
- tests/test_v8_fork_score.py — three import lines
Docs updated
============
- docs/v8-fork-score.md — header note explaining the move
- docs/_source/v8-fork-score.rst — :class: ref updated
- docs/tickets/ticket-000012-selection-consensus-protocol.md — §7
Phase 1a close-out paths refreshed (kept "Originally landed at
arborist/v8/..." parenthetical so the historical record survives);
§7 Phase 1b consensus-paper reference; §7 Phase 1c proposal §3
read-API path
- docs/tickets/ticket-000035-prg-choice-phi-prg.md — §7 Phase 1
close-out path refreshed (with full path-note explaining the
move); §3.1 + §5 left as the original design log per CLAUDE.md
"closed tickets stay in place as design log"
Left untouched
==============
- arborist/world/ — already topic-named; not version-prefixed; the
v7-W reservation lives there with its own planned subdir layout.
- docs/tickets/ticket-000037-prometheus-sigma-...md §13 still refs
``arborist/v9/prometheus.py`` and ``arborist/v8/fork_score.py`` —
fox has 792 lines of in-flight modifications on this file; those
refs should refresh to ``arborist/substrate/`` when the in-flight
edit lands. Avoiding interleaved edits.
Hygiene
=======
- make test → 1643 passed, 45 skipped (was 1643; refactor preserved)
- make chain-check-shards → 0 across all 7 shards
- arborist.substrate namespace picked up by the existing
pyproject.toml ``include = ["arborist*"]`` glob; no setup change.
Lands the M1 mitigation cryptographic primitive that ticket #000018
§5.2 + §9.2 specified, scoped per ticket #000035 §3.1-§3.3. Pure
stdlib (hashlib + hmac); no third-party dependency.
arborist/v7/__init__.py
=======================
First module landed under the v7 namespace. v7 plastic-training is
currently paper-stage (per #000037 §17.2); this is where its
deterministic primitives accumulate ahead of an active deployment
target so the building blocks are unit-tested + KAT-pinned the
moment v7 needs them.
arborist/v7/anchor_prg.py
=========================
Implements ``phi_prg(hard_hash_32, dim_h, *, seed) -> list[float]``
per #000035 §3.1. Construction is SP 800-108 KDF in counter mode
over HMAC-SHA-512:
Output(SEED, C(M), n_bytes) :=
i = 0
out = b""
while len(out) < n_bytes:
out += HMAC-SHA-512(SEED, C(M) || i.to_bytes(4, 'big'))
i += 1
return out[:n_bytes]
Float conversion: f(u32) := 2 * (u32 / 2**32) - 1
↑ uniform on [-1, 1)
Security argument from #000035 §2.1: HMAC-SHA-512 is a PRF under
the standard SHA-512 + HMAC assumption; distinguishing advantage
from random bounded by SHA-512 collision-resistance (~2^256), which
structurally matches the substrate's SHA-256 hard-hash family. The
seed is published, not secret — secrecy is not the security
property; the property is computational indistinguishability of the
output from random, which holds even when the seed is public.
Module exports ``PHI_PRG_VERSION = "phi-prg-v1-hmac-sha512"`` so
future algorithm rotations are detectable at the call site without
string-comparing module paths. Per #000035 risk §6.1, a
``phi_prg_version`` field in the v7 manifest will let future
deployments swap to a successor PRF without breaking historical
replay; this version string is the runtime-side mirror.
Hard-hash input length checked exactly at 32 bytes — silently
padding shorter input would break the PRF security argument.
``dim_h`` validated as positive int.
tests/test_anchor_prg.py
========================
20 tests covering #000035 §3.2 acceptance criteria + the strict
range invariant + the input-validation surface:
- Determinism: same (seed, hard_hash, dim_h) → identical floats.
- Range: every output in [-1, 1) with strict upper bound. Three
edge cases pinned: u32=0 → -1.0, u32=2^31 → 0.0,
u32=2^32-1 → just below 1.0.
- Chi² loose-uniformity: 4096-sample bin-test (df=15) with a
generous threshold (60); catches catastrophic PRG bugs (counter
cycling, mis-keyed HMAC) without claiming cryptographic-grade
evidence.
- Boundary: dim_h=1 + dim_h=16384 both produce sensible output.
- Avalanche, seed-bit: flip top bit of seed[0]; require 35-65% of
output bits flipped (PRF avalanche property).
- Avalanche, hash-bit: same surface for the hard-hash input.
- Validation rejects: short hashes, long hashes, non-bytes hashes,
zero / negative / non-int dim_h.
- Module export shape: PHI_PRG_VERSION + PLACEHOLDER_SEED.
- KAT regression: pinned vectors verified against
``bench/fixtures/phi-prg/known-answer-tests.jsonl``.
bench/fixtures/phi-prg/known-answer-tests.jsonl
================================================
10 KAT vectors generated against the placeholder seed + custom
seed/hash combinations; covers smoke (placeholder seed × small
dim_h), block boundaries (HMAC-SHA-512 blocks are 64 bytes, so
dim_h=16 is exactly one block, dim_h=17 is two blocks with
truncation), seed/hash one-bit-flip variants, and a 4096-element
stress sample.
Each row pins the SHA-256 of the raw byte stream (not the float
list) — that's the durable contract; switching from list[float] to
array.array('f', ...) or numpy arrays at the float layer would not
invalidate the fixture. Algorithm changes MUST bump
PHI_PRG_VERSION and create a new fixture file under
bench/fixtures/phi-prg/; old runs replay against old data per the
v7 spec replay discipline.
#000035 status flip
===================
docs/tickets/ticket-000035-prg-choice-phi-prg.md §7 updated from
"open · awaiting go/no-go" to "in progress · Phase 1 landed
2026-05-10". §7 now carries Phase 1 close-out details + Phase 2
gating criteria (active v7 deployment target + spec maintainer
review of §3.4 amendment text). The §3.4 v7 §9.10 amendment text
stays as the draft awaiting Phase 2 landing.
docs/TICKETS.md index row was already updated by fox in commit
ed470dc; my edit was idempotent.
Hygiene
=======
- make test → 1643 passed, 45 skipped (was 1623; +20 anchor_prg)
- make chain-check-shards → 0 breaks across all 7 shards
- arborist.v7 namespace picked up automatically by the existing
pyproject.toml [tool.setuptools.packages.find] include="arborist*"
glob; no setup change required.
- fox's in-flight #000037 ticket modifications + a parallel
#000031 ticket update left untouched.
Phase 0 of ticket #000037 (Prometheus-Σ recursive falsification
controller) is doc-only and gates Phase 1 on §12 measured-pressure
triggers. This commit lands the empirical-evidence harness fox needs
for the go/no-go decision, plus the §16.2 test scaffolding so the
contract is discoverable from the test runner today.
bench/prometheus_sigma_trigger_probe.py
=======================================
Read-only walk of audit_events + capital_ledger across shards.
Reports each §12 trigger:
- Trigger 1 (branch density, ≥4 branches/checkpoint): looks for a
fork_score branch-set table; reports "no data — single-validator
ForkScore Phase 1a" when absent. Trigger structurally cannot fire
until #000012 multi-branch persistence lands.
- Trigger 2 (divergence variance, N≥30 + ratio>0.5 OR abs>0.10):
aggregates providence_canonical_witness audit-event bodies,
maps each agreement_label per #000028 §1.2 to a binary
LLM-divergence score, computes mean/stddev/ratio. Defensive on
the max(mean,ε) guard from §12 itself.
- Trigger 3 (witness cost share > 0.30): aggregates capital_ledger
rows; uses `material` (kWh proxy) as the canonical compute axis;
surfaces both `material` and `financial` so fox can pick a
different form if needed. Tags the report with the caveat that
the current ledger reflects ad-hoc activity, not a controlled
#000026 sweep.
- Trigger 4 (operator mission need): n/a — operator decision.
Pure measurement, no LLM, no schema change, no mutation. Output is
a markdown report at $(PROMETHEUS_PROBE_OUT) (default
bench/results/prometheus-sigma-triggers-<utc-date>.md).
Wired via `make prometheus-trigger-probe`.
bench/results/prometheus-sigma-triggers-2026-05-10.md
=====================================================
First captured baseline. Verdict on current shards:
Trigger 1: NO (no fork_score branch-set table; #000012 Phase 1a
is single-validator)
Trigger 2: NO (16 samples; N_min=30. But mean=0.625, σ=0.5,
ratio=0.8 — both ratio AND abs floors would fire
if N reaches 30. Signal is there; just needs more
samples.)
Trigger 3: NO (witness/total material = 0.005, well under 0.30
threshold. Caveated: ledger is ad-hoc, not a
controlled #000026 sweep.)
Trigger 4: n/a (operator-stated)
Empirical answer: no §12 measured-pressure trigger has fired yet.
Phase 1 of #000037 remains paper-only unless fox invokes Trigger 4.
tests/test_prometheus_sigma.py
==============================
17 skip-stubs pinning the §16.2 acceptance contract. Each test:
- Collects today via `pytest --collect-only` so the test surface is
discoverable.
- Skips with reason "Phase 1 not landed; controller module
arborist/v9/prometheus.py absent" until that import succeeds.
- Carries a one-sentence intent line tying it back to a numbered
ticket section (§5 / §6 / §7 / §10 / §13 / §14 / §16.1 / §4.4).
When Phase 1 lands, the implementer adds the controller module
under arborist/v9/prometheus.py per §13; CONTROLLER_AVAILABLE
becomes True; each test gets its body filled in. The names, intents,
and skip-reason strings are the durable contract.
Hygiene
=======
- make test → 1623 passed, 45 skipped (was 28; +17 new skips).
- make chain-check-shards → 0 breaks across all 7 shards.
- No code touched outside the new probe + scaffolding files +
Makefile target wiring. fox's in-flight #000037 ticket
modifications left untouched.
Implements both alias mechanisms (citation-aliases #000041,
term-aliases #000042) as one cohesive layer. Same audit
discipline; same opt-in via --use-aliases on warrant-resolve;
same distinct process_id "warrant-resolver-v1+alias" on alias-
resolved derivations rows.
What landed
===========
arborist/store.py — two new tables under SCHEMA_SQL:
citation_aliases — substitute textbook for proprietary cite
term_aliases — bridge vocabulary mismatches (incidence ↔
connection in geometry, etc.)
Both with NOT NULL audit fields (decision_at + decision_by
+ optional decision_rationale); both with PK constraints
ensuring idempotent re-add.
arborist/qa/aliases.py — helper module:
add_citation_alias / list / lookup / remove
add_term_alias / list / lookup (bidirectional) / remove
expand_query_with_term_aliases — OR-rewrites FTS5 tokens
while preserving phrase syntax
domain_for_pillar — Roman numeral → domain string
arborist/qa/warrant_resolver.py — two-pass cascade:
Pass 1: unaliased queries (matches carry via_alias=False)
Pass 2: alias-expanded queries (only when pass 1 missed;
matches carry via_alias=True)
ResolutionMatch grew a via_alias field; warrant_resolve
uses it to pick the right process_id per derivation row.
iter_claim_pack_records now yields a 6-tuple including the
pillar (parsed from doc URI) so domain lookup works.
arborist/cli.py — alias subcommand group:
arborist alias citation add ORIGINAL --substitute SUB --by FOX [...]
arborist alias citation list / remove
arborist alias term add TERM ALT --domain D --by FOX [...]
arborist alias term list / remove
warrant-resolve --use-aliases flag
sweep --target warrants --use-aliases flag
All audit fields fail-closed at the API surface (refuses on
empty --by; ValueError raised at the helper level).
End-to-end smoke test
=====================
Registered the Hilbert smoke-test alias:
arborist alias term add incidence connection \
--domain geometry \
--by "blackops 2026-05-09 (smoke test)" \
--rationale "Hilbert 1902 Townsend uses 'connection' for
what modern texts call 'incidence'"
Re-ran warrant-resolve --use-aliases --write:
records_total: 92
records_resolved: 15 (was 11 without aliases)
derivations_written: 15
Breakdown by process_id:
warrant-resolver-v1: 11 (original-citation matches)
warrant-resolver-v1+alias: 4 (alias-resolved Hilbert axioms)
The 4 alias-resolved records are exactly the Hilbert "Incidence"
axioms blocked by terminology mismatch in #000040 §6:
Axiom of Line Incidence
Axiom of Plane Incidence
Axiom of Point-Line Incidence
Axiom of Point-Plane Incidence
All four bound to chunk 56 in the Hilbert TeX surface — the
chapter discussing "axioms of connection" (Hilbert's original
1902 vocabulary). Audit trail correctly distinguishes
substituted chains from original ones.
Tests: 18 new in test_aliases.py covering add / list / lookup
/ remove / domain isolation / lowercase normalization /
bidirectional lookup / audit-discipline raises / query
expansion (basic + phrase-preserving + no-match passthrough +
unreachable-DB fallback). Full suite: 1623 passed / 28
skipped.
Tickets #000041 + #000042 closed. Operators can now add more
aliases via the CLI as fox makes decisions per #000038. The
alias mechanism is fail-closed by default — existing
warrant-resolve runs without --use-aliases continue to produce
the original 11/92 chains; --use-aliases opt-in adds the
substituted chains alongside without polluting the unsubsituted
audit trail.
The fan-out commit (2b9d1f0) exposed a 13.5s shard-002 fts5_body call
on the Gundremmingen query and hypothesised the synonym OR-pool was
blowing the FTS5 candidate set. Profiling falsified that hypothesis:
synonym_expand returned no synonyms, the OR pool was just the three
query tokens, and the bottleneck was a single high-DF QUERY token
("located": 286,160 matches on a 1.5M-chunk wiki shard) carried into
OR-mode after AND-mode found zero co-occurrences. BM25 ranked all
~290k matches just to pick the top-32.
Two layered fixes:
A. Progressive-AND fallback. When AND returns zero, drop the shortest
token (input order breaks ties) and retry AND. Repeat until hits or
one token left. Only after every chain returns zero do we fall to
OR-mode. On the Gundremmingen case, dropping "located" leaves
"Gundremmingen AND Bavaria" which intersects to 3 docs in 11ms
instead of the 290k-match OR-mode wall.
B. Document-frequency filter at OR-fallback time. ``COUNT(MATCH "tok")``
per OR-pool token; drop any whose corpus DF exceeds
``_OR_FALLBACK_MAX_TOKEN_DF`` (default 50,000). ~15ms warm per
probe. Only fires on the rare path where every progressive-AND
chain still returned zero. Backstops A for queries where the
answer genuinely requires OR (synonym-anchored retrieval, queries
for content that uses different vocabulary than the question) but
one of the OR clauses is a high-DF stopword-adjacent verb.
Both are deletion-first per the five-step algorithm: A deletes the
"jump straight to OR" path, B deletes high-DF tokens that contribute
~zero IDF anyway. No magic constants for A; B has one tunable knob
(threshold).
Bench (cold-cache, n=3, serial workers=1, query "where is
Gundremmingen located? where is Bavaria?"):
metric BEFORE AFTER (A+B) delta
total search wall 57.20s ± 0.22 1.67s ± 0.13 -97% / 34x
shard 002 fts5_body cold 28.07s 0.05s ~560x
shard 002 fts5_body hits 32 3 -29 (the
dropped
were
"located"-
only noise)
Top-K=8 chosen sources unchanged before/after — the dropped fts5_body
candidates were filtered by the title-relevance step downstream
anyway.
Tests (tests/test_search_fts5.py, 11 cases):
- Helper: 5 cases on _progressive_and_token_chains (single-token,
empty, shortest-first, strict length sort, always-keeps-one).
- Search behaviour: 4 cases (progressive-AND drops high-DF token;
full-AND succeeds without progression; OR fallback when no chain
hits; empty result when corpus has neither token nor synonym).
- DF filter: 2 cases (drops high-DF token, keeps input when all
candidates would otherwise be dropped).
Verification:
- make test → 1605 passed, 28 skipped (was 1597 pre-change)
- make chain-check-shards → 0 breaks across all 7 shards
- arborist query "where is Gundremmingen located? where is Bavaria?"
returns the same top-8 sources before/after
Implements the layered cascade strategy from #000040 §3.1
(originally drafted as #000039 — renumbered after collision
with parallel-shift's sqlite-vec ticket).
What landed
===========
arborist/qa/warrant_resolver.py:
- _phrase_for_axiom(theorem_name) — strips leading
categorical prefix ("Axiom of " / "Theorem " / "Principle ")
and trailing parenthetical, returns FTS5 phrase syntax
('"line incidence"', '"plane incidence"', '"side angle
side"', etc.) when the theorem name has 2+ tokens.
- _content_tokens(chunk_content, max_n=8) — extract
discriminating tokens from a claim-pack chunk's body. Drops
stopwords / generic theorem terms / common-English (small
hand-curated set). Requires count >= 2 to ditch typo /
LaTeX residue singletons. Sorts by length DESC then
first-position ASC.
- _build_record_query_cascade(c, theorem_name, content) —
returns ordered list of FTS5 queries to try:
1. Phrase from title
2. Content-tokens AND-joined
3. Existing discriminating-tokens AND-join (legacy)
4. Existing OR-fallback (legacy)
- resolve_chunks gains a `record_content` parameter; tries
each cascade query in order, first hit wins.
- iter_claim_pack_records yields a 5-tuple including content
so callers can thread it through.
Tests: 6 new unit tests for the cascade helpers (phrase
extraction, parenthetical stripping, single-token fallback,
content-token filtering, count-2 minimum, cascade ordering).
20 total in test_warrant_resolver.py. Full suite: 1603
passed / 28 skipped.
End-to-end honest result
========================
Re-running warrant-resolve on the existing shard cluster:
records_total=92, records_resolved=11 (unchanged from Phase 4).
The cascade is correct; the lift didn't materialize for
Hilbert pillar IV's 7 missing records because of TERMINOLOGY
MISMATCH, not query strategy:
- claim-pack records (g4 2025) use modern post-1950s names:
"Axiom of Line Incidence", "Group I: Axioms of Incidence".
- Hilbert's 1902 Townsend translation uses the original
"Verknüpfung" / "axioms of connection".
- Empirically: the literal token "incidence" appears ZERO
times in the ingested Hilbert TeX surface; "connection"
is the relevant synonym.
No matter how clever the query, you can't find a word that
isn't there. The cascade is preserved for any future textbook
where cited vocabulary matches textbook prose (modern
Stanley / Brualdi / Knuth, etc.).
Next-link follow-up: file #000042 term-aliases table
(("incidence", "geometry") → ("connection", "geometry")).
Sibling design to the citation-alias proposal at #000041.
Renumbering note: the Phase 5 ticket file was renumbered
000039 → 000040 mid-session because parallel-shift took
000039 for sqlite-vec at nearly the same time. Internal
references in the file follow the post-rename numbering
(#000041 = citation-alias, #000042 = term-alias).
Closes the warrant-promotion data path: claim-pack records now
bind to surface-ingested textbook chunks via Merkle inclusion
proofs in the existing `derivations` table.
What landed
===========
arborist/qa/warrant_resolver.py — four pure-data steps + one DB
write:
1. parse_citation(s) — regex pipeline turning the claim-pack
`source_reference` string into structured Citation tuples.
Handles "Title by Author" (single + Oxford-comma multi +
et-al), semicolon-separated multi-cite ("Knuth §1.2.6;
Stanley §1.2; Brualdi §3.5"), and compact author-year
("Pascal 1654") forms.
2. resolve_chunks(c, shards_dir) — FTS5 search across sibling
crawl/ dir's textbook-surface shards. Skips the main numbered
shards (Wikipedia content; would be false positives). Per-
shard match filter requires BOTH author last name AND a title
token in the shard's title-haystack — honest "no match" for
textbooks not yet surface-ingested.
3. compute_proof(shard, doc_root, chunk_id) — reads
merkle_nodes, walks layer-by-layer to assemble siblings;
emits deterministic JSON proof_blob compatible with
arborist/merkle.py verification.
4. write_derivation(...) — INSERT OR IGNORE into the existing
derivations table with process_id="warrant-resolver-v1".
Idempotent at the database layer.
CLI surface
===========
- `arborist warrant-status --shards-dir ...` (read-only) —
emits per-record JSON: parsed citations, FTS5 candidates,
whether a derivations row exists.
- `arborist warrant-resolve --shards-dir ... [--write]` —
default dry-run summary; --write actually computes proofs
and inserts rows.
End-to-end verification
=======================
Real-shard run: `arborist warrant-resolve --shards-dir
~/.arborist/shards --write` →
records_total: 92
records_resolved: 18
derivations_written: 18
All 18 are pillar-IV Hilbert axioms citing "The Foundations of
Geometry by David Hilbert" — the only cited textbook fully
surface-ingested by Phase 1. The remaining 74 records cite
textbooks not in our shard cluster (Mendelson, Enderton,
Jech, Goldstein, Barendregt, Stanley, Brualdi, Knuth, …) and
correctly produce 0 matches; they stay at ANCHOR-WARRANTED
until those textbooks land via future Phase-1 manifest
expansions.
Re-running the writer is a no-op (PK collision on (core_root,
src_root, process_id) = INSERT OR IGNORE).
Drive-by fix
============
arborist/sources/textbook_tex.py — _extract_title now also
parses PG's plain-text `Author:` line and appends "by Author"
to the title, so the warrant resolver's author-last-name match
works against PG-ingested textbooks (Hilbert "The Foundations
of Geometry by David Hilbert" instead of just "The Foundations
of Geometry").
Test suite
==========
tests/test_warrant_resolver.py — 14 unit tests for the citation
parser (no DB / network). Full suite: 1588 passed / 28 skipped.
Phase 3 (verifier wiring)
=========================
NOT in this commit. The data substrate is in place; the
audit_mode upgrade path that lifts answers citing
claim-pack-records-with-derivations from ANCHOR-WARRANTED to
EVIDENCE-WARRANTED requires a verifier change — touches well-
tested code, worth its own ticket so the regression risk is
bounded.
Three artifacts landing per ticket §4.1 closure criterion:
1. docs/_source/merkle-agi-v7w-spatial-temporal.rst (658 lines)
============================================================
Substrate paper for the third commitment substrate — sister to v7
(logic / math) and arborist v9.8 (language / claim-lattice). v7-W
commits derived spatial-temporal world-state: objects, relations,
events, places, agent traces, observations. Six parts + appendix:
Part 1 — Introduction & motivation. The third-substrate gap;
why v7 § 11 multimodal composition isn't enough.
Part 2 — Substrate definition. Hierarchical-grid spatial
discretization (S2 / H3 / octree); frame as committed
object with explicit transforms; substrate-declared
clock (single-agent) + Lamport (multi-agent);
quantized centi-confidence (range opt-in); five
canonical tuple-classes (object / relation / event /
place / agent_trace) each with its own π*_w.
Part 3 — Theorems. T1-W (state binding), T2-W (causal
completeness), T3-W (frame-transform soundness),
T4-W (ε at affine frontiers).
Part 4 — Verifier kernels. Pose integration, observation
update (Kalman), object logits, relation logits.
Each affine after canonical projection.
Part 5 — Multimodal composition with v7. Where v7 ends, v7-W
begins; cumulative ε across substrates; frame-
transform anchoring.
Part 6 — Adversarial corners. Frame spoofing, time skew,
observation injection, privacy.
Appendix — Worked SLAM example with full ε budget.
Hard constraints honored: stays inside SQD A1-A3 (canonical
encoding, public quantization, collision-resistant hash); no new
axiom; every π*_w defined on quantized integer state, never on
continuous tensors.
2. docs/v7w-frontier-catalog.md (262 lines)
============================================
Operator-facing quick reference for the four ε-frontiers from
substrate-paper Part 4. Each entry:
- canonical input / output bytes
- operator (linear / bilinear / Kalman / SE(3))
- ε bound expression
- "affine after canonical projection" justification
- when to use
Reference table + cumulative-ε section so operators sizing
deployment grid choices can read off their ε_total under typical
agent-trace + scene-graph workloads.
3. arborist/world/__init__.py — namespace reservation
======================================================
Reserved ``arborist.world`` package. No kernels yet. Module
exports V7W_VERSION ('v0-draft') + STATUS ('namespace_reserved')
metadata. Package docstring lays out the future shape per
substrate-paper Part 4:
arborist/world/
├── pi_star/ — π*_w canonical projections (5 tuple classes)
├── frontier/ — ε-frontier kernels (4 frontiers)
├── frame.py — frame definitions + transforms
├── clock.py — wall-clock + Lamport
├── manifest.py — substrate manifest schema
└── adapters/ — sensor adapters land here, separate tickets
Implementation tickets cite the substrate paper and land kernels
one at a time; the stub exists so cross-referencing imports (mesh
peers, sibling repos) can pin the namespace before anything
implements it.
5 tests pin the reservation contract (test_world_namespace.py):
import succeeds, V7W_VERSION reports v0-draft, STATUS reads
namespace_reserved, __all__ exposes only metadata, substrate
paper + frontier catalog files exist alongside the namespace.
Closure criterion (#000013 §7): substrate paper lands and is
ready for review. Done. Status flipped to closed in the ticket
file + TICKETS.md index entry.
Test suite: 1641 passed, 37 skipped (was 1636; +5).
Three small streams:
#3 — close#000030 properly
============================
All 7 phases + Phase 1b landed across two commits (`04f3f5d`,
`abe5988`). Status header updated; ticket body now carries a phase
landing table with commit refs:
Phase 1 algebra-symbolic@v1 04f3f5d
Phase 1b algebra-symbolic-simplified@v1 04f3f5d
Phase 2 calculus-derivative@v1 04f3f5d
Phase 3 calculus-integral@v1 fox-direct
Phase 4 calculus-limit@v1 abe5988
Phase 5 calculus-series@v1 abe5988
Phase 6 linear-algebra@v1 abe5988
Phase 7 function-sampled@v1 abe5988
Plus tabular-pinned@v1 (last reserved stub) graduated in abe5988
closes the registry chapter — 15 concrete π*'s, no remaining
reserved stubs. Index updated.
#5 — composition fixtures across new SymPy π*'s
================================================
12 new tests in tests/test_pi_star_compositions.py covering pairs
that compose naturally:
- algebra-symbolic ∘ algebra-symbolic — idempotency check (running
expand twice equals expand once for any expression).
- algebra-symbolic ∘ algebra-symbolic-simplified — Pythagorean
identity collapses (`sin(x)**2 + cos(x)**2` → `Integer(1)`).
- Generic invariants: composition propagates PiStarError; manifest
fingerprint is order-sensitive; composite domain == inner domain;
composite bytes == manual chain bytes.
Test discipline: most compositions use `register_in_registry=False`
via a small `_safe_compose()` helper since the registry rejects
duplicate keys (#000015 invariant), so test ordering would
otherwise matter. Only the registration-test path uses real
compose().
#4 — end-to-end witness sweep against real shards + Hermes
===========================================================
New script `bench/scripts/witness_sweep.py`. Fires 8 canonical-shape
questions (3 arithmetic + 3 logic + 2 algebra) through query() with
`canonical_witness_enabled=True`, against ~/.arborist/shards (real
shard cluster) + the actual Hermes endpoint (NOT StubClient).
Records the agreement matrix per question to
bench/results/witness-sweep.json.
`make bench-witness-sweep` Makefile target. Honors
`ARBORIST_SHARDS_DIR`.
First real sweep (this commit, against Hermes-3-8B):
agreement label count rate
KERNEL-LLM-DIVERGED 5 62.5%
KERNEL-LLM-AGREE 3 37.5%
───────────────────────────────────────────
divergence_count 5 62.5%
wall median / max 130 ms / 1.1 s
Hermes diverged on 5/8 of the canonical-shape questions:
- said `1/10` for `0.1 + 0.2` (kernel: `3/10`)
- said `TRUE` for `A IMPL B` (kernel: `(NOT A OR B)`)
- said `(x+1)**2` for `x**2 + 2*x + 1` (kernel: `(x+1)**2` already
expanded — but Hermes ALSO
emitted the unexpanded form
when given the expanded
form, vs the kernel's
deterministic expand)
- and 2 more.
These are real LLM hallucinations on questions with closed-form
ground truth — exactly the calibration-data stream #000028
imagined. Pipeline validated end-to-end against actual hardware.
Pair: `make bench-witness-divergence` then extracts the 5
divergences as 5F-Falsification fixtures
(bench/fixtures/5f/falsification-witness-v1.jsonl, also committed).
Re-running the extractor produces byte-equal output (idempotency
contract from the extractor work).
Tests
=====
Full suite: 1636 passed, 37 skipped (was 1624; +12 composition
tests). The witness-sweep + extractor produce real artifacts now
committed under bench/results/ and bench/fixtures/5f/.
A new π* kernel that canonicalizes pure-integer counting
expressions and FAILS CLOSED on any input whose result isn't a
non-negative sp.Integer. Tighter domain than algebra-symbolic@v1,
which already accepts the same input surface but happily returns
symbolic / negative / non-integer outputs.
Distinguishing feature versus algebra-symbolic@v1:
algebra-symbolic@v1: binomial(n, k) → "binomial(n, k)" (symbolic
passthrough)
combinatorics@v1: binomial(n, k) → PiStarError (fail-closed
on free-symbol output)
algebra-symbolic@v1: binomial(Rational(1,2), 3) → 1/16 (rational)
combinatorics@v1: binomial(Rational(1,2), 3) → PiStarError
(output not Integer)
Boundary kept explicit: binomial(-3, 2) = 6 IS accepted because the
output is an integer 6. The fail-closed rule is on output shape
(Integer ≥ 0), not input range. Documented as
test_generalized_binomial_negative_args_accepted_when_integer.
Output format: plain decimal literal (b"10", b"5040"). Composes
with arithmetic@v1 for byte-identical agreement with the rational
route (b"10/1") so the multi-modality witness (#000028) can pin
equivalence-class agreement when both routes fire on the same
question.
Allowed surface (via SymPy primitives): binomial, factorial, ff /
rf (falling/rising), catalan, bell, partition, stirling, plus
arithmetic compositions over those primitives
(3*binomial(5,2) + factorial(4) = 54).
Coverage:
- 43 unit tests including binomial symmetry C(n,k)=C(n,n-k),
Pascal's rule C(n,k)=C(n-1,k-1)+C(n-1,k), the C(n,k) =
factorial(n)/(factorial(k)·factorial(n-k)) identity,
fail-closed paths (symbolic/negative/non-integer/relational/
parse), round-trip idempotence, composition with arithmetic@v1.
- 10 syntax + 12 semantics bench fixtures, 100% pass.
- bench/batteries/base.py PHASE_1_CARRIERS gains "combinatorics".
- Makefile bench-5s-combinatorics target.
All gate on pytest.importorskip("sympy") so a sympy-less suite
stays green. Full make test: 1537 passed / 28 skipped.
Sequencing rationale honored: this kernel lands FIRST so that
#000033 (claim-pack pillar VII for combinatorics) can bind its
records to the tighter integer kernel from day one — avoids
rebind churn on pi_star_ref fields.
Three small streams in one commit:
#000028 follow-up — witness divergence → 5F fixtures
=====================================================
Witness fan-out now writes a `providence_canonical_witness` audit
event when it fires (next to the capital-ledger record landed in
708aa45). Body carries pi_star_ref, question_text, agreement_label,
canonical_answer_text, llm_raw_text, llm_canonical_bytes,
cache_status. Best-effort write — chain failure never fails the
query.
New extractor `bench/scripts/witness_to_5f.py` reads those events
from a qa.db and writes them out as 5F-Falsification fixtures
matching the existing `falsification-live-v1` schema. Filtering
includes only divergence labels (LLM-DIVERGED / KERNEL-LLM-DIVERGED
/ CACHE-DRIFT); skips KERNEL-LLM-AGREE / STRICT-WITNESSED (no
calibration signal) and KERNEL-ONLY (LLM unparseable, not a
supervised-correction sample).
Idempotent: sorted by audit-event seq, so re-running against the
same qa.db produces byte-equal fixture files. The existing
fixture-digest discipline stays valid.
Makefile: `make bench-witness-divergence` (override default
qa.db / output path via WITNESS_QA_DB / WITNESS_OUT env-vars).
Closes the divergence → calibration data loop the witness ticket
imagined: every LLM hallucination on a canonical-shape question
becomes a supervised-correction fixture downstream prompt
improvements can grade against.
#000030 Phase 7 demo — function-sampled@v1 end-to-end
======================================================
`bench/scripts/demo_plot.py` — closes the loop on opencompletion's
activity24-math-plot.yaml. SymPy expression → quantized
integer-vector signature (canonical bytes) → optional matplotlib
PNG. Canonical bytes are the proof; PNG is just a downstream view
of the same evidence.
$ make demo-plot Q='sin(x)' PNG=/tmp/sin.png
Output JSON contains canonical_bytes_sha256 + canonical_bytes_preview
+ canonical_bytes_total_chars + grid metadata + the optional png_path.
matplotlib is gated — when absent, --png prints a warning to stderr
and skips the render; the canonical bytes still print. Tests skip
the PNG-presence assertion via `pytest.importorskip("matplotlib")`.
Public docs polish (#7)
========================
- docs/_source/bench.rst: updated fixture-count narrative (~660 →
662 default tasks + ~110 math π* fixtures); `make` quick-reference
now lists all per-π* 5S targets (tabular, calculus-limit/series,
linear-algebra, function-sampled) plus bench-real-shard,
bench-fork-baseline/score, bench-witness-divergence.
- docs/_source/v8-fork-score.rst: CLI section gained --out flag
documentation + a Make-harness sub-section covering
bench-fork-baseline / bench-fork-score / FORK_PARENT/CHILD/REPORT
env-vars.
Tests
=====
- tests/test_witness_to_5f.py — 8 new tests covering the audit-event
write (3) + extractor logic (5).
- tests/test_demo_plot.py — 6 new tests covering canonical-bytes
determinism + equivalence-class collapse + matplotlib gating.
Full suite: 1624 passed, 37 skipped (was 1568; +56).
