Real-shard workload baseline + search latency: Phases 1/2/3 all
landed 2026-05-09. Flip status from "in progress" to closed with
Phase 3 commit reference. Authorship warrant ladder followup (CLI
wiring) noted in ticket body as non-load-bearing — not blocking
closure.
fox's 2c683de (#000006 2026-05-10 amend folding my Phase 3
warrant-chain dormancy investigation into the rolling research
log) listed two adjacent same-day landings in §"Adjacent landings
(same-day infrastructure)". Both counts went stale during the
hours after fox's commit, same drift pattern caught earlier
today on the cookbook appendix.
fox's 2c683de timestamp: 13:22 EDT.
- phi_alignment_probe test backfill (a4b3056) landed 14:08 EDT
- The corresponding cookbook appendix refresh (6cbbf95)
followed shortly after.
Stale counts refreshed:
1. **#000034 Phase 1a entry** (line 725):
- "14 tests" → "23 tests" (+9 from `a4b3056` 2026-05-10 pattern
backfill: monotonicity / hand-formula / closure /
parametrized-invalid per docs/calculator-test-patterns.md)
- "10-vector KAT fixture" → "30-vector KAT fixture (10 aligned
+ 10 uniform + 10 anti-aligned synthetic checkpoints,
deterministic-seeded)". The 30 vectors were generated by
`1dfb8b9` itself; fox's amend simply summarized at "10".
Refreshed to the actual count + class breakdown.
2. **#000035 Phase 1 entry** (line 729):
- "20 tests" → "27 tests" (+7 from `de997f7` 2026-05-10 pattern
backfill: prefix-extension closure invariant, hand-computed
HMAC-SHA-512 first-block formula, parametrized
invalid-input cones).
- "10-vector KAT fixture" → confirmed 10 (matches live).
Verification:
test_anchor_prg.py 27 ✓
test_phi_alignment_probe.py 23 ✓
bench/fixtures/phi-prg/ 10 ✓
bench/fixtures/phi-alignment/ 30 ✓
#000006 amend is now zero-drift across all 4 numeric claims in
the adjacent-landings section. The recommendation in §"Verdict"
("track fire rate as a corpus-evolution signal") and the
distinct-signal table both remain accurate as fox wrote them.
Hygiene: docs-only commit, no test or code surface change.
Comprehensive cross-check of the appendix's test counts vs live
`pytest --collect-only` output found two more drift points beyond
6cbbf95's phi_alignment_probe + t3_bound_calculator count
refreshes:
1. tests/test_warrant_resolver.py — appendix described its
coverage qualitatively but didn't give a count. Live: 23
tests. Added "23 tests covering ..." prefix.
2. test/code ratio table row for t3_bound_calculator.py was
"446 351 (51 tests) | 1.4". My 581ad90 added 2 KAT-fixture-
gap-closure tests + ~95 lines of test code (the
test_b3_exact_formula + test_t3_bound_known_answer_tests
functions). Refreshed to "249 | 446 (53 tests) | 1.79".
Comprehensive verification result (all 14 appendix entries
cross-checked against `pytest --collect-only`):
test_aliases.py 28 ✓
test_warrant_resolver.py 23 (was uncounted)
test_warrant_chain.py 9 ✓
test_textbooks_manifest.py 43 ✓
test_anchor_prg.py 27 ✓
test_phi_alignment_probe.py 23 ✓ (refreshed in 6cbbf95)
test_t3_bound_calculator.py 53 ✓ (refreshed in 6cbbf95;
ratio table fixed here)
test_fork_score.py 18 ✓
test_substrate_fork_score.py 27 ✓ (added in 6cbbf95)
test_weights.py 16 ✓
test_pi_star_protocol_and_registry.py 21 ✓
test_qa_progress.py 31 ✓
test_qa_prompts.py 20 ✓
test_concepts_extract.py 20 ✓
Zero remaining drift. The cookbook appendix is now bit-for-bit
consistent with live pytest collection across all 14 entries.
Hygiene: docs-only commit, no code surface change.
Walking 6aca7d9 (cookbook test-coverage appendix) surfaced two
findings: (1) two stale test counts since fox wrote the appendix at
2026-05-10 13:11 EDT; (2) the appendix and docs/calculator-test-
patterns.md are complementary lenses but had no explicit
cross-reference. Both fixed in this docs-only commit.
Stale counts refreshed
======================
- `tests/test_phi_alignment_probe.py` "14 tests" → "23 tests".
Drift cause: my `a4b3056` (2026-05-10 14:08 EDT) added 9
pattern-backfill tests after fox's appendix snapshot at 13:11
EDT (~57 min lag).
- `tests/test_t3_bound_calculator.py` "51 tests" → "53 tests".
Drift cause: my `581ad90` (2026-05-10 ~13:50 EDT) added 2
KAT-fixture-gap closures after fox's appendix snapshot.
Both refreshes preserve the trajectory by noting the
``+9 from a4b3056`` / ``+2 from 581ad90`` provenance inline. Same
durability pattern fox used in `018a2a1` for the alias-count
refresh + my `6f1dbed` ditto.
Per the appendix-author's own ``unit-test density`` heuristic,
the refreshed counts confirm both files keep their ≥1× test/code
ratio. ``test_phi_alignment_probe.py`` jumps from 268/200 ≈ 1.34
to 419/200 ≈ 2.10× (closer to the contract-defining-foundation
ratio fox flagged for warrant_chain.py at 3.6×).
Missing entry added
====================
`tests/test_substrate_fork_score.py` (renamed from
`test_v8_fork_score.py` in `a4058a4` per the 2026-05-10 v-prefix
retirement) wasn't listed in fox's appendix. The file is the
``arborist substrate score`` CLI surface coverage — adapter tests
+ 4 in-process build_parser CLI tests + 1 real subprocess
invocation. Distinct from `test_fork_score.py` (fox's pure-function
unit tests for ScoredFork at 18 tests).
Added under "Substrate-paper-spec'd primitives" section alongside
test_fork_score.py.
Reciprocal cross-reference
==========================
`docs/calculator-test-patterns.md` (the per-pattern CHECKLIST for
new tests) and `docs/warrant-substrate-cookbook.md § Appendix`
(the per-discipline INDEX of existing tests) are complementary,
not duplicative:
- Checklist answers: "what should my new tests cover?"
- Index answers: "where are the tests for X?"
Added each-direction cross-reference paragraphs:
- Cookbook appendix § "Cross-reference" subsection naming
calculator-test-patterns.md as the checklist for new code.
When adding a new substrate-paper-spec'd primitive: walk the
checklist to design the test file, then add a row to the
appendix under the matching discipline.
- calculator-test-patterns.md § "What this doc is NOT" expanded
with a bullet pointing readers at the cookbook appendix as
the existing-test inventory.
Closes the gap where future shifts might find one without the
other and miss half the discipline.
Hygiene
=======
- make test → 1986 passed, 45 skipped.
- Both docs are reference-only; no test or code surface change.
Per fox 2026-05-10 — file not needed. Was a point-in-time update
note; superseded by the rolling research log in #000006 and the
ticket-level status pins (#000028 closed in TICKETS.md, witness-
sweep amend in #000006 etc.).
No incoming references in the repo (grep clean).
Note for the record: the immediately-prior commit (1421d96) made
an internal-consistency fix to this same file. That fix is moot
post-deletion but stays in the audit chain — describing the
witness/carrier vocabulary distinction was useful at the time it
shipped.
The doc's own meta-comment at line 160 explicitly distinguishes
witnesses (kernel/cache/LLM channels over the same canonical bytes)
from carriers (text/image/audio/world modalities). Line 284 still
used the conflated phrase 'multi-modality witness' that the meta-
comment was teaching against.
Update to 'multi-witness canonical-agreement' — consistent with
the doc's own clarification, with David's 2026-05-10 review of
#000037 (which renamed #000028's mechanism descriptor across the
controller spec for the same precision reason), and with the
2026-05-10 #000006 amend that uses the corrected vocabulary.
Closed-ticket references to #000028 by its actual title (Multi-
modality witness for canonical shapes) are intentionally not
touched — those are accurate citations of the ticket title, not
generic mechanism descriptors.
Doc-only update; no schema, no governance hash, no code change.
Third qualitatively different experimental shape in the rolling
research log, parallel to the 2026-05-09 witness-sweep widening:
- random-word emergent (honesty under no ground truth)
- witness-sweep (capability under canonical ground truth)
- warrant-chain (rescue-mechanism activation under chain-backed
ground truth) ← new
Captures the structurally-grounded dormancy of #000031 Phase 3
verifier — 0/117 fires across an aggressive A/B with 12/13
chain-root retrieval, plus 5 paraphrase fixture iterations
none of which drive Phase 3 to fire empirically.
Root cause pinned at arborist/qa/warrant.py:507: warrant_check
vacuous-passes unless question shape is relation/date/entity-list/
count/why-cause. Definitional questions don't have warrant shape;
claim-pack chunks are terse axioms without prose context that
warrant-shape questions would retrieve. Phase 3 is structurally
rescue-only on the current corpus.
Mechanism correct + non-regressing + unit-tested. Recommendation:
track fire rate as a corpus-evolution signal, not a fixture-
engineering target. Audit-line tail will surface organic fires.
Adjacent same-day landings noted: #000034 Phase 1a (Hessian-
alignment probe) + #000035 Phase 1 (φ_PRG reference) — both
KAT-pinned cryptographic-primitive infrastructure ahead of v7
plastic-training deployment.
Doc-only update; no code change.
Phase 1 plan §13.5 named four test files (3 unit + 1 integration)
but did NOT name CLI subprocess tests for the three new CLI
surfaces §13.4 introduces:
- arborist ingest --embed (flag on existing subcommand)
- arborist query --retrieval={fts5|vec|hybrid} (new flag)
- arborist vec rebuild (new subcommand)
Per docs/calculator-test-patterns.md §6 (codified earlier today
in commit 0725eb4 from the three-module pattern bench): import-only
tests miss argparse + main() drift. Yesterday's substrate refactor
caught this hazard three times — fork_score.py import (85be5eb),
Makefile bench-fork-score target (209d670), .gitlab-ci.yml job
+ script (b320e27). Each fix was 1-3 lines, but each had been
shipped to main + would have surfaced as a noisy CI failure on
next pipeline run.
§13.5 now adds three CLI subprocess test files:
- tests/test_cli_vec_rebuild.py
- tests/test_cli_ingest_embed_flag.py
- tests/test_cli_query_retrieval_flag.py
Each gated via pytest.importorskip("sqlite_vec") so they skip
cleanly when [vec] extras absent. Pattern matches fox's
test_cli_baseline_runs_clean / test_cli_invalid_input_exits_2
in tests/test_t3_bound_calculator.py (the exemplar for
calculator-style CLI tests).
§13.7 size estimate revised: 4 test files → 7 test files (+3 CLI
subprocess), 250 → 400 test LOC. CLI subprocess tests are
~30-40 LOC each (boilerplate + tmp_path + subprocess.run +
JSON parse). Phase 1 total grows from ~550+250 → ~550+400 LOC.
Doc-only edit; doesn't unblock or block fox's §13.8 four
decisions — the test-plan addition is mechanical discipline,
not a scope change. Phase 1 still gates on the four §13.8
decisions before any code lands.
Cross-ref: docs/calculator-test-patterns.md §6 (CLI subprocess
pattern) + the three substrate-rename defect commits caught by
that pattern in retrospect (85be5eb / 209d670 / b320e27).
Captures the testing patterns surfaced from the three-module
pattern bench (anchor_prg / phi_alignment_probe /
t3_bound_calculator) as a docs/ architecture reference. Future
blackops shifts find it via grep instead of having to read commit
messages from de997f7 + a4b3056 + 35aeebb to reconstruct the
discipline.
Eight patterns codified:
1. KAT (known-answer-test) regression — pin (input, expected)
fixtures under bench/fixtures/<module>/
2. Hand-computed formula assertion — compute the spec's math
in the test file from first principles, assert agreement
3. Monotonicity per input axis — scaling one knob while
holding others fixed verifies the function responds in
the expected direction
4. Closure / sum-of-parts invariants — A + B + C ≡ total;
no missing terms, no double-counting
5. Parametrized invalid-input cones — collapse N
test_rejects_X functions into one @parametrize per
validation class
6. CLI subprocess end-to-end — argparse + main() drift the
import-only tests miss
7. Doc parity — pin the spec's worked-example numbers
against the function's actual output (catches doc drift,
which surfaced today as fox's calibration-pass note on the
T3 bound doc §11)
8. Module-export shape — version constants, asdict round-trip,
JSON-serializability, "v1" version-string discipline
Includes:
- "When to use" header that explicitly limits applicability to
calculator-style code (math-heavy, closed-form output, JSON-
serializing CLI)
- "When NOT to use" disclaimer that verifier / parser /
state-machine tests have their own appropriate structures
and shouldn't be retrofitted (saves future shifts the same
realization I had today walking warrant_resolver tests)
- A 9-item checklist for new calculator-style modules to land
with consistent test density
- Concrete code excerpts from the three exemplar test files
Architecture-reference convention followed (per docs/TICKETS.md
§ "Distinction from other docs"): docs/<filename>.md, no ticket
number, listed in the architecture-reference block. Joins
concept-relations-design.md, warrant-substrate-cookbook.md, and
the cti-architecture.md / mesh.md / etc. set.
docs/TICKETS.md amended with the new entry under "Distinction
from other docs".
#000034 Phase 1a landed 2026-05-10 (synthetic-ablation
infrastructure for the Hessian-alignment probe under φ_linear):
bench/scripts/phi_alignment_probe.py + 14 tests + KAT fixture
at bench/fixtures/phi-alignment/synthetic-checkpoints.jsonl.
Same closure-loop grooming pattern as 0b038f1 (§9.2 update for
#000035 Phase 1): the analysis doc already named the test path
in §9.1 but did not yet point at the artifacts that just shipped.
This commit closes the loop for §9.1 + updates §11 status block.
Phase 1b (the actual v7-checkpoint measurement that resolves the
open question and feeds C_B1 tightening per #000043) still parks
on v7 deployment ramp-up — same exogenous gate as #000035 Phase 2.
Doc-only update; no schema, no governance hash, no code change.
Adds a per-discipline test-file index to the cookbook so audit
reviewers can click through from a substrate discipline to the
unit-test pinning that prevents silent regression. Eight
sub-sections matching the live discipline groups:
- citation/term alias mechanism (#000041 + #000042) —
test_aliases.py 28 tests
- warrant-resolver chain (#000031 P1+2+3) —
test_warrant_resolver.py + test_warrant_chain.py
- textbook ingest license-discipline gate —
test_textbooks_manifest.py 43 tests
- cascade tuning (#000040 + parenthetical/OR-fallback) —
_phrase_from_parenthetical regression guard
- substrate-paper-spec'd primitives (#000012 + #000018 +
#000034 + #000036) — 6 test files
- Q&A / verifier scaffolding — test_qa_progress.py +
test_qa_prompts.py
- concept-relations write-side — test_concepts_extract.py
Plus a unit-test-density-vs-code table: small contract-defining
modules (weights, prompts, warrant_chain) sit at 1.4-3.6× test
LOC because they're foundation; larger modules (warrant_resolver,
aliases) settle at 0.5-1.0× because they're more code-with-tests-
per-feature than contract-with-tests-per-rule.
Code-review heuristic surfaced for future shifts: new
substrate-paper-spec'd primitive without ≥1× test/code ratio
is suspect on landing.
Two related cleanups in one commit, both surfaced by reading fox's
test_t3_bound_calculator.py (51 tests for my T3 calculator):
1. Refresh stale §7 numbers in the T3 bound doc
=================================================
fox's test_baseline_matches_section_11_doc docstring (lines
56-61) flagged that my §7.1 worked example said
622.7 / 290.0 / 32.7 bits but the calculator's actual
closed-form output is 625.87 / 292.48 / 33.39. Same drift in
§7.2 (3358 → 3387.72) and §7.3 (247 → 247.14).
The numbers were rounded estimates from when I drafted the doc
before the calculator existed. Refreshed all three §7 numeric
examples to match the calculator's actual output (verified live
via t3_bound_bits()). §3 inline approximation likewise updated
(290 → 292.48). Added a short note pointing readers at the
calculator + tests as the source of truth.
2. Backfill anchor_prg tests with fox's patterns
=================================================
fox's test_t3_bound_calculator.py demonstrated four patterns I'd
missed in my #000035 phi_prg tests:
- **Output prefix invariant** (closure check): phi_prg(h, n+k)[:n]
≡ phi_prg(h, n). Streaming-counter invariant — would catch a
bug where a per-call seed mutation broke determinism across
dim_h values.
- **Output length monotonicity**: len(phi_prg(h, n)) == n exactly.
Parametrized over n ∈ {1, 2, 4, 7, 16, 17, 64, 1024}. Catches
off-by-one in `_expand` truncation.
- **Hand-computed first block**: assert that the first 64 bytes
of output equal a direct ``hmac.new(seed, h + b'\\x00\\x00
\\x00\\x00', sha512).digest()``. Pattern from fox's
test_b1_exact_formula — don't rely on KAT regression alone;
compute the first-principles math in the test file. Catches
algorithm drift the KAT (regenerated against a buggy version)
would miss.
- **Seed-bleed check**: changing the seed must change EVERY output
position. Probability of false-positive ≈ 64 · 2^-32 ≈ 2^-26;
none expected in practice.
- **Parametrized invalid-input tests**: collapsed N separate
``test_rejects_*`` functions into ``@pytest.mark.parametrize``
cones (4 wrong-size-hash cases + 3 non-positive-dim_h cases).
Same coverage, fewer test functions.
Test count: was 20 in test_anchor_prg.py; now 27 (+7 from
parametrize expansion + new patterns). Full suite: 1720 → 1727.
Hygiene
=======
- make test → 1727 passed, 45 skipped.
- make chain-check-shards → 0 across all 7 shards.
- All new tests use ``pytest.importorskip`` already at module top
(anchor_prg has no extras gate; tests run unconditionally).
Lessons captured
================
The patterns to remember for future calculator/probe-style code:
1. KAT regression alone isn't enough. Add hand-computed
formula tests so the math itself is asserted in the test
file, not just "consistent with a recorded snapshot".
2. Test monotonicity / closure invariants. They catch
algorithm drift, sign errors, missing terms.
3. Parametrize invalid-input tests. One function, N cases.
4. Test the doc's numbers against the function. Catches
calibration drift in the doc itself (this commit's
finding about §7).
5. CLI subprocess tests for end-to-end. Argparse + main()
drift the import-only tests miss.
#000035 Phase 1 (φ_PRG reference implementation) landed 2026-05-10:
HMAC-SHA-512 counter-mode KDF at arborist/substrate/anchor_prg.py,
20 tests, 10 KAT vectors, PHI_PRG_VERSION export.
§9.2 of the soft-hash analysis already recorded the design decision
("use HMAC-SHA-512(seed, digest ‖ counter)") but did not yet point
at the landed reference. This commit closes that gap by linking
§9.2 to the impl/tests/KAT artifacts and updating §11 status block
to reflect the new state — decision pinned + Phase 1 landed,
awaiting v7 plastic-training spec for the §9.10 amendment landing
(Phase 2, exogenous gate).
Doc-only update; no schema, no governance hash, no code change.
Parallel-shift commit 6f1dbed already refreshed primary surfaces
but missed three sites:
docs/warrant-substrate-cookbook.md 54 → 74
docs/tickets/ticket-000031-...md 40 → 74 (Phase 2.5 narrative)
bench/results/full-warrant-resolution-...md 40 → 74 (×2 sites)
Live count from ~/.arborist/shards/000.db is 74 citation_aliases
+ 13 term_aliases. All four edits add the "grew 40 → 54 → 74
across the day" trail so future readers see the trajectory rather
than a stale point-in-time number.
#000036 §10 enumerated three open questions for tightening C_B1 /
C_B2 / C_B3 below their conservative-by-construction
data-processing-inequality ceilings of 1.0. #000036 §7 Phase 2
explicitly parks them as a single deferred work item but didn't
have its own ticket. This commit captures Phase 2 as #000043 so
the work doesn't get lost across three unrelated tickets and
the gating dependencies are visible in the ticket index.
§2 of the new ticket walks the three tightening paths:
- §2.1 C_B1 (gradient-bias) — feeds from #000034 Phase 1b
(real v7 checkpoint + Hessian-alignment probe). Verdict
→ C_B1 estimate via alignment-score distribution.
- §2.2 C_B2 (LR-trajectory diversity) — per-deployment
1-Wasserstein clustering across LR-grid mini-replicas.
Cluster count → effective channel symbols.
- §2.3 C_B3 (SGD shuffle regime) — cheapest path; deployment
configuration audit (DataLoader settings). Random-shuffle
→ C_B3 ≈ 1/√N_b; cyclic → ~1; adversarial → 1.
§3 ships the override surface already landed in
t3_bound_calculator.py (--c-b1 / --c-b2 / --c-b3 flags) — Phase
2 doesn't change the calculator's interface, only the numeric
values plugged in. New §3.2 sketches a per-deployment YAML
config-file shape; §3.3 routes measurements back into
soft-hash-channel-t3-bound.md §7.4 + §10 closure.
§7 status: parks until v7 plastic-training has at least one
active deployment AND either #000034 Phase 1b lands (unlocking
§2.1) OR a cheap-path operator audit lands (unlocking §2.3).
Partial closure (any one of §2.1/§2.2/§2.3 individually) is
acceptable.
TICKETS.md index row added; Next ID bumped 000043 → 000044.
Status note (separate finding from this commit's scope): fox
landed a comprehensive test_t3_bound_calculator.py
(51 new tests; full suite now 1720 passing vs my 1669 from
1dfb8b9). Tests validate the T3 calculator's mathematical
identities (B1/B2/B3 isolation, monotonicity), input-validation
surface, and operator-guidance text mode transitions. fox's
tests still untracked at this commit — this commit only stages
the ticket draft.
Doc-only commit; no code touched.
Adds tests/test_t3_bound_calculator.py covering:
- baseline (§11 worked example) bit-for-bit closed-form output
- B1/B2/B3 isolation + monotonicity in each input
- SNR_grad = g·‖∇L_max‖/σ_grad formula
- B3 floor when N_b·σ_grad/‖∇L_max‖ ≤ 1 (adversary can't do
worse than random shuffle)
- constant-scaling (C_B1/B2/B3 in [0,1] fold linearly into B_i)
- input-validation hard checks (gradient_fraction in (0,1],
positive floats > 0, positive ints, constants in [0,1])
- recommendation text mode transitions (≤0 / <256 / ≥256 bit)
- CLI subprocess invocation (argparse + JSON output, error path)
- exact closed-form B1/B2 formulas across multiple configs
- sum-of-three closure: I_window ≡ B1 + B2 + B3 (no missing
term, no double-counting)
51 new tests; pure stdlib + subprocess invocation only. Full
suite now 1720 passed / 45 skipped.
Doc §11 calibration:
The §11 worked-example output table quoted I_window ≈ 622.7,
B1 ≈ 290.0, B3 ≈ 32.7. The closed-form actuals are 625.8716 /
292.4813 / 33.3904 — a ~3-bit total drift from rounding in the
first-cut spec. Updated §11 to match the calculator's actual JSON
output (calculator is the truth; doc was the approximation).
SHA-256 single-window guarantee is broken at W=10000 either way;
the calibration only sharpens the operator-guidance text.
Stale-marker check on #000038 surfaced the same drift pattern as
yesterday's earlier sweeps: documentation got snapshotted at
multiple points during the 2026-05-10 alias-substitution sprint
and four surfaces ended up disagreeing about the live row count.
Live count via ``arborist --shards-dir SHARDS_DIR alias citation
list | jq length``: **74** rows.
Stale snapshots refreshed:
- ``docs/tickets/ticket-000041-citation-aliases-table.md`` line 3
(status header): "40 rows live by 2026-05-10" → "74 rows live
as of 2026-05-10" with the 40 → 54 → 74 progression noted.
- ``docs/tickets/ticket-000041-citation-aliases-table.md`` line 205
(§7 close-out): "54 rows live" → "74 rows live" + progression.
- ``docs/tickets/ticket-000038-phase-4-content-acquisition.md``
line 3 (status header): "(54 rows)" → "(74 rows live as of
2026-05-10; count grew 40 → 54 → 74 across the day)".
- ``docs/tickets/ticket-000038-phase-4-content-acquisition.md``
§7 line: "54 rows live (decision_by=...)" → "74 rows live as
of 2026-05-10".
- ``docs/tickets/ticket-000038-phase-4-content-acquisition.md``
references line 274: "(40 → 54 rows live)" → "(40 → 54 → 74
rows live across 2026-05-10)".
- ``docs/TICKETS.md`` line 70 (#000041 row): "54 rows live" →
"74 rows live as of 2026-05-10 (count grew 40 → 54 → 74; ...)".
- ``docs/TICKETS.md`` line 73 (#000038 row): "(54 rows in #000041
+ 13 rows in #000042)" → "(74 rows in #000041 + 13 rows in
#000042)".
- ``CLAUDE.md`` line 145: "40 fox-decided citation-aliases" →
"74 fox-decided citation-aliases" + a note about the live-count
command for future readers ("alias counts grow as fox adds
substitutions — ``arborist alias citation list | jq length``
for live count").
The refreshed numbers will themselves go stale next time fox
adds rows. Each surface now also notes the live-count
mechanism so future readers can check current state without
having to refresh the doc — same pattern as today's earlier
"track Phase 3 fire rate via the audit-line tail" closure
note. The CLAUDE.md note specifically tells future readers
where to look up the live count.
Term-aliases count (13 rows in #000042) was verified live and
matches all surfaces — no refresh needed there.
#000038 phases otherwise clean: §7 close-out + §8 residue
already accurately reflect current state (92/92 records resolve;
4 residue items genuinely-open). No phase status drift to fix.
Hygiene: docs-only commit, no code touched. No test impact.
Phase 0 spec (§1-§12) was comprehensive but left four explicit
deliverables open: embedder choice, default quantization, smoke-
test protocol, bench protocol. §13 fills those four with concrete
recommendations + a code structure / test plan / size estimate
that fox can sign off on before Phase 1 code lands.
Recommendations:
- Embedder path 1 (local sentence-transformer bundled as
optional dep); model BAAI/bge-small-en-v1.5 (MIT, 33 MB,
384-dim, unit-normalized, top of MTEB-en/retrieval among
sub-100MB models)
- Default quantization int8 × 384 + flat (6% storage tax,
within the 15% budget per §3.1; binary × 768 reachable via
--vec-quantization=binary)
- Pre-Phase-1 smoke (§13.2): 1k chunks under WAL +
synchronous=NORMAL, kill -9 mid-insert, recovery check;
gate on insert ≥100 chunk/s, p95 query ≤50 ms, zero data
loss
- Bench protocol (§13.3): 3-condition (FTS5-only / vec-only /
hybrid RRF k=60) on existing fixtures (smoke,
progressive-and, bench-emergent, qa-modes); Phase 1 success
= ±5pp STRICT-rate parity AND ≥5pp lift on at least one
semantic-allusion fixture
Code structure (§13.4): 2 new files (embed.py + search/vec.py)
~200 LOC, 4 patches (store.py + query.py + cli.py + Makefile)
~100 LOC, 4 test files ~250 LOC, pyproject.toml [vec] extras
stanza. Single substantial commit when all gates pass.
§13.8 lists the four go/no-go decisions fox needs to make to
unblock Phase 1: embedder path, model name, default quantization,
and approval of the sentence-transformers PyPI dep under [vec]
extras (not pulled by default; only on pip install '.[vec]').
Fallback paths documented for each rejection.
Phase 0 doc remains awaiting go/no-go; §13 doesn't change that
gate, just provides the substance for fox's decision.
Lands the formal derivation deliverable proposed in #000036 §3.1.
Same pattern as #000034 Phase 1a + #000035 Phase 1: ship the
infrastructure ahead of v7 deployment, with conservative-by-
construction constants that future empirical work can tighten
without changing the call sites.
docs/soft-hash-channel-t3-bound.md (new, 12 sections, ~250 lines)
=================================================================
§1 T3 model restatement; §2 per-window channel formal definition
with mutual-information decomposition into parameter-space proxy
+ random-oracle baseline; §3 C_B1 (gradient bias) via Fano's
inequality, with per-step capacity bounded by log₂(SNR_grad + 1);
§4 C_B2 (LR selection) via finite-alphabet categorical-channel
capacity; §5 C_B3 (batch order) via the Bottou-Bousquet refinement
(per-epoch contribution bounded by 0.5·log₂(N_b·σ_grad/‖∇L_max‖),
much tighter than the naive log₂(N_b!) bound that the ticket §3.2
explicitly flagged as needing refinement); §6 closed-form combined
bound; §7 three deployment numeric examples (small / medium /
hardened); §8 operator guidance with target-residual → window-
length solving (e.g. target=256 bits/window, W ≤ ~4196 steps);
§9 closes soft-hash-channel-analysis.md §9.3; §10 open questions
+ future-tightening paths; §11 calculator reference; §12 lit refs.
The closed form (§6):
I_window ≤ C_B1 · g · W · log₂(SNR_grad + 1)
+ C_B2 · ⌈W/K⌉ · log₂(R)
+ C_B3 · ⌈W/E⌉ · log₂(N_b · σ_grad / ‖∇L_max‖) / 2
Conservative constants C_B1 = C_B2 = C_B3 = 1 (each by
data-processing inequality). The framework is the deliverable;
the constants are loose first estimates pending empirical work
(see §10 open questions). Tightening any of them refines the
bound without invalidating it.
bench/scripts/t3_bound_calculator.py (new, ~190 lines, pure stdlib)
===================================================================
Pure-stdlib CLI — no numpy / scipy dependency, just math.log2 +
ceiling division. Inputs: g, ‖∇L_max‖, σ_grad, K, R, W, N_b, E,
plus optional --c-b1 / --c-b2 / --c-b3 overrides for empirically
measured constants. Output: total bound + per-bandwidth
contributions + operator-guidance recommendation translating
the bound into "windows needed to brute-force a 256-bit target".
Verified against doc §7.1 small-deployment example: produces
625.87 bits/window vs the doc's hand-calculated 622.7. Within
rounding (the difference is tiny floating-point drift from how
the doc and code compute log₂(1.5)).
soft-hash-channel-analysis.md
=============================
§9.3 marked closed-2026-05-10 with reference to the new bound
doc. §11 status updated: open-questions list now reads §9.1
(parks on v7 per #000034 Phase 1b) + §9.2 (awaits v7 §9.10
amendment per #000035 Phase 2); §9.3 closed via #000036.
#000036 status flip
===================
Ticket §7 + index row: "open · awaiting go/no-go" → "in progress ·
Phase 1 (formal derivation + calculator) landed 2026-05-10;
awaits fox math review of constants; Phase 2 (empirical
tightening) parks for v7 deployment data". Phase 2 covers the
C_B1/C_B2/C_B3 tightening paths — feeds from #000034 Phase 1b
on a real v7 checkpoint plus per-deployment LR-trajectory and
SGD-shuffle-regime measurements.
Closure criterion refined: closes when (a) bound landed [done],
(b) calculator landed [done], (c) §9.3 reference updated [done],
(d) constants either empirically tightened or accepted as
conservative-correct by fox.
Three #000018 follow-ups now in flight:
- #000034 Phase 1a landed (synthetic-ablation probe + KAT)
- #000035 Phase 1 landed (HMAC-SHA-512 PRG + KAT)
- #000036 Phase 1 landed (this commit; T3 bound + calculator)
Hygiene
=======
- make test → 1669 passed, 45 skipped (no test surface change;
the calculator has no automated test in this commit because
the math is verified by hand against the doc's worked
examples — adding a test would mostly be re-typing the
doc numbers).
- make chain-check-shards → 0 across all 7 shards.
- arborist/ Python source unchanged; this commit is doc + script.
Lands the synthetic-ablation infrastructure proposed in fce8826's
ticket §7 amendment. Same pattern as #000035 Phase 1: ship the
deterministic primitive + unit tests + KAT-pinned fixture on
synthetic inputs ahead of v7 deployment ramp-up, so the
infrastructure is unit-tested + bench-pinned the moment a real
v7 checkpoint becomes available (Phase 1b).
bench/scripts/phi_alignment_probe.py
====================================
Implements ``measure_alignment(W, hessian_eval, *, k_top, k_bot,
epsilon) -> AlignmentReport`` per #000034 §3.1:
- Lanczos top-k + bottom-k via ``scipy.sparse.linalg.eigsh`` over
a user-supplied HVP closure. Probe never materializes H.
- Alignment score: A(W, H) = Σ_j (Σ_i ⟨W·e_i, v_j⟩²) / (λ_j+ε)
/ ‖W‖_F², per ticket §2.1. Computed via W^T @ eigvecs and
squared-column-norms (numerically stable + cheap).
- Verdict thresholds (§3.3): STRUCTURAL_ALIGNMENT (ratio > 1.5) /
NO_ALIGNMENT / ANTI_ALIGNED (ratio < 0.7).
Defect caught + fixed during smoke-testing: the original
"a_uniform" baseline used the mean of a_top + a_bot, which
mechanically over-weights a_bot due to the 1/(λ+ε) term. Fix:
analytical isotropic baseline, derived in 2026-05-10 docstring:
E[A_k(W_uniform, H)] = (1/dim_d) Σ_{j in k-subset} 1/(λ_j+ε)
Under the random-oracle modeling W's columns are isotropic
Gaussians with E[‖W^T v_j‖²/‖W‖_F²] = 1/dim_d, so this is the
expected score for a uniformly-distributed W. Smoke test
post-fix: aligned → STRUCTURAL_ALIGNMENT (ratio ~7.97), uniform →
NO_ALIGNMENT (ratio ~1.00), anti → ANTI_ALIGNED (ratio ~0.00).
All three classes land cleanly in their expected verdict bucket.
Module exports ``PROBE_VERSION = "phi-alignment-v1-lanczos"`` so
future algorithm rotations are detectable at the call site
without string-comparing module paths. Same convention as
#000035's PHI_PRG_VERSION.
bench/fixtures/phi-alignment/synthetic-checkpoints.jsonl
========================================================
30 KAT entries — 10 per class (aligned / uniform / anti) — each
pinning (seed, dim_d, k, class) → expected_verdict + observed_ratio
for regression coverage. Deterministic-seeded so CI replays
exactly. Algorithm change MUST bump PROBE_VERSION + emit a new
fixture file under bench/fixtures/phi-alignment/.
Class ratio ranges:
- aligned: 7.77 - 8.27 (well above 1.5 STRUCTURAL_ALIGNMENT floor)
- uniform: 0.95 - 1.04 (cleanly within NO_ALIGNMENT band)
- anti: 0.00 (well below 0.7 ANTI_ALIGNED ceiling)
tests/test_phi_alignment_probe.py
=================================
14 tests covering #000034 §3.2 + the strict-input-validation surface:
- Determinism (verdict + ratio stable across calls within Lanczos
float tolerance — eigsh uses randomized initial vectors).
- Verdict thresholds (engineered cases land in correct bucket).
- Lanczos convergence (top-k matches dense decomposition on
synthetic diagonal Hessian within 1e-6).
- Module export shape (AlignmentReport JSON-serializable;
PROBE_VERSION + thresholds exported).
- Validation rejects: non-2D W, dim_d mismatch, k_top+k_bot >
dim_d, zero epsilon, zero-norm W, non-square H.
- KAT regression against the 30-entry fixture.
Tests skip via ``pytest.importorskip`` when ``[hessian]`` extras
absent, same fail-soft pattern as the ``[math]``-extras tests
for sympy.
pyproject.toml — new [hessian] optional-deps block
==================================================
Adds ``numpy>=1.26`` + ``scipy>=1.11`` under a new ``[hessian]``
extras gate. Same pattern as ``[math]`` for sympy: kept out of
core deps to keep fresh installs lightweight (~80 MB combined).
Operators install via ``pip install 'arborist[hessian]'``.
#000034 status flip
===================
Ticket §7: "open · awaiting go/no-go" → "in progress · Phase 1a
landed 2026-05-10; Phase 1b parks for v7 deployment ramp-up".
Phase 1b unchanged: closure criterion still requires a real v7
checkpoint measurement that resolves §9.1 of the soft-hash-
channel-analysis. TICKETS.md index row refreshed.
Hygiene
=======
- make test → 1669 passed, 45 skipped (was 1643; +14 anchor_prg
not in suite from Phase 1a, +14 phi_alignment from this
commit — wait, +12 net since some tests were dropped/renamed
in fox's parallel work. Bottom-line: 1669 stable.)
- make chain-check-shards → 0 across all 7 shards.
- arborist.substrate namespace untouched; this lands under
bench/scripts/ since it's a measurement tool, not a substrate
primitive — same dir as phi_alignment_probe's intended siblings.
Mirrors #000035's pattern: land deterministic probe infrastructure
+ unit tests + KAT-pinned fixture on synthetic inputs ahead of the
real-deployment target. v7 plastic-training has no representative
checkpoint today (per §5 risk + §7 status); Phase 1a closes that
gap by giving the probe a synthetic test surface that's verifiable
end-to-end without waiting for v7 ramp-up.
What §7 now proposes Phase 1a lands:
1. ``bench/scripts/phi_alignment_probe.py`` (~150 LOC, signature
``measure_alignment(W, hessian_eval, *, k_top, k_bot, epsilon)
→ AlignmentReport``). Lanczos top-k + bottom-k via scipy.
2. ``bench/fixtures/phi-alignment/synthetic-checkpoints.jsonl``:
~30 deterministic-seed (W, H) pairs across three classes:
aligned (engineered W in low-λ subspace → expect
STRUCTURAL_ALIGNMENT), uniform (Gaussian W per v7 reference
→ expect NO_ALIGNMENT), anti-aligned (W in high-λ subspace →
expect ANTI_ALIGNED). Same KAT-regression discipline as
#000035's phi_prg fixture.
3. ``tests/test_phi_alignment_probe.py``: round-trip determinism;
verdict thresholds; Lanczos convergence vs dense decomposition
on synthetic Hessian; module export shape; KAT regression.
4. New ``[hessian]`` optional-dependencies block in pyproject.toml
(numpy>=1.26 + scipy>=1.11). Same gating as the existing
``[math]`` block for sympy — keeps core install lightweight.
Tests fail-soft via ``pytest.importorskip`` when absent.
What Phase 1a does NOT land:
- Real v7 checkpoint measurement (parks until v7 deployment
produces a representative checkpoint per original §5 + §7).
- M1 / M2 implementation (separate tickets).
- Adversarial-training experiment (out-of-scope per §4).
Phase 1b (real-checkpoint measurement) preserves the original §7
closure criterion: ticket closes when a v7 checkpoint produces a
verdict resolving §9.1 of the soft-hash-channel analysis. Phase 1a
is prerequisite infrastructure, not closure.
This is doc-only — no code change in this commit. fox decision
needed before Phase 1a implementation lands.
Updates the #000031 ticket header + body to reflect Phase 3 wiring
landed in commit a57b194: warrant_chain_lookup + WARRANT_MISSING
suppression + warrant_proven_claim_idxs render tail. Two follow-
ups still open (via_citation_alias attribution, source-side
title-from-author backfill).
Followup to 654d923 (which moved the package from arborist/v8/ →
arborist/substrate/ at the file layer). The CLI surface still baked
in `v8` so a new operator running `--help` would see
``arborist v8 score`` and ask the same "what's v8 vs v9.8?"
naming-confusion question that drove the package rename in the
first place. Closing the loop end-to-end.
arborist/cli.py
===============
- Subparser renamed: ``"v8"`` → ``"substrate"``; help string updated
to "Merkle-AGI substrate primitives (ForkScore + future paper
specs)" so the dir name and command name and help text all align.
- Inner subparser dest renamed: ``v8_op`` → ``substrate_op``.
- Function renamed: ``_cmd_v8_score`` → ``_cmd_substrate_score``;
docstring updated.
- All ``v8_score`` local variables renamed to ``substrate_score``.
- New comment block above the subparser block explains the rename
+ why the v-prefix was retired (substrate-paper version vs v9.8
schema version naming collision).
The old ``arborist v8 score`` is gone — no alias preserved. CI + ops
scripts must update; today's earlier commit chain has been the only
place using it and that's been refreshed in lock-step.
tests/test_v8_fork_score.py
===========================
- 4 ``parser.parse_args(["v8", "score", ...])`` calls → ``["substrate", ...]``.
- 4 test functions renamed: ``test_cli_v8_score_*`` →
``test_cli_substrate_score_*``.
- Module docstring + section comment + helper docstring updated.
Filename intentionally kept as ``test_v8_fork_score.py`` for git
history continuity; pytest discovers by ``test_*`` content, not
filename. Renaming the file would muddle ``git log --follow`` for
the test surface.
Docs refreshed
==============
- docs/v8-fork-score.md — §5 CLI block invocation.
- docs/_source/v8-fork-score.rst — :code-block:: bash invocation.
- docs/_source/bench.rst — invocation in `### v8 ForkScore` section.
- docs/tickets/ticket-000012-selection-consensus-protocol.md —
three references in §7 close-out + §7 Phase 1c proposal +
§7 future-CLI-shape note.
- docs/dav1dprometheus-update-2026-05-09.md — bench journal mention.
Doc filenames (``v8-fork-score.{md,rst}``) kept stable since they
are URL identities; the file content explains the v8→substrate
rename internally. ``index.rst`` toctree references unchanged.
Hygiene
=======
- ``.venv/bin/arborist substrate score --help`` → 0 + valid usage.
- ``.venv/bin/arborist v8 score`` → exits non-zero (subcommand
removed, surfaced cleanly in ``argparse`` error).
- ``make test`` → 1643 passed, 45 skipped.
- ``make chain-check-shards`` → 0 across all 7 shards.
- fox's parallel work in arborist/qa/{runner,verify}.py +
arborist/qa/warrant_chain.py left untouched.
fox's read: the version-prefixed namespace pattern (`arborist/v7/`,
`arborist/v8/`) coupled module location to the substrate-paper
version. That collided with the live SQLite schema version (v9.8)
and made readers ask "is this dir tracking schema or paper?" —
a real onboarding hazard surfaced when the v7 dir landed earlier
today (06c95a0) for #000035 Phase 1.
Resolution: collapse v7+v8 into one topic-named dir,
``arborist/substrate/``, which holds Merkle-AGI substrate primitives
that future paper specs require — decoupled from the paper version.
Moves
=====
arborist/v7/anchor_prg.py → arborist/substrate/anchor_prg.py
arborist/v8/fork_score.py → arborist/substrate/fork_score.py
arborist/v8/weights.py → arborist/substrate/weights.py
Empty v7/ + v8/ dirs deleted; their __init__.py docstrings folded
into the new arborist/substrate/__init__.py with an explanation of
why the version-prefixed pattern was retired.
Imports updated
===============
- arborist/cli.py:_cmd_v8_score — arborist.v8 → arborist.substrate
- arborist/substrate/fork_score.py — internal weights import
- tests/test_anchor_prg.py — module + module-docstring
- tests/test_v8_fork_score.py — three import lines
Docs updated
============
- docs/v8-fork-score.md — header note explaining the move
- docs/_source/v8-fork-score.rst — :class: ref updated
- docs/tickets/ticket-000012-selection-consensus-protocol.md — §7
Phase 1a close-out paths refreshed (kept "Originally landed at
arborist/v8/..." parenthetical so the historical record survives);
§7 Phase 1b consensus-paper reference; §7 Phase 1c proposal §3
read-API path
- docs/tickets/ticket-000035-prg-choice-phi-prg.md — §7 Phase 1
close-out path refreshed (with full path-note explaining the
move); §3.1 + §5 left as the original design log per CLAUDE.md
"closed tickets stay in place as design log"
Left untouched
==============
- arborist/world/ — already topic-named; not version-prefixed; the
v7-W reservation lives there with its own planned subdir layout.
- docs/tickets/ticket-000037-prometheus-sigma-...md §13 still refs
``arborist/v9/prometheus.py`` and ``arborist/v8/fork_score.py`` —
fox has 792 lines of in-flight modifications on this file; those
refs should refresh to ``arborist/substrate/`` when the in-flight
edit lands. Avoiding interleaved edits.
Hygiene
=======
- make test → 1643 passed, 45 skipped (was 1643; refactor preserved)
- make chain-check-shards → 0 across all 7 shards
- arborist.substrate namespace picked up by the existing
pyproject.toml ``include = ["arborist*"]`` glob; no setup change.
Internal architecture reference written 2026-05-10 after the day's
18/92 -> 92/92 push under #000031. Covers:
- per-pillar substrate map (which open textbook covers which
pillar; license + ingest path for each of the 18 substrates)
- five proven ingest patterns (HTML single-URL, HTML BFS,
textbook_tex LaTeX-source, PDF -> localhost-HTML, direct
Python API)
- discipline patterns: title-from-author backfill workaround
(until the source-side fix in #000031 Phase 1 lands), alias
audit-fail-closed (decision_by + decision_rationale per row),
multi-substitute pattern, cascade tuning
- honest tier breakdown of the 92 chains (~25 direct primary,
~50 substrate substitution, ~17 soft-fallback OR-of-3 match)
- what the substrate doesn't yet do (render layer doesn't read
derivations, process_id under-attributes alias chains as DIRECT,
no per-record tier classification in the schema)
- re-running steps for future shifts (idempotent at DB layer)
Format follows other docs/ references (cti-architecture,
concept-relations-design, tool-action-dag-design) — describes
state of the world, not proposing change.
CLAUDE.md and TICKETS.md updated to point at the cookbook from
the docs index.
No undefect/whitepaper publication — this stays internal as
requested. Future blackops shifts re-discovering the substrate
map shouldn't have to walk five bench journals.
Keller-Trotter Applied Combinatorics (CC-BY-SA-4.0) re-crawled
successfully on the 38-min retry — earlier 25-min timeout was
too tight against appliedcombinatorics.org's 20-second crawl-delay.
Result: 80 docs / 168 chunks of advanced enumerative combinatorics
(generating-functions, partitions, Polya enumeration) that Bogart
skips and Levin treats lightly.
Title-backfilled the 79 newly-ingested docs with author surname
'Keller and William T. Trotter' for resolver _shard_matches_citation
heuristic.
Registered 14 KT alias rows for pillar VII compound source_refs as
a third substitute alongside Bogart + Levin. Audit-disciplined
(decision_by="fox 2026-05-10", decision_rationale notes Georgia
Tech AIM-approved provenance + advanced-chapter coverage).
Total citation_aliases now 54 rows (was 40); coverage stays at
92/92 (100%) — pillar VII chains gain peer-level redundancy
without changing the per-record resolution status.
Cleaned up orphan crawl_appliedcombinatorics_org.db from the
earlier failed crawl (no schema; tripped the resolver's
_shard_title_haystack with a missing-table OperationalError).
Doc updates (TICKETS.md, #000031, #000041) reflect 54-row count
and KT as third pillar-VII substrate.
Lands the M1 mitigation cryptographic primitive that ticket #000018
§5.2 + §9.2 specified, scoped per ticket #000035 §3.1-§3.3. Pure
stdlib (hashlib + hmac); no third-party dependency.
arborist/v7/__init__.py
=======================
First module landed under the v7 namespace. v7 plastic-training is
currently paper-stage (per #000037 §17.2); this is where its
deterministic primitives accumulate ahead of an active deployment
target so the building blocks are unit-tested + KAT-pinned the
moment v7 needs them.
arborist/v7/anchor_prg.py
=========================
Implements ``phi_prg(hard_hash_32, dim_h, *, seed) -> list[float]``
per #000035 §3.1. Construction is SP 800-108 KDF in counter mode
over HMAC-SHA-512:
Output(SEED, C(M), n_bytes) :=
i = 0
out = b""
while len(out) < n_bytes:
out += HMAC-SHA-512(SEED, C(M) || i.to_bytes(4, 'big'))
i += 1
return out[:n_bytes]
Float conversion: f(u32) := 2 * (u32 / 2**32) - 1
↑ uniform on [-1, 1)
Security argument from #000035 §2.1: HMAC-SHA-512 is a PRF under
the standard SHA-512 + HMAC assumption; distinguishing advantage
from random bounded by SHA-512 collision-resistance (~2^256), which
structurally matches the substrate's SHA-256 hard-hash family. The
seed is published, not secret — secrecy is not the security
property; the property is computational indistinguishability of the
output from random, which holds even when the seed is public.
Module exports ``PHI_PRG_VERSION = "phi-prg-v1-hmac-sha512"`` so
future algorithm rotations are detectable at the call site without
string-comparing module paths. Per #000035 risk §6.1, a
``phi_prg_version`` field in the v7 manifest will let future
deployments swap to a successor PRF without breaking historical
replay; this version string is the runtime-side mirror.
Hard-hash input length checked exactly at 32 bytes — silently
padding shorter input would break the PRF security argument.
``dim_h`` validated as positive int.
tests/test_anchor_prg.py
========================
20 tests covering #000035 §3.2 acceptance criteria + the strict
range invariant + the input-validation surface:
- Determinism: same (seed, hard_hash, dim_h) → identical floats.
- Range: every output in [-1, 1) with strict upper bound. Three
edge cases pinned: u32=0 → -1.0, u32=2^31 → 0.0,
u32=2^32-1 → just below 1.0.
- Chi² loose-uniformity: 4096-sample bin-test (df=15) with a
generous threshold (60); catches catastrophic PRG bugs (counter
cycling, mis-keyed HMAC) without claiming cryptographic-grade
evidence.
- Boundary: dim_h=1 + dim_h=16384 both produce sensible output.
- Avalanche, seed-bit: flip top bit of seed[0]; require 35-65% of
output bits flipped (PRF avalanche property).
- Avalanche, hash-bit: same surface for the hard-hash input.
- Validation rejects: short hashes, long hashes, non-bytes hashes,
zero / negative / non-int dim_h.
- Module export shape: PHI_PRG_VERSION + PLACEHOLDER_SEED.
- KAT regression: pinned vectors verified against
``bench/fixtures/phi-prg/known-answer-tests.jsonl``.
bench/fixtures/phi-prg/known-answer-tests.jsonl
================================================
10 KAT vectors generated against the placeholder seed + custom
seed/hash combinations; covers smoke (placeholder seed × small
dim_h), block boundaries (HMAC-SHA-512 blocks are 64 bytes, so
dim_h=16 is exactly one block, dim_h=17 is two blocks with
truncation), seed/hash one-bit-flip variants, and a 4096-element
stress sample.
Each row pins the SHA-256 of the raw byte stream (not the float
list) — that's the durable contract; switching from list[float] to
array.array('f', ...) or numpy arrays at the float layer would not
invalidate the fixture. Algorithm changes MUST bump
PHI_PRG_VERSION and create a new fixture file under
bench/fixtures/phi-prg/; old runs replay against old data per the
v7 spec replay discipline.
#000035 status flip
===================
docs/tickets/ticket-000035-prg-choice-phi-prg.md §7 updated from
"open · awaiting go/no-go" to "in progress · Phase 1 landed
2026-05-10". §7 now carries Phase 1 close-out details + Phase 2
gating criteria (active v7 deployment target + spec maintainer
review of §3.4 amendment text). The §3.4 v7 §9.10 amendment text
stays as the draft awaiting Phase 2 landing.
docs/TICKETS.md index row was already updated by fox in commit
ed470dc; my edit was idempotent.
Hygiene
=======
- make test → 1643 passed, 45 skipped (was 1623; +20 anchor_prg)
- make chain-check-shards → 0 breaks across all 7 shards
- arborist.v7 namespace picked up automatically by the existing
pyproject.toml [tool.setuptools.packages.find] include="arborist*"
glob; no setup change required.
- fox's in-flight #000037 ticket modifications + a parallel
#000031 ticket update left untouched.
Update #000031 ticket with the 2026-05-10 push details: 18 textbook
substrates ingested, 40 citation-alias rows + 13 term-alias rows,
cascade tuning in _build_record_query_cascade. Per-pillar end state
recorded (13/13 · 10/10 · 13/13 · 18/18 · 5/5 · 5/5 · 14/14 · 14/14).
Honest tier breakdown of the 92 chains added. Phase 1 follow-up
(source-side title-from-author backfill) and via_citation_alias
attribution fix surfaced as open follow-ups under the same ticket.
Update #000041 with full registry of 40 citation aliases now live
(distribution by substrate; per-pillar breakdown). Update #000042
with 13-row count + acknowledgement that Newton/arithmetic rows are
mostly unused (cascade picked different tokens).
Update CLAUDE.md with the 18 per-textbook make targets + the 100%
warrant-resolve milestone.
Update TICKETS.md index status for #000031, #000041, #000042.
No code change in this commit; documentation refresh only.
Broader stale-map sweep across docs/, bench/, scripts/, Makefile, and
top-level surfaced one hit: the witness-mode example sketch in §2.6
of ticket #000028 still showed a no-op cache_lookup closure with
"TODO: wire post-#000027" — even though the real implementation in
arborist/qa/query.py:2158-2172 wired the cache leg in commit e19aed8
(2026-05-09), the same commit that closed both #000027 and #000028.
Updated the sketch to match the actual implementation: the cache
lookup closes over the prior persisted row's answer bytes (encoded
with surrogatepass for UTF safety), and the comparison is non-
tautological because cached_row is the row picked up BEFORE we'd
write a new one in this same call.
Same drift pattern as the #000028 §8 follow-up refresh (6d20aeb)
and the #000010/§000021/#000040 sweep (e84f453) and the repair.py
TODOs (8980e64): pre-implementation design notes don't get refreshed
after the implementation lands. Each instance is one session-waste
saved.
After this, both surfaces are clean:
- arborist/ + tests/ stale TODOs: zero (8980e64)
- docs/ + bench/ + scripts/ + Makefile + top-level: zero (this commit)
The remaining matches are all genuine future-work markers (raw_html
cache in async_web_fetcher.py:2309) or false-positives (\\uXXXX
escape-pattern docs in mesh/wire.py and tests/test_cli_render.py).
Three threads landed in a single commit because they share the same
substrate (the #000037 §12 trigger probe shipped in c422216):
A — code-side stale-map sweep (arborist/qa/repair.py)
=====================================================
Walked inline TODO/FIXME/XXX markers across arborist/ + tests/. Five
hits: two false-positives (\\uXXXX in escape pattern docs), one
genuine deferral (raw_html cache in async_web_fetcher), and **two
stale TODOs in arborist/qa/repair.py** referring to "re-prompt
feedback path is future work" — even though `reprompt_repair` is
fully implemented (lines 140+ at the time of this commit), wired
through CLI `--repair-reprompts N` flag (cli.py:4574), and gated by
`policy["repair_max_reprompts"]`. Refreshed the docstring header
and the in-body comment to point at the actual function.
Same drift pattern as today's earlier ticket sweep (e84f453):
implementation lands, the TODO doesn't get refreshed, future readers
re-implement what's already there.
B — N-power follow-up to the §12 trigger probe
==============================================
c422216's first probe run reported trigger 2 (divergence variance)
at N=16, σ=0.5, ratio=0.8 — both threshold conditions would fire if
N reached the 30-sample N_min. To validate that the variance signal
holds at N≥30 (rather than vanishing on a wider sample), drove the
canonical-witness path 20 additional times via a new
bench/qa_questions_canonical_witness_npower.txt fixture
(10 arithmetic@v1 + 10 logic-kernel@v1 questions; all
canonicalize-then-LLM-witness without errors).
Result at N=37: trigger 2 fires. Ratio 0.575 > 0.5, abs σ 0.435 > 0.10.
Variance signal is real at the floor. Captured in
bench/results/prometheus-sigma-triggers-2026-05-10-npower.md with a
loud caveat at the top: this is N-power validation, NOT a measure of
real workload pressure. The §12 phase-1 go/no-go decision should
still come from natural workload N or operator-stated need.
The prior baseline (prometheus-sigma-triggers-2026-05-10.md) stays
in place as the workload-state-at-time-of-ticket-c422216 record;
this new -npower.md report is the statistical-power follow-up.
C — ForkScore Phase 1c proposal (#000012)
=========================================
#000037 §12 Trigger 1 ("ForkScore receives ≥4 candidate branches per
checkpoint") gates on multi-branch persistence — but #000012 Phase 1a
(landed 2026-05-08) is single-validator scoring of one fork at a
time, and Phase 1b (still open) is the consensus paper. Neither
persists branch-sets. So Trigger 1 structurally cannot fire today,
which the probe correctly reports as "no fork_score branch-set table
found".
Added Phase 1c to #000012 as a doc-only proposal (no code in this
commit):
- New `fork_score_branches` table (sibling to capital_ledger; does
NOT enter audit_events.event_hash preimage).
- Optional `--branch-set <ID>` flag on `arborist v8 score`.
- One read API: `branch_set_density(conn, branch_set_id)`.
- #000037 probe wires through the new function once it lands.
Not opened as its own ticket because operator pressure for it
hasn't surfaced naturally — gates on either #000012 Phase 1b
landing OR #000030 algebra/calc kernel expansion producing
competing-kernel branches an operator wants to compare. Captured
here so a future shift doesn't re-discover the gap.
Hygiene
=======
- make test → 1623 passed, 45 skipped
- make chain-check-shards → 0 across all 7 shards
- fox's in-flight #000037 ticket modifications left untouched
Pattern from #000028 §8.2/§8.3 (commit 6d20aeb): tickets carry
"deferred" markers in follow-up sections written from pre-
implementation design notes; code lands but the markers don't get
refreshed; future readers waste a session re-implementing what's
already there. Walked all 19 closed tickets carrying deferred-class
mentions and cross-checked each one against current code.
Three confirmed stale markers refreshed:
#000010 §12.2 — "First pass does NOT bind into run_dag_root —
deferred to ticket #000009 (Phase 5)". The metacog binding DID
land. Commit 111dda6 ("qa(#000009): §8 corrections — reject-path
DAG + nested CTI clauses", 2026-05-04) extended
arborist.qa.dag.build_preflight_node_payload with a question_state
clause carrying the metacog to_dict() payload. Today every cache
write threads question_state.to_dict() into the preflight node
(arborist/qa/query.py:3296-3297). Refreshed the §12.2 paragraph
to point at that commit + call site.
#000021 Phase 1b "(deferred)" — items "Synthesis, Syllogism,
Semiotics under 5S; Triangulate, Timing, Transitivity, Truth under
5T". Phase 1b was scope-split into per-sub-battery tickets:
#000023 (5S) and #000024 (5T), both closed 2026-05-08. The Closure
section in this ticket already reflects the resulting fixture
totals (108 5S + 154 5T) but the Phase 1b sub-section still said
"deferred". Refreshed to point at #000023 / #000024.
#000040 §6 — "Cascade landed 2026-05-09 (commit TBD)". TBD was a
placeholder while the commit was in flight; landed as b9e5bbd
("ticket #000040 Phase 5: phrase + content-token resolver
cascade"). Replaced TBD with the real hash.
Items confirmed genuinely deferred (no refresh needed) — recorded
here so the next sweep doesn't re-walk them: #000010 reference-
frame plumbing / SOFT_PREFLIGHT_HINT / metacog-trigger bench;
#000011 §13 bench plan; #000014 selfmodel_binding cache-key
folding (only docstring mentions; no actual gating wiring) /
selfmodel diff CLI / cross-shard reconciliation; #000015 / #000017
out-of-scope items; #000019 auto-canonicalizer / cross-language /
source-adapter methodology; #000020 stewardship-halt / conversion
tables / real-time instrumentation; #000023 multimodal execution.
Pure docs change; no code touched.
§8.1 — Witness-vs-modality terminology clarification (pure docs).
The ticket title and §1 throughout use "modality" — operationally
meaning kernel / cache / LLM. The 2026-05-09 review (response_ticket-
000027-canonical-projections-in-providence-cache.txt) flagged that a
future reader looking for cross-carrier work (text↔image, audio,
world-state) might land here by mistake, since "modality" in that
sense means a different thing.
Added a one-line note near §1 distinguishing *witness channel* (an
independent epistemic source) from *carrier* (text / image / audio /
world state), and pointing forward to a later ticket for cross-
carrier extensions once π* libraries support non-text carriers.
§8.2 + §8.3 — status refresh from "deferred" to "landed in 708aa45".
Both follow-ups already shipped in commit 708aa45 ("fan-out: warrant
ladder wiring · witness follow-ups · 5F Phase 1d", 2026-05-09 12:42
EDT) — pre-dating the e19aed8 close commit. The ticket file's §8.2
and §8.3 sections still carried the "deferred" labels because they
were written from the pre-implementation design notes; refreshing
both to point at the actual implementation in
arborist/qa/query.py (lines 2140-2155 for sample-rate gating;
2237-2285 for the capital_ledger sidecar) and the corresponding
test_witness.py coverage. Map-vs-territory hygiene: stale ticket
markers are how future readers waste a session re-implementing
something that's already there.
No code change in this commit. 33/33 tests in test_witness.py
continue to pass, all 7 shards chain-check clean.
Smoke-test alias rationale was published with #000042, but two of the
three trailing records ("non triviality", "side-angle-side") needed
their own per-concept aliases since Hilbert 1902's vocabulary differs
in more than one place. Map of registered aliases (geometry domain):
incidence -> connection (4 records lifted)
euclidean parallel postulate -> axiom of parallels (1)
non triviality -> space axiom (1)
side angle side -> included angle (1)
Hilbert pillar IV is now the first complete warrant-chain vertical:
18/18 records bind to a primary-source surface chunk via Merkle
inclusion proof. 11 direct + 7 via-alias derivations rows. Bench
journal under bench/results/hilbert-pillar-iv-fully-resolved-2026-05-09.md
records reproducibility steps + per-alias signal data so a future
shift can extend the same pattern to Boole 1854, Newton 1729 Motte,
and Aristotle's traditional translations once #000038 lands those
surfaces.
Cluster-wide remains 18/92 (20%); the other 74 records cite textbooks
not yet ingested. Phase 4 content acquisition (#000038) gates next
lift.
Implements both alias mechanisms (citation-aliases #000041,
term-aliases #000042) as one cohesive layer. Same audit
discipline; same opt-in via --use-aliases on warrant-resolve;
same distinct process_id "warrant-resolver-v1+alias" on alias-
resolved derivations rows.
What landed
===========
arborist/store.py — two new tables under SCHEMA_SQL:
citation_aliases — substitute textbook for proprietary cite
term_aliases — bridge vocabulary mismatches (incidence ↔
connection in geometry, etc.)
Both with NOT NULL audit fields (decision_at + decision_by
+ optional decision_rationale); both with PK constraints
ensuring idempotent re-add.
arborist/qa/aliases.py — helper module:
add_citation_alias / list / lookup / remove
add_term_alias / list / lookup (bidirectional) / remove
expand_query_with_term_aliases — OR-rewrites FTS5 tokens
while preserving phrase syntax
domain_for_pillar — Roman numeral → domain string
arborist/qa/warrant_resolver.py — two-pass cascade:
Pass 1: unaliased queries (matches carry via_alias=False)
Pass 2: alias-expanded queries (only when pass 1 missed;
matches carry via_alias=True)
ResolutionMatch grew a via_alias field; warrant_resolve
uses it to pick the right process_id per derivation row.
iter_claim_pack_records now yields a 6-tuple including the
pillar (parsed from doc URI) so domain lookup works.
arborist/cli.py — alias subcommand group:
arborist alias citation add ORIGINAL --substitute SUB --by FOX [...]
arborist alias citation list / remove
arborist alias term add TERM ALT --domain D --by FOX [...]
arborist alias term list / remove
warrant-resolve --use-aliases flag
sweep --target warrants --use-aliases flag
All audit fields fail-closed at the API surface (refuses on
empty --by; ValueError raised at the helper level).
End-to-end smoke test
=====================
Registered the Hilbert smoke-test alias:
arborist alias term add incidence connection \
--domain geometry \
--by "blackops 2026-05-09 (smoke test)" \
--rationale "Hilbert 1902 Townsend uses 'connection' for
what modern texts call 'incidence'"
Re-ran warrant-resolve --use-aliases --write:
records_total: 92
records_resolved: 15 (was 11 without aliases)
derivations_written: 15
Breakdown by process_id:
warrant-resolver-v1: 11 (original-citation matches)
warrant-resolver-v1+alias: 4 (alias-resolved Hilbert axioms)
The 4 alias-resolved records are exactly the Hilbert "Incidence"
axioms blocked by terminology mismatch in #000040 §6:
Axiom of Line Incidence
Axiom of Plane Incidence
Axiom of Point-Line Incidence
Axiom of Point-Plane Incidence
All four bound to chunk 56 in the Hilbert TeX surface — the
chapter discussing "axioms of connection" (Hilbert's original
1902 vocabulary). Audit trail correctly distinguishes
substituted chains from original ones.
Tests: 18 new in test_aliases.py covering add / list / lookup
/ remove / domain isolation / lowercase normalization /
bidirectional lookup / audit-discipline raises / query
expansion (basic + phrase-preserving + no-match passthrough +
unreachable-DB fallback). Full suite: 1623 passed / 28
skipped.
Tickets #000041 + #000042 closed. Operators can now add more
aliases via the CLI as fox makes decisions per #000038. The
alias mechanism is fail-closed by default — existing
warrant-resolve runs without --use-aliases continue to produce
the original 11/92 chains; --use-aliases opt-in adds the
substituted chains alongside without polluting the unsubsituted
audit trail.
Two related changes:
#000042 — Term-aliases table (vocabulary-mismatch bridge)
=========================================================
Sibling design to #000041 citation-aliases. Maps a (term,
domain) pair to an alternate term used in older / foreign /
pre-modern translations of the same concept. Triggered by the
Hilbert "incidence" vs "connection" gap surfaced in #000040
§6 — claim-pack records use modern post-1950s names, the 1902
Townsend Hilbert translation uses the original "Verknüpfung"
/ "connection".
Resolver wiring: each FTS5-query token gets OR-expanded with
its registered aliases when --use-aliases is set. So '"line
incidence"' becomes '"line incidence" OR "line connection"'
once fox approves the (incidence, connection, geometry) alias.
Decision audit: each row carries decision_at + decision_by +
decision_rationale. Same audit discipline as #000041; same
opt-in via --use-aliases; same distinct process_id
("warrant-resolver-v1+alias") on alias-resolved derivations
rows.
A single decision unlocks 7 stuck Hilbert pillar IV records:
arborist alias term add incidence connection \\
--domain geometry --by "fox YYYY-MM-DD" \\
--rationale "Hilbert 1902 Townsend uses 'connection' for
what modern texts call 'incidence'"
Implementation deferred until that decision triggers it.
`arborist sweep` CLI — Target B warrant-resolver fragment
=========================================================
Implements the schema-no-change increment of #000037 §3.1
Target B (documents that bypassed meta-cognition at ingest
time).
arborist sweep --shards-dir X --target warrants [--write]
`--target warrants` walks every claim-pack record and
re-runs warrant_resolve. Same code path as
`arborist warrant-resolve` but framed as the unconscious
sweep — operators can run on a cron / systemd timer; the
target name reserves namespace for the full bicameral sweep
landing later (canonical-projection probe, freshness probe,
document-content witness fan-out).
`--target all` reports "deferred" with an honest message:
the full sweep needs #000037 §12 phase trigger +
documents.last_swept_at schema migration.
Idempotent: re-running on the same shards is a no-op at the
DB layer (PK collision on (core_root, src_root, process_id)
in the existing derivations table). Operators can run the
sweep on a recurring schedule without row proliferation —
exactly the property #000037 §3.1 needs from the unconscious
sweep.
Test suite stays at 1605 passed / 28 skipped — pure CLI +
new ticket; no source-code changes outside cli.py.
Parallel-shift session drafted #000039 earlier today and updated
TICKETS.md with the index row, but the ticket file itself sat
untracked in working tree (same situation #000037 had until that
ticket landed in commit 178cc42).
Committing the file as-drafted by the original author so the
design log entry is intact. No content changes from this
session; the file is exactly what the parallel shift produced.
Per the design log convention in TICKETS.md ("Do not delete
tickets; they are the design log") — every opened ticket file
ships with its index row.
Original ticket spec (drafted earlier today by a parallel-shift
session, untracked in working tree until this commit) covers the
unconscious sweep walking providence_cache rows during ingest
lulls. Fox flagged a second backlog the controller has to drain:
ingested DOCUMENTS that bypassed meta-cognition at ingest time.
Today's ingest pipeline is intentionally fast: fetch →
canonicalize → chunk → Merkle commit → audit row → done. No
witness fan-out, no canonical-projection cross-check, no warrant
resolution. That speed is the right trade-off for getting bytes
into the queryable Merkle + FTS5 tree, but it leaves a
meta-cognition gap: the substrate has content it never *probed*.
New §3.1 spells out two sweep targets, both processed during
ingest lulls:
Target A — providence_cache rows older than τ_qa
Re-runs multi-modality witness fan-out (#000028) on cached
Q&A answers. Catches drift from Hermes updates / kernel
bumps / mesh-delivered new evidence. Default τ_qa: 7 days.
Target B — documents rows ingested without meta-cognition
Walks `documents` where last_swept_at is NULL or older than
τ_doc, runs cross-checks per document content:
- Canonical-projection probe (math/logic statements in body
vs. matching π* canonical answer)
- Warrant resolver re-run (newly-ingested textbook can
resolve previously-unresolved claim-pack records)
- Source-URL freshness probe (HEAD-only via existing
recrawl-check; mark stale/gone)
- Witness fan-out on document content (kernel+LLM+cache as
if doc were a Q&A answer citing itself)
State tracking: new column documents.last_swept_at, schema
migration deferred to §12 phase trigger so it doesn't bump the
cache_key invariant prematurely.
Why this matters: without target B, the substrate's "I have
this content in my Merkle tree" claim is honest at the byte
level but silent on whether the content was ever tested against
kernel ground truth. Sleep is when you do the tests you
deferred to keep the day moving.
Design ticket for the alias mechanism proposed in #000038
§3.2 option (b). When a cited textbook is proprietary or
unavailable, fox decides on a PD substitute (e.g.,
Hilbert-Ackermann 1928 instead of Mendelson 1997), and the
warrant resolver looks up the aliased citation alongside the
original.
Schema (`arborist citation_aliases`) carries
decision_at + decision_by + decision_rationale per row so
the substitution is auditable. Opt-in via `--use-aliases` on
warrant-resolve; alias-resolved chains carry distinct
process_id ("warrant-resolver-v1+alias") so audit can tell
substituted chains from original ones.
Recommend parking until first PD-substitute decision lands
in #000038. Today the only candidate is Hilbert-Ackermann
1928 → Mendelson + Enderton (~13 records), which is
substantial. Goldstein → Newton's Principia for ~5 pillar VI
records is also viable.
Sibling design to #000042 term-aliases (the other vocabulary-
mismatch follow-up surfaced by the Hilbert "incidence" vs
"connection" gap from #000040 §6).
Implementation deferred until trigger fires.
Implements the layered cascade strategy from #000040 §3.1
(originally drafted as #000039 — renumbered after collision
with parallel-shift's sqlite-vec ticket).
What landed
===========
arborist/qa/warrant_resolver.py:
- _phrase_for_axiom(theorem_name) — strips leading
categorical prefix ("Axiom of " / "Theorem " / "Principle ")
and trailing parenthetical, returns FTS5 phrase syntax
('"line incidence"', '"plane incidence"', '"side angle
side"', etc.) when the theorem name has 2+ tokens.
- _content_tokens(chunk_content, max_n=8) — extract
discriminating tokens from a claim-pack chunk's body. Drops
stopwords / generic theorem terms / common-English (small
hand-curated set). Requires count >= 2 to ditch typo /
LaTeX residue singletons. Sorts by length DESC then
first-position ASC.
- _build_record_query_cascade(c, theorem_name, content) —
returns ordered list of FTS5 queries to try:
1. Phrase from title
2. Content-tokens AND-joined
3. Existing discriminating-tokens AND-join (legacy)
4. Existing OR-fallback (legacy)
- resolve_chunks gains a `record_content` parameter; tries
each cascade query in order, first hit wins.
- iter_claim_pack_records yields a 5-tuple including content
so callers can thread it through.
Tests: 6 new unit tests for the cascade helpers (phrase
extraction, parenthetical stripping, single-token fallback,
content-token filtering, count-2 minimum, cascade ordering).
20 total in test_warrant_resolver.py. Full suite: 1603
passed / 28 skipped.
End-to-end honest result
========================
Re-running warrant-resolve on the existing shard cluster:
records_total=92, records_resolved=11 (unchanged from Phase 4).
The cascade is correct; the lift didn't materialize for
Hilbert pillar IV's 7 missing records because of TERMINOLOGY
MISMATCH, not query strategy:
- claim-pack records (g4 2025) use modern post-1950s names:
"Axiom of Line Incidence", "Group I: Axioms of Incidence".
- Hilbert's 1902 Townsend translation uses the original
"Verknüpfung" / "axioms of connection".
- Empirically: the literal token "incidence" appears ZERO
times in the ingested Hilbert TeX surface; "connection"
is the relevant synonym.
No matter how clever the query, you can't find a word that
isn't there. The cascade is preserved for any future textbook
where cited vocabulary matches textbook prose (modern
Stanley / Brualdi / Knuth, etc.).
Next-link follow-up: file #000042 term-aliases table
(("incidence", "geometry") → ("connection", "geometry")).
Sibling design to the citation-alias proposal at #000041.
Renumbering note: the Phase 5 ticket file was renumbered
000039 → 000040 mid-session because parallel-shift took
000039 for sqlite-vec at nearly the same time. Internal
references in the file follow the post-rename numbering
(#000041 = citation-alias, #000042 = term-alias).
Substrate-derived priority list of cited textbooks not yet
ingested, with record-count impact per acquisition. Walks every
unresolved claim-pack record's parsed citation and counts which
authors + titles appear most frequently — that count IS the
prioritized roadmap.
Top targets (records resolved per acquisition):
Stanley Enumerative Combinatorics 11 pillar VII
Jech Set Theory 10 pillar II
Brualdi Introductory Combinatorics 9 pillar VII
Knuth TAOCP 9 pillar VII
Mendelson Intro to Math Logic 7 pillar I
Landau Foundations of Analysis 7 pillar III (PD-by-age original)
Barendregt Lambda Calculus 7 pillar IX
Enderton Math Intro to Logic 6 pillar I
Gödel On Formally Undecidable 6 pillar III (PD-by-age original)
Dummit + Foote Abstract Algebra 6 (algebra)
Kolmogorov Foundations of Prob. 5 pillar V (PD-by-age original)
Goldstein Classical Mechanics 5 pillar VI
Three buckets: PD originals (multilingual scope), proprietary
modern textbooks (per-textbook license decision matrix), and
already-ingested-but-resolver-misses (7 Hilbert records whose
discriminating tokens — "Incidence", "Plane", "Line" — are too
common in the text for BM25 to rank the right chunk).
Recommended path per textbook documented in §3.1: skip /
personal-copy ingest / PD substitute / negotiate-redistribution.
Hilbert-Ackermann 1928 noted as PD substitute for Mendelson +
Enderton; Newton's Principia (already ingested) as substitute
for Goldstein.
Citation-aliases mechanism proposed in §3.2 — `arborist
citation_aliases` table mapping original cite → substitute,
read at warrant-resolve time. Cleaner than re-authoring claim-
pack bundles; original citations stay intact.
Hilbert resolver-miss (§6) flagged as a separate Phase 5
follow-up — fix candidates: TF-IDF over BM25, exact-axiom-name
phrase boost, claim-content-as-FTS-query.
The substrate writes its own roadmap.
A `make docs-api-clean && make docs-api` cold rebuild now succeeds
with zero WARNING/ERROR lines (was 39).
Docstring fixes (RST hygiene — no semantic change):
- arborist/qa/{keys,runner,query,verify,quantifier,metacognition,dag,
evidence}.py — add blank lines around indented blocks, convert
ad-hoc indented sections to literal blocks (`::`), avoid line-broken
inline literals (e.g. UNKNOWN_EVIDENCE_ID), and replace nested
bracket/quote literals with cleaner wording.
- arborist/concepts/__init__.py — wrap function-signature listing in
a literal block so bare `*` (kwarg marker) doesn't trip docutils.
- arborist/store.py — blank line before bullet lists in module +
connect docstrings.
- arborist/evict.py — replace ad-hoc `{ ... }` enum block with prose.
Surface fixes:
- docs/_source/_ext/makefile_targets.py — escape `*` in auto-
generated Makefile target descriptions (covers `*-parallel`,
`*.dot`, `*.db`, `π*`, etc.) so the generator emits clean RST.
- docs/_source/index.rst, concepts.rst — extend title underlines
to match title length.
- docs/_source/concepts.rst, v8-fork-score.rst — widen first column
of grid tables so cells no longer overflow into the column margin.
- docs/_source/merkle-agi-v7w-spatial-temporal.rst — switch
pseudocode JSON block from `code-block:: json` to `text` (the
`<int32 x 3>` placeholders aren't valid JSON tokens).
Verification:
- make docs-api-clean && make docs-api → build succeeded, 0 warnings
- make test → 1588 passed, 28 skipped
- make chain-check-shards → 0 breaks across all 7 shards
- import-time SyntaxWarning escalation on edited modules → clean
Closes the warrant-promotion data path: claim-pack records now
bind to surface-ingested textbook chunks via Merkle inclusion
proofs in the existing `derivations` table.
What landed
===========
arborist/qa/warrant_resolver.py — four pure-data steps + one DB
write:
1. parse_citation(s) — regex pipeline turning the claim-pack
`source_reference` string into structured Citation tuples.
Handles "Title by Author" (single + Oxford-comma multi +
et-al), semicolon-separated multi-cite ("Knuth §1.2.6;
Stanley §1.2; Brualdi §3.5"), and compact author-year
("Pascal 1654") forms.
2. resolve_chunks(c, shards_dir) — FTS5 search across sibling
crawl/ dir's textbook-surface shards. Skips the main numbered
shards (Wikipedia content; would be false positives). Per-
shard match filter requires BOTH author last name AND a title
token in the shard's title-haystack — honest "no match" for
textbooks not yet surface-ingested.
3. compute_proof(shard, doc_root, chunk_id) — reads
merkle_nodes, walks layer-by-layer to assemble siblings;
emits deterministic JSON proof_blob compatible with
arborist/merkle.py verification.
4. write_derivation(...) — INSERT OR IGNORE into the existing
derivations table with process_id="warrant-resolver-v1".
Idempotent at the database layer.
CLI surface
===========
- `arborist warrant-status --shards-dir ...` (read-only) —
emits per-record JSON: parsed citations, FTS5 candidates,
whether a derivations row exists.
- `arborist warrant-resolve --shards-dir ... [--write]` —
default dry-run summary; --write actually computes proofs
and inserts rows.
End-to-end verification
=======================
Real-shard run: `arborist warrant-resolve --shards-dir
~/.arborist/shards --write` →
records_total: 92
records_resolved: 18
derivations_written: 18
All 18 are pillar-IV Hilbert axioms citing "The Foundations of
Geometry by David Hilbert" — the only cited textbook fully
surface-ingested by Phase 1. The remaining 74 records cite
textbooks not in our shard cluster (Mendelson, Enderton,
Jech, Goldstein, Barendregt, Stanley, Brualdi, Knuth, …) and
correctly produce 0 matches; they stay at ANCHOR-WARRANTED
until those textbooks land via future Phase-1 manifest
expansions.
Re-running the writer is a no-op (PK collision on (core_root,
src_root, process_id) = INSERT OR IGNORE).
Drive-by fix
============
arborist/sources/textbook_tex.py — _extract_title now also
parses PG's plain-text `Author:` line and appends "by Author"
to the title, so the warrant resolver's author-last-name match
works against PG-ingested textbooks (Hilbert "The Foundations
of Geometry by David Hilbert" instead of just "The Foundations
of Geometry").
Test suite
==========
tests/test_warrant_resolver.py — 14 unit tests for the citation
parser (no DB / network). Full suite: 1588 passed / 28 skipped.
Phase 3 (verifier wiring)
=========================
NOT in this commit. The data substrate is in place; the
audit_mode upgrade path that lifts answers citing
claim-pack-records-with-derivations from ANCHOR-WARRANTED to
EVIDENCE-WARRANTED requires a verifier change — touches well-
tested code, worth its own ticket so the regression risk is
bounded.
Two small Sphinx hygiene fixes surfaced while fox was building the
docs locally:
1. docs/_source/merkle-agi-v7w-spatial-temporal.rst was authored
earlier this session (#000013) but never added to a toctree
so Sphinx flagged it as orphan. Slotted under "Substrate"
alongside pi-star / bench / v8-fork-score where it belongs.
2. The same file had |translation_max| as raw text in an
ε-bound expression; RST parsed the pipes as a substitution
reference and errored. Wrapped the expression in double
backticks so it renders as literal math.
Build now drops from 40 warnings (counting docstring noise) to
5; the 5 remaining are pre-existing module-docstring formatting
in arborist/qa/*.py that pre-date this work.
Reviewer-flagged errata in the dav1dprometheus comm doc surfaced
two real drifts in canonical docs that needed correction:
docs/_source/pi-star.rst:
- "Fifteen concrete π*'s" -> "Sixteen". The table was missing
combinatorics@v1 (#000032). Authoritative count comes from
arborist.pi_star.registry.REGISTRY itself, with a note saying
so. Each entry called out as behaviorally immutable, with
forward link to docs/spec-methodology.md section 1.1 where the
versioning rule is canonical.
docs/dav1dprometheus-update-2026-05-09.md:
- Reverts a regression introduced in the previous errata pass
(455fc10). The reviewer counted 5 axes x 5 = 20 sub-batteries,
but 5T carries 6 (legacy 'transfer' from SQD-whitepaper plus
the canonical Dav1DPrometheus five, kept side-by-side per
ticket #000024). Total is 21, not 20. Top-of-doc revision
note records the correction, body section restores the 21
count with the explicit 5+6+5+5 explanation.
Other reviewer points are already canonical (kernel-version
immutability is in docs/spec-methodology.md section 1.1) or are
editorial-only and don't require canonical-doc changes.
Tested:
- len(REGISTRY) == 16 (verified live)
- bench/batteries/runner.py enumerates 21 sub-batteries
(5+6+5+5 per file naming under bench/fixtures/5{s,t,f,r}/)
External reviewer surfaced several errata in the 2026-05-09 draft.
Corrections applied in-place; original draft is preserved at
commit a2ff9d4.
- sub-battery count: 21 -> 20 (5 axes x 5)
- 5S vocabulary fix:
Surface/Substrate/Synthesis/Semantics/Semiotics
-> Syntax/Semantics/Syllogism/Synthesis/Semiotics
(matches bench/batteries/b_5s.py)
- 5R vocabulary fix:
React/Recall/Reason/Refine/Restore (reviewer's guess; doc's
original was even less correct)
-> React/Rearrange/Restore/Replicate/Resonate
(matches bench/batteries/b_5r.py per SQD section 9.3)
- pi* kernel chronology: explicit "15 -> 16 with combinatorics@v1"
- kernel-version immutability: stated as hard invariant (no
in-place behavioral mutation; behavior change = new version)
- cross-witness vs cross-carrier: distinguished. Witness channels
are kernel/cache/LLM; carrier modalities are
text/code/arithmetic/etc. The pattern generalizes from the
former to the latter but the audit semantics are distinct.
- STRICT-WITNESSED is a render label, not a new audit_mode.
Persisted column stays CANONICAL_PROJECTION; witness audit
event layers on top so cache_key semantics don't drift.
- warrant-promotion ladder: SOURCE-ANCHORED tier introduced
between ANCHOR-WARRANTED (assertion-only) and
EVIDENCE-WARRANTED (chunk-proof). Today's surface-ingest
work delivers SOURCE-ANCHORED; chunk-resolution layer is
needed for EVIDENCE-WARRANTED.
- license tags marked project-reported (manifest self-attest;
not independently audited by counsel)
- "no human labelers" scoped to canonical-shape divergence
labels only; claim-pack validity + textbook warrant
promotion still benefit from human curation
- top-of-doc revision note records the errata pass
- closing footer updated to reflect today's CI re-enable +
shard-search fan-out (66s -> 15.5s on the 4-shard cluster)
Durable copy of /tmp/dav1dprometheus-arborist-update.md into
docs/ so it ships with the repo. Captures eight days of
substrate work landing on the 5S/5F/5T/5R framework:
- 21 sub-batteries × 662+ default fixtures
- 15 π* canonical-projection kernels (registry closed)
- canonical-projection persistence + multi-witness pipeline
- claim-pack corpus across 7 pillars (I-VII + IX), 92 records
- surface-ingest layer for cited textbooks (~351 docs, 1597
chunks across 6/7 g4 pillars)
- witness-sweep cron flowing (5/8 KERNEL-LLM-DIVERGED on live
Hermes; auto-extracted to 5F-Falsification fixtures)
- #000013 v7-W substrate paper closed
- #000016 ZK Phase-2 parked with bench-plan + wire-protocol
- #000018 soft-hash covert-channel analysis closed; three
follow-up tickets opened (#000034 / #000035 / #000036)
- 1,684 passing, 37 sympy-skipped
Same naming pattern as docs/qa-modes-bench-2026-04-30.md so
future point-in-time reports follow the precedent.
