Commit graph

190 commits

Author SHA1 Message Date
0379e4c23a
docs/tickets: open #000045 — Prometheus-Σ Phase 3 sleep-sweep scaffold
Phase 3 of #000037 (the actual sleep-sweep scheduler that runs the
Phase 1 controller on a cadence over real shards) gates on a
measured retrigger, not a calendar date. This ticket is the gate.

§2 commits 8 governance parameters that fold into governance_policy_hash
when Phase 3 lands: chunk_size (Hermes concurrency), per-mode τ_qa
seconds (CP/LLM split from Finding 3), weight profile (default
"sweep" from Phase 1.c), active sweep targets, per-window budget
cap, scheduling cadence, quarantined-row policy.

§3 names 4 retrigger gates: ≥1000 advisory rows from Phase 2 wiring
showing reproducible REJECT/DEFERRED structure (Retrigger 1); three
consecutive weekly dry-runs with sustained ACCEPT/MARGINAL on Target
A (Retrigger 2); 5F-fixture funnel demand from #000025 plateauing
on Target A's stream and needing Target B's larger candidate pool
(Retrigger 3); operator mission need (Retrigger 4, mirrors #000037
§12 Trigger 4).

§4 explicitly excludes implementation, schema migration,
governance_policy_hash bump, dry-run sweep_weights swap, and
Hermes-call planner — all deferred to the implementation ticket
that this ticket gates.

Includes TICKETS.md index row + Next ID bump 000045 → 000046.
2026-05-10 18:38:18 -04:00
1f882df22c
#000037 §22 Finding 3: per-audit-mode τ_qa in dry-run
Per Finding 3, a uniform τ_qa=7d filtered out every recent
CANONICAL_PROJECTION row (the π* graduations from #000027/#000030/
#000032 are all younger than 7d), so the sweep saw zero high-value
kernel-only work. Splitting τ_qa by audit_mode lets the cheap kernel
re-probe path (CP) run on a short cycle while the expensive LLM
re-witness path (STRICT/HYBRID/UNGROUNDED) keeps the long cycle.

bench/scripts/prometheus_sigma_sweep_dryrun.py: new build_tau_by_mode()
helper + per-mode CASE in iter_target_a_candidates; sweep_target_a now
takes the dict instead of a single seconds value. New CLI flag
--tau-qa-cp-days (default 1d); --tau-qa-days now scopes to LLM-witness
modes only (default 7d). Report renders the per-mode τ table in the
header and marks Findings 2 and 3 RESOLVED with their landing commits.

Makefile: PROMETHEUS_SWEEP_TAU_DAYS bumped to 7 (was 1, the prior
Finding-3 workaround); new PROMETHEUS_SWEEP_TAU_CP_DAYS=1 makevar.

bench/results/prometheus-sigma-sweep-dryrun-2026-05-10.md: regenerated
under the new defaults — 9 CP rows surface alongside 1,904 LLM-
witness candidates → 1,913 total Target A candidates → 1 ACCEPT,
2 MARGINAL, 205 REJECT, 271 DEFERRED chunks; 2 cache_drift vetoes
preserved end-to-end.

docs/tickets/ticket-000037 §22: Findings 2 + 3 marked RESOLVED with
landing-commit references; total dry-run cost line updated to the
new measurements (37.6 ms / 3,913 branches / 9.6 µs per branch).
2026-05-10 18:35:19 -04:00
8da29fde69
ticket #000044: AUTOCOUNT doc-drift discipline + code-fence parser fix
Lock the AUTOCOUNT regression-test pattern as the design log
canonical record. Previously declined when surface was 1-metric
+ 29 tags; now mature enough (4 metrics + 58 tags + 1 same-day
drift-catch since landing) to formalize.

== Ticket content ==

10 sections covering:

1. Why this exists — the 4-drift-day baseline (6cbbf95 / 14bcb99 /
   5c21e83 / 30a9488) that motivated mechanization. Five-step
   walk through justifying each choice (Step 5 last).
2. Format — `<!--AUTOCOUNT:metric:path-->N<!--/AUTOCOUNT-->`.
3. Four supported metrics with examples + skip semantics:
   `tests`, `fixture-rows`, `db-rows`, `db-where`.
4. Skip-on-absence — operator state (shards, qa.db) absence is a
   logged skip, not a fail. Smoke verified 2026-05-10 with
   HOME=/tmp/empty.
5. What NOT to tag — closed-ticket point-in-time snapshots,
   aggregate floors ("2000+"), historical journey arcs.
6. Install discipline at write time + at refresh time.
7. Future metrics deferred (file-lines, gh-pr-comments-count,
   module-loc, commit-hash-exists) with the "add a metric"
   recipe.
8. Empirical baseline at landing (3 test functions, 58 active
   tagged claims across 8 doc files, harness runtime 2-4s).
9. Scope boundaries — does NOT auto-rewrite, does NOT validate
   prose quality, does NOT scan docstrings, does NOT lock
   values, does NOT add deps.
10. References — every landing commit + sister doc.

Closed at landing (status quo since fc5ba50 2026-05-10 morning;
this ticket is retroactive design log per the convention "every
ticket flips to `closed · landed in commit <sha>` when the work
ships").

== Code-fence parser fix ==

Adding the ticket itself surfaced an oversight: my AUTOCOUNT
examples in §3.3 + §3.4 used literal tag pairs in ``` fenced
code blocks. The parser was reading them as live claims and
firing on the illustrative `db-rows:002.db:concept_relations`
claim (compared 1234 vs live 72576 — both meaningless because
it's an example).

Fix: `_strip_fenced_code_blocks` substitutes the body of every
triple-backtick block with newlines before regex scanning. Line
numbers stay aligned (newline-preserving substitution); tags
inside fences are skipped because their parent text no longer
matches the regex.

Both helper functions (`_iter_claims` and the well-formed-tags
test) walk through the stripped text, so the strip discipline
is consistent across all three test functions.

== TICKETS.md index ==

Added #000044 row marked closed with the 5-commit landing trail.
Bumped Next ID 000044 → 000045.

== Verification ==

  $ pytest tests/test_doc_counts.py
  3 passed in 2.80s

  $ pytest tests/ -q
  2337 passed, 37 skipped in 108.29s

Hygiene: fox's in-flight changes to arborist/qa/runner.py +
arborist/substrate/prometheus.py + tests/test_prometheus*.py
left untouched in working tree.
2026-05-10 18:24:50 -04:00
3b301267ff
docs: #000037 §20+§17.1 refresh + #000025 Phase 1f closure + cookbook harvest section
Three coordinated doc landings tying together fox's evening
#000037#000025 closed-loop work (commits f625cac through
ff1752c + 8999b55):

== #000037 §20 Status — stale prose refresh ==

§20 said "open · awaiting go/no-go" but file header + TICKETS.md
both say "in progress" with phases 0/1/1.b/2 all landed. Refresh
§20 to reflect actual state with per-phase commit anchors:

- Phase 0 (doc): landed; David review applied per §21
- Phase 1 (pure-function controller): f625cac
  (arborist/substrate/prometheus.py + test_prometheus.py)
- Phase 1.b (gap-close): f9f5ae4 (§14 row 4 Hermes-saturation
  guard, §13 step 11 falsification-fixture proposal, §15 entropy
  + memory gates weight-tunable, ESCALATE > QUARANTINE > REJECT
  priority cascade)
- Phase 2 (advisory audit writes): a786d6d
  (prometheus_audit.py + controller_events sibling table; does
  NOT enter audit_events.event_hash preimage)
- §12 Trigger 2 fired 2026-05-10: divergence variance ratio
  0.575 > 0.5 with N=37 — Phase 1 opening is now empirically
  gate-satisfied per 8999b55
- Phase 3: deliberately NOT landed; replaced with read-only
  dry-run simulator surfacing five design findings (see §22)
- Closed-loop signal: 40 corpus-derived 5F fixtures harvested
  per ff1752c

== #000037 §17.1 — "78 atomic claim-pack records" → 92 ==

§17.1 said "#000031 Phase 2 has 78 atomic claim-pack records
that max out at ANCHOR-WARRANTED". #000031 closed at 92 records
(78 was an interim count during Phase 2). Refresh with the
journey (78 → 92) + AUTOCOUNT-tagged via db-where metric so
future drift fires immediately. Note the resolution context: all
92 now resolve via 74 citation-aliases + 13 term-aliases under
#000031 Phase 2.5 + B-1 + B-2. The unconscious sweep drains the
ANCHOR-WARRANTED → EVIDENCE-WARRANTED promotion backlog
(derivations.proof_blob rows still need computation even on
resolved chains).

== #000025 §11 Status — Phase 1f closure note ==

#000025 file header lists Phase 1a/1b.2/1c/1d/1e but the §11
Status section was frozen at "Open · awaiting go/no-go" — a
two-versions-old prose snapshot. Refresh with the per-phase
landing trail + add a Phase 1f section for the corpus-derived
falsification harvest that ff1752c shipped:

- Phase 1f closes the controller → 5F battery loop fox designed
  in #000037 §3 ("Divergence → candidate falsification fixture")
- bench/scripts/harvest_falsification_proposals.py reads qa.db,
  stratifies top-20-by-cache_key per audit_mode (HYBRID +
  UNGROUNDED), writes the 41-line JSONL pack (1 _meta + 40
  fixtures, AUTOCOUNT-tagged via fixture-rows)
- Every fixture row carries _harvest_meta with cache_key,
  witness_divergence at harvest time, audit_mode_at_harvest,
  harvest_threshold, source_ticket: "#000037 §13 step 11"
- 5F battery exercises them every test run;
  test_5f_falsification_harvested_pack_runs_clean asserts
  error_detection_rate == 1.0 by construction (every harvested
  row IS a falsification)
- Self-amplifying — coverage grows with corpus, not with
  hand-curation

Listed open items (§10.11 / 10.13 / 10.14) preserved verbatim
from file header so the index claim "still open" stays in sync.

== Cookbook: new "Adjacent: live-corpus → bench-fixture
   harvest" section ==

New section in warrant-substrate-cookbook.md between "Re-running
the substrate build" and "References" documenting the harvest
pattern. Three discipline patterns reused from the textbook
substrate noted explicitly:

1. Attribution metadata on every derived artifact (same shape as
   derivations.proof_blob carrying inclusion proofs back to
   source chunks)
2. Determinism via sort-and-cap (same shape as citation-alias
   cascade's "top 5 AND-join then top 3 OR-join" stratification)
3. Pin the metadata contract in tests (same shape as the
   cookbook appendix's discipline pins — silent regression
   becomes loud test failure)

Plus the reusable recipe for any controller emitting Proposal
records: define the dataclass, write a harvester filtering +
stratifying, pin metadata in tests, wire a make target.

References section gets four new entries pointing to #000037,
#000025, the harvest script, and the fixture pack.

== Drift caught + refreshed during this commit ==

While editing the cookbook, the AUTOCOUNT regression test
(from fc5ba50 / 6c6defb) caught two stale counts from fox's
in-flight prometheus work:

- test_prometheus.py: 36 → 42 (fox's uncommitted +6 for
  Phase 1.c sweep weight profile)
- test_prometheus_audit.py: 14 → 17 (fox's uncommitted +3)

Refreshed both inline + in the test/code-density table
(prometheus row test LOC also bumped 804 → 955 to match wc -l).

This is the harness firing exactly as designed — fox's
uncommitted tests changed live state and my doc claims went
stale within minutes. The test message named the file + line
+ claimed-vs-live, refresh was a 60-second turnaround.

== Verification ==

  $ pytest tests/test_doc_counts.py
  3 passed in 2.92s

  $ pytest tests/ -q
  2337 passed, 37 skipped in 107.43s

Hygiene: only docs/ paths staged. fox's in-flight changes to
arborist/qa/runner.py + arborist/substrate/prometheus.py +
tests/test_prometheus.py + tests/test_prometheus_audit.py
remain in their working tree, untouched by this commit.
2026-05-10 18:04:24 -04:00
8999b55c78
#000037 §12 Trigger 2 fires empirically — Phase 1 gate satisfied
Re-running `make prometheus-trigger-probe` after today's controller
landings shows the divergence-variance trigger has crossed both
thresholds:

  Trigger 2 — divergence variance
    Sample count: 37 (N_min = 30 ✓)
    Mean: 0.7568, σ: 0.435
    σ/mean ratio: 0.5748 (> 0.5 threshold)
    Absolute σ: 0.435 (> 0.1 threshold)

Same-day morning probe (commit baseline) had only 16 samples and
did not fire; the additional witness-sweep / dry-run / harvest
activity through the afternoon brought sample count above N_min.
Agreement-label distribution across all shards:

  KERNEL-LLM-DIVERGED  22
  KERNEL-LLM-AGREE      6
  LLM-DIVERGED          6
  STRICT-WITNESSED      3

Trigger 1 (branch density) and Trigger 3 (witness cost share) did
NOT fire. Per §12 a single trigger firing is sufficient for Phase 1
gating — and Phase 1 has already landed. This commit captures the
empirical evidence that Phase 1 was on the right side of the gate.

Trigger 1 remains structurally blocked on #000012 Phase 1c
(fork_score_branches sibling table); that's the natural next move
if anyone wants to surface multi-branch consensus signals.
2026-05-10 17:55:19 -04:00
6c6defbcb2
tests/doc_counts: db-where metric + tag prometheus controller (#000037 Phase 1+2)
Fan-out follow-up: extends AUTOCOUNT with a new metric for filtered
SQL-row claims, then tags fox's prometheus controller test surfaces
shipped this evening under #000037.

== Task 3: db-where metric ==

New metric ``db-where`` for tagging single-column equality
predicates. Target syntax::

    <table>?<column>=<value>
    <shard>:<table>?<column>=<value>

Resolves to ``SELECT COUNT(*) FROM <table> WHERE <column> = ?``
with ``<value>`` bound as a SQL parameter (no string
interpolation), so author typos or stray content can't escape
the predicate. Column + table names validated as bare
identifiers before string-interpolating into the query template;
sqlite3 connection opens with ``mode=ro`` URI flag.

Same skip-on-absence semantics as ``db-rows``: missing DB or
table yields a logged skip note, not a test failure. Sentinel
returns reuse the same _DB_MISSING / _TABLE_MISSING / _DB_ERROR
constants.

Smoke verified::

    _live_db_where('documents?source_type=claim_pack')      → 92
    _live_db_where('documents?source_type=wikipedia_xml')   → 866782
    _live_db_where('001.db:documents?source_type=wikipedia_xml')
                                                            → 867695
    _live_db_where('documents')  # malformed (no ?)         → -4

Tagged claims using the new metric (cookbook):

- L36 ``92 records total`` for the claim-pack source
- L349 ``The 92 chains have three quality tiers``

Both resolve to ``documents WHERE source_type='claim_pack'``
in shard ``000.db`` — the live count of claim-pack records.

L5 ``18/92 → 92/92`` historical narrative left untagged
(expressing a journey arc, not current state).

== Task 2: tag prometheus controller test surfaces ==

fox shipped two test files this evening under #000037 that
weren't previously inventoried in any reference doc:

- ``tests/test_prometheus.py`` — 36 tests covering Phase 1
  controller (commits ``f625cac`` + ``f9f5ae4``). Verifier-style
  discipline (NOT calculator pattern — it's a pure-function
  state-machine controller with no closed-form math).
- ``tests/test_prometheus_audit.py`` — 14 tests covering Phase 2
  ``controller_events`` sibling table (commit ``a786d6d``). Pins
  no-chain-mutation invariant (advisory writes never enter
  audit_events.event_hash preimage).

Added two paragraphs to cookbook §"Substrate-paper-spec'd
primitives" describing the test discipline + algorithmic surfaces
each pins. Renamed section header from
"(#000012 + #000018 + #000034)" to
"(#000012 + #000018 + #000034 + #000037)" to keep the
ticket-set roster current.

Two new rows in the test/code-density table:

  | substrate/prometheus.py       | 893 | 804 (36 tests) | 0.90 |
  | substrate/prometheus_audit.py | 200 | 388 (14 tests) | 1.94 |

All four counts AUTOCOUNT-tagged (2 inline prose + 2 table rows
= 4 new tags). prometheus_audit's ratio of 1.94 is high because
the test file pins a lot of write-path invariants for what is
nominally a small (200 LOC) sibling-table module — appropriate
for foundation-level audit-discipline code.

== Task 4: test_full_suite_total_fixture_count flake — investigated ==

Earlier today's transient ``-x`` flake (1 failed, 1799 passed)
did NOT reproduce in current tree state (2328 passed, 37 skipped
under same flags). Root-cause investigation:

- ``_DEFAULT_FIXTURES`` is read-only at module scope; no test
  mutates it.
- The test reads JSONL fixtures from ``bench/fixtures/`` via
  ``_run_one``; those files weren't being written by parallel
  tests.

Hypothesis (not confirmed, since flake didn't repro): transient
filesystem state during heavy-parallel-commit window
(``f625cac`` / ``a786d6d`` / ``6142437`` / ``f9f5ae4`` all
landed in succession around 17:23-17:24 EDT 2026-05-10 while my
test run was in flight). No structural defect identified. If
flake recurs, capture stdout + filesystem state at failure time
to confirm.

== Task 1: Walked fox's 6 evening commits via Explore agent ==

Agent reported what shipped under #000037 + #000012 evening
push (Prometheus-Σ Phases 0/1/2/dry-run + v8 consensus paper).
Findings used to drive task 2 above. Notable design choices
worth surfacing as reference:

- prometheus.py is **pure function** (no DB / LLM / scheduler);
  returns advisory ``ControllerDecision`` + optional proposal
  records, never mutations. Verifier-style test discipline.
- ``controller_events`` sibling table never enters
  ``audit_events.event_hash`` preimage — audit chain unaffected.
- Phase 3 sleep-sweep scheduler **deferred** in favor of
  read-only dry-run simulator surfacing 5 design constraints
  (chunk_size = Hermes concurrency NOT candidate pool;
  capital_cost flat=1.0 needs split; τ_qa per audit_mode;
  Target B = 4.4% canonical-shape match; quarantined-row veto
  exercises end-to-end). Calibration substrate for eventual
  scheduler.
- Per-branch controller latency 12.5 µs at chunk_size=4 → not
  the bottleneck; Hermes witness fan-out is.

v8 consensus paper at ``docs/_source/merkle-agi-v8-consensus.rst``
(834 lines; 11 parts) closes the loop from single-validator
Proof-of-Upgrade to multi-validator BFT selection. Phase 1c
(branch-set persistence) remains proposed-not-opened.

== Coverage ==

  Total tags after this commit:   54 (was 49; +5)
  Tags by metric:
    tests:           45 (+4 new prometheus + table rows)
    fixture-rows:     2
    db-rows:          3
    db-where:         2 (new metric, both 92 claim-pack)

  Files with tags:
    docs/warrant-substrate-cookbook.md             32 (+5)
    docs/calculator-test-patterns.md                8
    docs/soft-hash-channel-analysis.md              5
    docs/tickets/ticket-000006-bench-emergent...    4
    docs/seven-point-program.md                     3
    docs/tickets/ticket-000035-prg-choice-phi-prg.md 2

== Verification ==

  $ pytest tests/test_doc_counts.py -v
  3 passed in 2.60s

  $ pytest tests/ -q
  2328 passed, 37 skipped in 106.06s
2026-05-10 17:51:08 -04:00
12bf2df9d0
docs/TICKETS: refresh #000006 status row + calculator-test-patterns.md description
Two drifts caught in TICKETS.md sweep after today's wave:

1. **#000006 status row drift** — fox amended ticket
   #000006's file header on 2026-05-10 to:

     Status: open · rolling research log (verifier-ladder
             healthy at 134 cycles; emergent gems still being
             mined)

   But the TICKETS.md index row still said just `open · rolling`.
   Convention from peer rows (#000031, #000037, #000025) is for
   the index status field to carry the same descriptors that the
   ticket file header carries — the index is the load-bearing
   surface for at-a-glance reading. Refresh.

2. **calculator-test-patterns.md description drift** — the
   description in §"Distinction from other docs" was authored
   when the doc was a 9-item-checklist codification of a
   three-module pattern bench (t3_bound_calculator /
   phi_alignment_probe / anchor_prg, 2026-05-10 morning). Today
   evening's commit `f5dbfab` extended the doc with three new
   patterns (projective-contract, dispatch-order pin,
   tie-breaking-rule pin) sourced from fox's overnight π*
   kernel commits, plus four new exemplar test files
   (pi_star_arithmetic / _logic / _code / _time_series). The
   description should reflect the 12-pattern split (9 core +
   3 domain-specific contract pins) and the now-seven-exemplar
   inventory. Refresh.

Spot-checked status text in 9 in-progress / open / parked
tickets against their file headers — all in sync:

  #000043 #000037 #000036 #000035 #000034 #000033 #000025
  #000016 #000012

#000006 was the only divergence. Closed-ticket alias counts
(74 / 13 / 92/92) all match live; refreshed in earlier commits
this session.

Hygiene: docs-only commit, no code, no tests touched.
Verification: ``pytest tests/test_doc_counts.py`` 3/3 pass in 1.97s.
2026-05-10 17:00:37 -04:00
245543bda9
update everything with today's learnings — math review forward + Phase 3 dry-run amend
Two coupled doc updates capturing today's session state:

1. #000036 status pin — math review in flight with dav1d
   - Ticket status line: 'awaits fox math review' → 'pre-review
     polish pass 8916bf3; math review in flight with dav1d
     (forwarded 2026-05-10 — Tier 2 bundle)'
   - TICKETS.md index row mirrors same change
   - Future shifts can now see review is live, not blocked on fox.

2. #000006 rolling research log — 2026-05-10b amend
   - Fourth qualitatively different experimental shape:
     Prometheus-Σ dry-run simulator (joining random-word,
     witness-sweep, warrant-chain)
   - Captures the five scheduler-calibration findings (F1-F5)
     from bench/scripts/prometheus_sigma_sweep_dryrun.py:
     - F1: chunk_size = Hermes concurrency, not pool size
     - F2: capital_cost must split by audit_mode (CP=0.05 vs
       STRICT=1.0); flat-1.0 blocks every allocation
     - F3: τ_qa must split by audit_mode (1d for kernel-only,
       7d for LLM-witness); single-τ hides CP-rows
     - F4: Target B headline = 4.40 percent of docs are
       canonical-shape candidates (~152K across the corpus)
     - F5: quarantined-row veto exercises end-to-end on
       real-corpus data, no fixture-only mocking
   - Updates the distinct-signal table to four rows
   - Cross-references #000037 §22 for the full per-shard log

Phase 3 scheduler (when it ships) inherits F1-F5 as known-good
defaults — the dry-run is the calibration substrate the eventual
implementation will reference for choice justification.

Doc-only updates; no schema, no governance hash, no code change.
2026-05-10 16:57:19 -04:00
61424370bd
#000037 Phase 3 dry-run + Phases 0/1/2 status flip
Phases 1 (controller) and 2 (sibling-table audit writes) landed in
prior commits. This commit adds the Phase 3 dry-run simulator
instead of the actual sleep-sweep scheduler, since Phase 3's value
is mostly in what we'd learn from running it — and the dry-run
captures those findings without committing to a scheduler design
prematurely.

bench/scripts/prometheus_sigma_sweep_dryrun.py — read-only
simulator that classifies §3 Target A (providence_cache) + Target B
(documents) sweep candidates, synthesizes ControllerBranches from
real shard data, runs the Phase 1 controller, reports decision
distribution + Phase-3-design findings. No LLM calls, no
mutations.

make prometheus-sweep-dryrun — produces a dated markdown report
at bench/results/prometheus-sigma-sweep-dryrun-YYYY-MM-DD.md.

Five findings surfaced by three dry-run iterations against the
live ~/.arborist/shards corpus (3.5M docs + 2839 providence_cache
rows) — captured in ticket §22:

  1. chunk-size dominates Kelly threshold (must = Hermes
     concurrency, not candidate pool)
  2. flat capital_cost blocks every allocation (split kernel-cost
     vs LLM-cost on the contract)
  3. τ_qa=7d filters every CANONICAL_PROJECTION row (all 29 are
     <7d old; need per-audit-mode τ)
  4. Target B canonical-shape detection is the real headline
     (~152K candidates extrapolated; controller correctly returns
     MARGINAL on shape-match chunks)
  5. quarantined rows correctly veto via cache_drift hard-veto

Mean per-branch controller latency in dry-run: 12.5 µs at
chunk_size=4. Phase 3's actual bottleneck is the witness fan-out
(Hermes calls), not the controller itself.

Ticket #000037 status flipped to in-progress with Phases 0+1+2
landed; Phase 3 scheduler remains future work but is informed by
the five findings.
2026-05-10 16:47:02 -04:00
f5dbfabed5
docs/calculator-test-patterns: 3 new patterns from π* kernel work + 4 exemplar files
Land the three test-pattern shapes the Explore-agent investigation
of fox's overnight π* kernel commits surfaced. Patterns 1-8 in
this doc were the original 9-item checklist; patterns 9-11 are
new domain-specific contract pins that the π* kernels require but
that t3_bound_calculator (the original exemplar) does not.

== New patterns added ==

**§9 Projective-contract pin (one-way canonicalizers)**

For canonicalizers whose output type ≠ input type — output is
not in the input domain by design — assert that re-applying the
function raises. Pinned in code-py-ast@v1 (Python → S-expr) and
time-series-quantized@v1 (JSON → quantized text). When the output
type DOES equal the input type (arithmetic kernel's ℚ → ℚ),
use the dual: round-trip idempotence
``canonicalize(canonicalize(x)) == canonicalize(x)``. Both pin a
contract; pick by the kernel's type signature.

**§10 Dispatch-order pin (Python type-hierarchy gotchas)**

Python's ``bool`` subclasses ``int``, so a naive ``isinstance(x,
int)`` chain never reaches a bool branch. Kernels distinguishing
``True`` from ``1`` (Python AST normalizers, etc) must check
``bool`` first. Pin the branch order so a "simplify the dispatch"
PR fires loud. From ``593550b``.

**§11 Tie-breaking-rule pin (banker's rounding)**

Python's ``round()`` uses ties-to-even (PEP 3141): 0.5→0, 1.5→2,
2.5→2. Naive switch to ``math.floor(x + 0.5)`` (round-half-up)
produces 0.5→1, 1.5→2, 2.5→3 — different output for tie inputs
without breaking non-tie tests. From ``2585d3c``.

Each pattern has worked-example pseudocode + cross-reference to
the actual test file in fox's commit. The pattern numbers extend
the existing 1-8 sequence; renumbering would have invalidated
prior references.

== Checklist updates ==

Items 10/11/12 added (conditional — only when the kernel's shape
exposes the corresponding surface). Many calculator modules
(t3 bound, anchor PRG) need only items 1-9.

== Exemplar files reorganized ==

Replaced the single-exemplar reference (t3_bound_calculator only)
with a 5-file table cross-referencing the 9-12 checklist items
each exemplar covers:

  test_t3_bound_calculator.py    items 1-9 (53 tests)
  test_pi_star_arithmetic.py     items 1-6 + 9 + idempotence (56)
  test_pi_star_logic.py          items 1-6 + 9 + 11 (53)
  test_pi_star_code.py           items 1-6 + 9 + 10 + 11 (32)
  test_pi_star_time_series.py    items 1-6 + 9 + 10 + 12 (35)

All five test counts AUTOCOUNT-tagged so future drift fires the
regression test landed in ``fc5ba50`` / ``03c0f6a``. Total tagged
claims now 49 (was 44; +5).

== Source ==

Patterns surfaced from the Explore-agent investigation of fox's
overnight 2026-05-10 commits (``6c9bc04`` arithmetic, ``e7bef5f``
logic, ``593550b`` code, ``2585d3c`` time-series — 176 KATs total
across 4 π* canonical-projection kernels). The agent walked each
commit, noted the test patterns that didn't appear in the
original 9-item checklist, and reported the pattern shapes back.
This commit promotes those findings from session memory to
architecture-reference docs.

Verification:

  $ pytest tests/test_doc_counts.py -v
  3 passed in 3.39s
2026-05-10 16:33:19 -04:00
03c0f6a6d5
tests/doc_counts: extend with db-rows metric + backfill 15 tags (cookbook table + #000035)
Fan-out follow-up to ``fc5ba50``. Two thrusts in one commit since
they exercise the same surface:

== Task 3: extend AUTOCOUNT with db-rows metric ==

New metric ``db-rows`` for tagging live SQLite row counts (alias
tables, claim-pack records, etc — operator state that drifted on
``30a9488`` and earlier). Target syntax::

    <!--AUTOCOUNT:db-rows:citation_aliases-->74<!--/AUTOCOUNT-->
    <!--AUTOCOUNT:db-rows:002.db:concept_relations-->1234<!--/AUTOCOUNT-->

Default shard: ``~/.arborist/shards/000.db`` (where the alias
tables live per ``arborist.cli._aliases_db_path``). Operator state
is graceful-skip semantics: when DB or table is absent (CI, fresh
checkout, sibling repo), the claim is logged as skipped and the
test still passes. Drift only fires when the DB IS present and
the count diverged.

Sentinel returns:
- ``_DB_MISSING`` (-2): shards dir not present → skip
- ``_TABLE_MISSING`` (-3): DB present but table absent → skip
- ``_DB_ERROR`` (-4): malformed table name or sqlite error → skip

Table name validated against ``[A-Za-z_][A-Za-z0-9_]*`` regex
before string-interpolating into ``SELECT COUNT(*) FROM <table>``;
this is belt-and-suspenders since AUTOCOUNT tags are author-
controlled, but the dynamic SQL surface deserves a bouncer.

Smoke verified under HOME redirect to ``/tmp/<empty>``: 3 db-rows
claims gracefully skip with informative line-numbered messages,
suite still passes.

== Task 2: backfill 15 tags ==

Cookbook test/code-density table (lines 569-579, 10 rows) — every
``(N tests)`` cell now machine-checked:

    | aliases.py | 512 | 469 (28 tests) | 0.92 |
    →
    | aliases.py | 512 | 469 (<!--AUTOCOUNT:tests:tests/test_aliases.py-->28<!--/AUTOCOUNT--> tests) | 0.92 |

Markdown renderers strip HTML comments — table cells display
``28 tests`` unchanged. The ``warrant_resolver.py`` row stays
untagged because its test count is split across two test files
(verifier + parser) and the cell encodes a combined "~430"
instead of one collected count.

Cookbook alias-count surfaces (3 db-rows tags):
- L364 ``citation_aliases (74 rows live as of 2026-05-10)``
- L437 ``#000041 — citation-aliases table + 74 live rows``
- L438 ``#000042 — term-aliases table + 13 live rows``

Ticket #000035 (in progress, line 274) — refresh ``20 tests``
→ ``27 tests`` for ``test_anchor_prg.py`` + tag. Same drift
pattern as ``5c21e83``: ticket prose was written before the
``de997f7`` 2026-05-10 pattern backfill that added 7 tests
(prefix-extension closure, hand-formula, parametrized
invalid-input cones). Also tagged ``L279``'s 10-vector KAT
fixture claim with ``fixture-rows`` metric.

== Closed-ticket counts deliberately not tagged ==

#000028, #000030, #000042, #000031, #000004, #000026, #000009,
#000032, #000008 all carry historical "N tests pass" snapshots
from their landing date. Those are point-in-time records, not
live claims — drifting from current state is BY DESIGN. Tagging
them would fire the test on every successive change to the
codebase. Closed tickets are the design log; we don't backfill
them.

== Coverage summary ==

  Total tags after this commit:   44 (was 29; +15)
  Tags by metric:
    tests:           39
    fixture-rows:     2
    db-rows:          3

  Files with tags:
    docs/warrant-substrate-cookbook.md             27 (was 14)
    docs/soft-hash-channel-analysis.md              5
    docs/tickets/ticket-000006-bench-emergent...    4
    docs/seven-point-program.md                     3
    docs/calculator-test-patterns.md                3
    docs/tickets/ticket-000035-prg-choice-phi-prg.md 2 (new)

== Verification ==

  $ .venv/bin/pytest tests/test_doc_counts.py -v
  3 passed in 4.32s

  $ .venv/bin/pytest -q
  2276 passed, 54 skipped in 168.34s

  $ HOME=/tmp/empty pytest tests/test_doc_counts.py -v -s
  3 db-rows AUTOCOUNT claim(s) skipped:
    docs/warrant-substrate-cookbook.md:364 db-rows:citation_aliases skipped — /tmp/empty/.arborist/shards not present (CI / fresh checkout)
    docs/warrant-substrate-cookbook.md:437 db-rows:citation_aliases skipped — /tmp/empty/.arborist/shards not present (CI / fresh checkout)
    docs/warrant-substrate-cookbook.md:438 db-rows:term_aliases skipped — /tmp/empty/.arborist/shards not present (CI / fresh checkout)
  3 passed in 4.78s

No new dependencies. No schema changes.
2026-05-10 16:24:53 -04:00
fc5ba507dc
tests/doc_counts: regression test for numeric claims in docs/ (4x drift fix)
The doc-drift pattern recurred four times today on 2026-05-10
(commits 6cbbf95, 14bcb99, 5c21e83, 30a9488). Each fix was the
same shape: walk a doc, find a count that drifted from live truth
during the hours after the doc was written, refresh it. Cost: ~5
min per drift × 4 = 20 min of manual catching, with no guarantee
the next drift gets caught before someone external reads it.

Per fox's selection: regression test that makes drift loud at
test time instead of relying on visual catching.

== Mechanism ==

`tests/test_doc_counts.py` scans `docs/**/*.md` for AUTOCOUNT
tags of the form:

  <!--AUTOCOUNT:metric:path-->N<!--/AUTOCOUNT-->

Two metrics supported:

- `tests` — pytest collected count for path. Batches every
  tagged path into one `pytest --collect-only` subprocess
  (~0.5s total).
- `fixture-rows` — non-blank-non-comment line count in a JSONL
  fixture.

GitHub and most markdown renderers strip HTML comments, so
readers see only `N`. The tags are invisible in rendered output
but make the claim machine-checkable. Three tests in the file:

1. `test_doc_autocount_claims_match_live` — the core invariant
2. `test_autocount_tags_are_well_formed` — open/close balance
3. `test_autocount_metric_names_are_documented` — fail-closed on
   undocumented metrics (catches typos)

Failure message names the doc file, line number, and the
claimed-vs-live diff. Example:
`docs/foo.md:42 AUTOCOUNT(tests:tests/test_x.py) claims 23, live is 27`

== 29 tags installed across 5 docs ==

While installing tags I had to read the surrounding prose, which
surfaced six stale counts that had drifted same-day:

`docs/soft-hash-channel-analysis.md`:
- L392 14 → 23 tests for phi_alignment_probe
- L417 20 → 27 tests for anchor_prg
- L463 14 → 23 tests for phi_alignment_probe (status section)

`docs/seven-point-program.md`:
- L77 68 → 58 tests for metacognition (drift -10; the file
  shed tests during a refactor and the doc didn't catch up)
- L78 9 tests for `test_dag.py::test_preflight_*` — removed
  count entirely; pytest selector subsets aren't currently
  supported by the AUTOCOUNT metric set (would need a
  `tests-matching` metric; not worth the surface for one claim).
- L110 24 → 33 tests for test_dag.py

`docs/calculator-test-patterns.md`:
- L35 33 → 23 tests for warrant_resolver
- L35 10 → 9 tests for warrant_chain
- L16, L265 51 → 53 tests for t3_bound_calculator (kept
  initial-shipment provenance in prose)

== Coverage installed ==

  calculator-test-patterns.md           3 tagged claims
  soft-hash-channel-analysis.md         5 tagged claims
  warrant-substrate-cookbook.md        14 tagged claims
  seven-point-program.md                3 tagged claims
  tickets/ticket-000006-bench-...      4 tagged claims
                                      ---
                                       29 tagged claims

Every count that drifted today is now tagged. Future drift
fires the regression test at the next pytest run instead of
waiting for human catching.

== Discipline pattern ==

Walk this pattern for any new doc that names a count:

1. Surround the number with the tag pair:
   `<!--AUTOCOUNT:tests:tests/test_foo.py-->N<!--/AUTOCOUNT-->`
2. Run `pytest tests/test_doc_counts.py` (~3.5s)
3. If it passes, the claim is now machine-verified

Aim to tag counts on first authorship. Retrofitting is cheap
but only catches drift after the fact.

== Out of scope ==

Test counts inside source code (docstrings, CLI --help) are not
scanned — would expand the test surface significantly and the
drift pattern hasn't manifested there. Add `**/*.py` scope when
that pattern surfaces.

Alias-row counts and claim-pack-record counts could be tagged
with new `db-rows:<table>` and `db-where:<sql>` metrics; deferred
until the next drift on those numbers (none caught today after
30a9488's cookbook refresh).

== Verification ==

  $ .venv/bin/pytest tests/test_doc_counts.py -v
  3 passed in 3.89s

  $ .venv/bin/pytest -q
  2276 passed, 54 skipped in 153.21s

No new dependencies. No schema changes. No source-code changes.
2026-05-10 16:15:52 -04:00
fd643fe1a5
#000012 Phase 1b: Merkle-AGI v8 consensus paper landed
`docs/_source/merkle-agi-v8-consensus.rst` (834 lines, RST sister
to the v7-W substrate paper at the same path). Closes Phase 1b of
ticket #000012 — the loop-closing consensus protocol that turns
single-validator Proof-of-Upgrade into Darwinian selection across
an open validator set.

11 parts:

  Part 1   Introduction & motivation — gap table from v7 § 13.4,
           concrete backdoor-attack scenario, paper IS/IS-NOT
           scope.
  Part 2   Substrate definition — SQD A1/A2/A3 inheritance,
           consensus_events row schema, consensus_policy_hash
           sibling (never enters cache_key).
  Part 3   Validator state machine — bonding/active/challenged/
           slashed/unbonding with full transition graph + invariants.
  Part 4   Acceptance protocol — proposer submission, layered
           fitness floor (canonical + lab-declared ceiling),
           audit-replay procedure, 2/3-stake quorum + GRANDPA-
           style finalization, liveness floor.
  Part 5   Challenge protocol — counter-evidence shape,
           adjudication, challenger reward, frivolous-challenge
           bond.
  Part 6   Stake mechanics — bond/unbond/challenge window
           recommendations, offense-class slashing schedule,
           reward distribution, optional stake cap + sqrt-weighting.
  Part 7   Fork choice rule — GRANDPA-style finality, pre-finality
           constraints, liveness recovery.
  Part 8   Mesh wire format extension — three new message kinds,
           BLS-or-concat aggregate signatures, bandwidth profile.
  Part 9   BFT analysis — safety, liveness, Sybil resistance,
           bootstrap honesty, re-staking attacks.
  Part 10  Worked example — 7-validator deployment, one upgrade
           cycle with successful challenge against one fraudulent
           validator.
  Part 11  Out of scope — implementation, calibration, cross-chain
           anchoring, fixture selection, bootstrap-set membership,
           cross-instance slashing accumulator, branch-set
           persistence.

Closure §: open questions tracked separately (initial validator
set composition, threshold-key ceremony, ZK-replay, policy-hash
transition mechanics).

Ticket #000012 status updated; Phase 1c (branch-set persistence)
remains proposed-not-opened. Implementation follow-up tickets that
cite this paper land later — one per validator-state-machine,
mesh-wire-format extension, audit-replay harness, slashing
accountant.
2026-05-10 16:12:00 -04:00
883289d00e
#000025 Phase 1e: 5F falsification motif coverage (§10.12 closed)
Pre-Phase-1e: falsification-v1.jsonl covered 10 motif tags across 50
fixtures (the high-traffic warrant/title/anchor/format set). Phase
1e adds 12 fixtures (5f-fal-051..062) for the previously-uncovered
motifs from the verifier+soft-demote registries:

  CITATION_MISMATCH            DEFLECTION_DETECTED
  MANUAL_QUOTE_VIOLATION       SCHEMA_INVALID
  SOURCE_ROLE_BLOCKED          SUBJECT_TOKENS_ABSENT
  TOO_MANY_EVIDENCE_IDS        UNKNOWN_EVIDENCE_ID
  BROAD_QUANTIFIER_RUNAWAY     BROAD_QUANTIFIER_CAP_APPLIED
  BROAD_QUANTIFIER_SCOPE_UNBOUND  BROAD_QUANTIFIER_REJECTED

Coverage now: 22 unique motif tags across 62 fixtures.

Harness changes:
- test_bench_batteries.py: bump pass_count assertion 50 → 62 in both
  falsification tests; add test_5f_falsification_covers_every_documented_motif
  that pins the motif set against the verifier+soft-demote registries
  so adding a new violation upstream surfaces here as a missing
  fixture (loud signal, no silent drift).
- test_session_integration.py: bump full-suite total 662 → 674.

Closes #000025 §10.12 (every documented failure-motif tag).
Still open in Phase 1b: §10.11 (real shard finetuning chains),
§10.13 (Feedback Loop latency/efficiency against real workload),
§10.14 (threshold handoff to #000012).
2026-05-10 16:04:34 -04:00
30a9488578
docs/cookbook: non-test-count drift sweep — 3 staleness fixes
Sweep after closing the #000006 amend refresh: three areas in
warrant-substrate-cookbook.md drifted apart from TICKETS.md
authoritative status while the day-long substrate/alias sprint
was running.

Findings:

1. **Line 422 `#000041` count stale**: cookbook listed "54 live
   rows"; TICKETS.md row authoritative since 2026-05-10:
   "74 rows live as of 2026-05-10 (count grew 40 → 54 → 74)".
   Live `arborist alias citation list | wc` = 74. Refresh.

2. **Line 416 cascade-completion-state line ambiguous**: the
   `# → 92 / 92 (100%) under the 18-substrate + 54-alias state`
   comment is in the `make textbook-*` re-run code block. Read
   as "current state of a re-run today" it's stale (74 not 54);
   read as "historical state at 100% achievement" it's accurate.
   Refreshed to current state with explicit `(counts as of
   2026-05-10)` so a future reader knows what era it pins to.
   Split out citation vs term aliases since both feed cascade.

3. **Title-from-author backfill section over-states the
   workaround**: cookbook framed the SQL UPDATE pattern as
   "Workaround until source-side fix lands". The source-side
   fix already shipped — commit `551c969` 2026-05-10 (#000031
   follow-up B-2: `--author` flag in HTML + textbook_tex +
   crawler ingest paths; every `make textbook-*` Makefile
   target already wires it). The SQL pattern is now legacy
   fix-up for already-ingested shards that pre-date B-2;
   refresh the section header + body to reflect that.

Other claims sanity-checked + accurate:

- Line 122 `4 term-aliases` for Pillar VI Newton vocab — live
  domain breakdown is 5/4/4 (arithmetic / classical-physics /
  geometry); 4 classical-physics matches.
- Line 350 `(74 rows live as of 2026-05-10)` — accurate.
- Pillar I-IX record counts (13/10/13/18/5/5/14/14 = 92) —
  matches live `SELECT COUNT(*) FROM documents WHERE
  source_type='claim_pack'` in shard 000.db (92).
- Phase-status references on lines 261/373/424/458/494/499/512
  — all internally consistent with TICKETS.md authoritative
  statuses (#000031 closed, #000034/35/36 Phase 1 in progress,
  #000038 Phase 4 still-blocked).

Hygiene: docs-only commit, no schema, no tests. Pre-existing
order-dependent flake on test_full_suite_total_fixture_count
unrelated; test passes in isolation.

2326 tests collected; 1799 + 53 skipped pass when run with -x.
2026-05-10 16:02:28 -04:00
8916bf3728
soft-hash T3 bound: pre-review polish pass for external cryptographer
Four targeted edits before forwarding to dav1d:

1. Add §0 cover note — names the three things the reviewer should
   confirm (decomposition, per-surface derivations, conservative-
   constant choice) and what's explicitly out of scope (empirical
   validation against a real adversarial-training run).

2. Tighten §2 decomposition. The previous text invoked DPI to
   produce an additive split I(A;C) ≤ I(A;Θ) + I(Θ;C), which DPI
   alone doesn't justify. Replace with a clean Markov-chain DPI
   statement (A → Θ → C(M) is a Markov chain conditional on
   (H_{≤t}, n_t); DPI gives I(A;C) ≤ I(A;Θ)) and frame the T1+T2
   baseline as threat-model-additive (disjoint adversary surfaces),
   not information-additive in the same MI sense.

3. Rename §3 'Apply Fano's inequality' → 'discrete-distinguishability
   counting'. The derivation log₂(SNR+1) is the discrete channel-
   capacity bound on K distinguishable outputs, not Fano's
   inequality (which bounds error probability from MI). Add an
   explanatory note that LR factors cancel per-step (LR's distinct
   channel contribution is §4, not double-counted here). Update §10
   item 1 cross-reference for the same naming consistency.

4. Resolve §5 conjecture. Move the random-shuffle conjecture out of
   the headline derivation; commit C_B3 = 1 strictly under the
   adversarial-order assumption stated in §5. The random-shuffle
   tightening C_B3 → O(1/√N_b) stays referenced via §10 + #000043
   as the formal tightening path operators can opt into via the
   --c-b3 calculator flag.

No numeric examples changed; no calculator behavior changed; no
reference list changed. Pure pre-review polish to remove three
specific things a careful cryptographer would catch and ask
about, plus a cover note that frames the kind of review wanted.
2026-05-10 15:59:16 -04:00
0cb943561a
tickets: close #000026 (Phase 3 landed 60b5748)
Real-shard workload baseline + search latency: Phases 1/2/3 all
landed 2026-05-09. Flip status from "in progress" to closed with
Phase 3 commit reference. Authorship warrant ladder followup (CLI
wiring) noted in ticket body as non-load-bearing — not blocking
closure.
2026-05-10 15:54:32 -04:00
5c21e83b33
ticket #000006 amend: refresh stale test/KAT counts in 2c683de adjacent landings section
fox's 2c683de (#000006 2026-05-10 amend folding my Phase 3
warrant-chain dormancy investigation into the rolling research
log) listed two adjacent same-day landings in §"Adjacent landings
(same-day infrastructure)". Both counts went stale during the
hours after fox's commit, same drift pattern caught earlier
today on the cookbook appendix.

fox's 2c683de timestamp: 13:22 EDT.
- phi_alignment_probe test backfill (a4b3056) landed 14:08 EDT
- The corresponding cookbook appendix refresh (6cbbf95)
  followed shortly after.

Stale counts refreshed:

1. **#000034 Phase 1a entry** (line 725):
   - "14 tests" → "23 tests" (+9 from `a4b3056` 2026-05-10 pattern
     backfill: monotonicity / hand-formula / closure /
     parametrized-invalid per docs/calculator-test-patterns.md)
   - "10-vector KAT fixture" → "30-vector KAT fixture (10 aligned
     + 10 uniform + 10 anti-aligned synthetic checkpoints,
     deterministic-seeded)". The 30 vectors were generated by
     `1dfb8b9` itself; fox's amend simply summarized at "10".
     Refreshed to the actual count + class breakdown.

2. **#000035 Phase 1 entry** (line 729):
   - "20 tests" → "27 tests" (+7 from `de997f7` 2026-05-10 pattern
     backfill: prefix-extension closure invariant, hand-computed
     HMAC-SHA-512 first-block formula, parametrized
     invalid-input cones).
   - "10-vector KAT fixture" → confirmed 10 (matches live).

Verification:

  test_anchor_prg.py            27 ✓
  test_phi_alignment_probe.py   23 ✓
  bench/fixtures/phi-prg/        10 ✓
  bench/fixtures/phi-alignment/  30 ✓

#000006 amend is now zero-drift across all 4 numeric claims in
the adjacent-landings section. The recommendation in §"Verdict"
("track fire rate as a corpus-evolution signal") and the
distinct-signal table both remain accurate as fox wrote them.

Hygiene: docs-only commit, no test or code surface change.
2026-05-10 15:17:13 -04:00
14bcb99db1
docs/cookbook: fill missing test count + refresh test/code ratio after 581ad90
Comprehensive cross-check of the appendix's test counts vs live
`pytest --collect-only` output found two more drift points beyond
6cbbf95's phi_alignment_probe + t3_bound_calculator count
refreshes:

1. tests/test_warrant_resolver.py — appendix described its
   coverage qualitatively but didn't give a count. Live: 23
   tests. Added "23 tests covering ..." prefix.

2. test/code ratio table row for t3_bound_calculator.py was
   "446 351 (51 tests) | 1.4". My 581ad90 added 2 KAT-fixture-
   gap-closure tests + ~95 lines of test code (the
   test_b3_exact_formula + test_t3_bound_known_answer_tests
   functions). Refreshed to "249 | 446 (53 tests) | 1.79".

Comprehensive verification result (all 14 appendix entries
cross-checked against `pytest --collect-only`):

  test_aliases.py                              28 ✓
  test_warrant_resolver.py                     23 (was uncounted)
  test_warrant_chain.py                         9 ✓
  test_textbooks_manifest.py                   43 ✓
  test_anchor_prg.py                           27 ✓
  test_phi_alignment_probe.py                  23 ✓ (refreshed in 6cbbf95)
  test_t3_bound_calculator.py                  53 ✓ (refreshed in 6cbbf95;
                                                  ratio table fixed here)
  test_fork_score.py                           18 ✓
  test_substrate_fork_score.py                 27 ✓ (added in 6cbbf95)
  test_weights.py                              16 ✓
  test_pi_star_protocol_and_registry.py        21 ✓
  test_qa_progress.py                          31 ✓
  test_qa_prompts.py                           20 ✓
  test_concepts_extract.py                     20 ✓

Zero remaining drift. The cookbook appendix is now bit-for-bit
consistent with live pytest collection across all 14 entries.

Hygiene: docs-only commit, no code surface change.
2026-05-10 15:11:55 -04:00
6cbbf9505e
docs: refresh cookbook appendix counts + reciprocal cross-reference
Walking 6aca7d9 (cookbook test-coverage appendix) surfaced two
findings: (1) two stale test counts since fox wrote the appendix at
2026-05-10 13:11 EDT; (2) the appendix and docs/calculator-test-
patterns.md are complementary lenses but had no explicit
cross-reference. Both fixed in this docs-only commit.

Stale counts refreshed
======================

- `tests/test_phi_alignment_probe.py` "14 tests" → "23 tests".
  Drift cause: my `a4b3056` (2026-05-10 14:08 EDT) added 9
  pattern-backfill tests after fox's appendix snapshot at 13:11
  EDT (~57 min lag).
- `tests/test_t3_bound_calculator.py` "51 tests" → "53 tests".
  Drift cause: my `581ad90` (2026-05-10 ~13:50 EDT) added 2
  KAT-fixture-gap closures after fox's appendix snapshot.

Both refreshes preserve the trajectory by noting the
``+9 from a4b3056`` / ``+2 from 581ad90`` provenance inline. Same
durability pattern fox used in `018a2a1` for the alias-count
refresh + my `6f1dbed` ditto.

Per the appendix-author's own ``unit-test density`` heuristic,
the refreshed counts confirm both files keep their ≥1× test/code
ratio. ``test_phi_alignment_probe.py`` jumps from 268/200 ≈ 1.34
to 419/200 ≈ 2.10× (closer to the contract-defining-foundation
ratio fox flagged for warrant_chain.py at 3.6×).

Missing entry added
====================

`tests/test_substrate_fork_score.py` (renamed from
`test_v8_fork_score.py` in `a4058a4` per the 2026-05-10 v-prefix
retirement) wasn't listed in fox's appendix. The file is the
``arborist substrate score`` CLI surface coverage — adapter tests
+ 4 in-process build_parser CLI tests + 1 real subprocess
invocation. Distinct from `test_fork_score.py` (fox's pure-function
unit tests for ScoredFork at 18 tests).

Added under "Substrate-paper-spec'd primitives" section alongside
test_fork_score.py.

Reciprocal cross-reference
==========================

`docs/calculator-test-patterns.md` (the per-pattern CHECKLIST for
new tests) and `docs/warrant-substrate-cookbook.md § Appendix`
(the per-discipline INDEX of existing tests) are complementary,
not duplicative:

  - Checklist answers: "what should my new tests cover?"
  - Index answers: "where are the tests for X?"

Added each-direction cross-reference paragraphs:

- Cookbook appendix § "Cross-reference" subsection naming
  calculator-test-patterns.md as the checklist for new code.
  When adding a new substrate-paper-spec'd primitive: walk the
  checklist to design the test file, then add a row to the
  appendix under the matching discipline.
- calculator-test-patterns.md § "What this doc is NOT" expanded
  with a bullet pointing readers at the cookbook appendix as
  the existing-test inventory.

Closes the gap where future shifts might find one without the
other and miss half the discipline.

Hygiene
=======
- make test → 1986 passed, 45 skipped.
- Both docs are reference-only; no test or code surface change.
2026-05-10 13:57:00 -04:00
d720b73d91
remove docs/dav1dprometheus-update-2026-05-09.md
Per fox 2026-05-10 — file not needed. Was a point-in-time update
note; superseded by the rolling research log in #000006 and the
ticket-level status pins (#000028 closed in TICKETS.md, witness-
sweep amend in #000006 etc.).

No incoming references in the repo (grep clean).

Note for the record: the immediately-prior commit (1421d96) made
an internal-consistency fix to this same file. That fix is moot
post-deletion but stays in the audit chain — describing the
witness/carrier vocabulary distinction was useful at the time it
shipped.
2026-05-10 13:24:51 -04:00
1421d96659
dav1dprometheus 2026-05-09: line 284 internal-consistency fix
The doc's own meta-comment at line 160 explicitly distinguishes
witnesses (kernel/cache/LLM channels over the same canonical bytes)
from carriers (text/image/audio/world modalities). Line 284 still
used the conflated phrase 'multi-modality witness' that the meta-
comment was teaching against.

Update to 'multi-witness canonical-agreement' — consistent with
the doc's own clarification, with David's 2026-05-10 review of
#000037 (which renamed #000028's mechanism descriptor across the
controller spec for the same precision reason), and with the
2026-05-10 #000006 amend that uses the corrected vocabulary.

Closed-ticket references to #000028 by its actual title (Multi-
modality witness for canonical shapes) are intentionally not
touched — those are accurate citations of the ticket title, not
generic mechanism descriptors.

Doc-only update; no schema, no governance hash, no code change.
2026-05-10 13:24:12 -04:00
2c683de6d8
ticket #000006: 2026-05-10 amend — Phase 3 warrant-chain dormancy
Third qualitatively different experimental shape in the rolling
research log, parallel to the 2026-05-09 witness-sweep widening:

- random-word emergent (honesty under no ground truth)
- witness-sweep (capability under canonical ground truth)
- warrant-chain (rescue-mechanism activation under chain-backed
  ground truth)  ← new

Captures the structurally-grounded dormancy of #000031 Phase 3
verifier — 0/117 fires across an aggressive A/B with 12/13
chain-root retrieval, plus 5 paraphrase fixture iterations
none of which drive Phase 3 to fire empirically.

Root cause pinned at arborist/qa/warrant.py:507: warrant_check
vacuous-passes unless question shape is relation/date/entity-list/
count/why-cause. Definitional questions don't have warrant shape;
claim-pack chunks are terse axioms without prose context that
warrant-shape questions would retrieve. Phase 3 is structurally
rescue-only on the current corpus.

Mechanism correct + non-regressing + unit-tested. Recommendation:
track fire rate as a corpus-evolution signal, not a fixture-
engineering target. Audit-line tail will surface organic fires.

Adjacent same-day landings noted: #000034 Phase 1a (Hessian-
alignment probe) + #000035 Phase 1 (φ_PRG reference) — both
KAT-pinned cryptographic-primitive infrastructure ahead of v7
plastic-training deployment.

Doc-only update; no code change.
2026-05-10 13:22:30 -04:00
7ea0fdf2b1
ticket #000039 §13.5: amend test plan with CLI subprocess coverage
Phase 1 plan §13.5 named four test files (3 unit + 1 integration)
but did NOT name CLI subprocess tests for the three new CLI
surfaces §13.4 introduces:

  - arborist ingest --embed (flag on existing subcommand)
  - arborist query --retrieval={fts5|vec|hybrid} (new flag)
  - arborist vec rebuild (new subcommand)

Per docs/calculator-test-patterns.md §6 (codified earlier today
in commit 0725eb4 from the three-module pattern bench): import-only
tests miss argparse + main() drift. Yesterday's substrate refactor
caught this hazard three times — fork_score.py import (85be5eb),
Makefile bench-fork-score target (209d670), .gitlab-ci.yml job
+ script (b320e27). Each fix was 1-3 lines, but each had been
shipped to main + would have surfaced as a noisy CI failure on
next pipeline run.

§13.5 now adds three CLI subprocess test files:

  - tests/test_cli_vec_rebuild.py
  - tests/test_cli_ingest_embed_flag.py
  - tests/test_cli_query_retrieval_flag.py

Each gated via pytest.importorskip("sqlite_vec") so they skip
cleanly when [vec] extras absent. Pattern matches fox's
test_cli_baseline_runs_clean / test_cli_invalid_input_exits_2
in tests/test_t3_bound_calculator.py (the exemplar for
calculator-style CLI tests).

§13.7 size estimate revised: 4 test files → 7 test files (+3 CLI
subprocess), 250 → 400 test LOC. CLI subprocess tests are
~30-40 LOC each (boilerplate + tmp_path + subprocess.run +
JSON parse). Phase 1 total grows from ~550+250 → ~550+400 LOC.

Doc-only edit; doesn't unblock or block fox's §13.8 four
decisions — the test-plan addition is mechanical discipline,
not a scope change. Phase 1 still gates on the four §13.8
decisions before any code lands.

Cross-ref: docs/calculator-test-patterns.md §6 (CLI subprocess
pattern) + the three substrate-rename defect commits caught by
that pattern in retrospect (85be5eb / 209d670 / b320e27).
2026-05-10 13:19:53 -04:00
0725eb418b
docs/calculator-test-patterns.md: codify the 2026-05-10 pattern bench
Captures the testing patterns surfaced from the three-module
pattern bench (anchor_prg / phi_alignment_probe /
t3_bound_calculator) as a docs/ architecture reference. Future
blackops shifts find it via grep instead of having to read commit
messages from de997f7 + a4b3056 + 35aeebb to reconstruct the
discipline.

Eight patterns codified:

  1. KAT (known-answer-test) regression — pin (input, expected)
     fixtures under bench/fixtures/<module>/
  2. Hand-computed formula assertion — compute the spec's math
     in the test file from first principles, assert agreement
  3. Monotonicity per input axis — scaling one knob while
     holding others fixed verifies the function responds in
     the expected direction
  4. Closure / sum-of-parts invariants — A + B + C ≡ total;
     no missing terms, no double-counting
  5. Parametrized invalid-input cones — collapse N
     test_rejects_X functions into one @parametrize per
     validation class
  6. CLI subprocess end-to-end — argparse + main() drift the
     import-only tests miss
  7. Doc parity — pin the spec's worked-example numbers
     against the function's actual output (catches doc drift,
     which surfaced today as fox's calibration-pass note on the
     T3 bound doc §11)
  8. Module-export shape — version constants, asdict round-trip,
     JSON-serializability, "v1" version-string discipline

Includes:

- "When to use" header that explicitly limits applicability to
  calculator-style code (math-heavy, closed-form output, JSON-
  serializing CLI)
- "When NOT to use" disclaimer that verifier / parser /
  state-machine tests have their own appropriate structures
  and shouldn't be retrofitted (saves future shifts the same
  realization I had today walking warrant_resolver tests)
- A 9-item checklist for new calculator-style modules to land
  with consistent test density
- Concrete code excerpts from the three exemplar test files

Architecture-reference convention followed (per docs/TICKETS.md
§ "Distinction from other docs"): docs/<filename>.md, no ticket
number, listed in the architecture-reference block. Joins
concept-relations-design.md, warrant-substrate-cookbook.md, and
the cti-architecture.md / mesh.md / etc. set.

docs/TICKETS.md amended with the new entry under "Distinction
from other docs".
2026-05-10 13:13:19 -04:00
28b57da381
soft-hash analysis §9.1: pin #000034 Phase 1a landing (mirror of §9.2)
#000034 Phase 1a landed 2026-05-10 (synthetic-ablation
infrastructure for the Hessian-alignment probe under φ_linear):
bench/scripts/phi_alignment_probe.py + 14 tests + KAT fixture
at bench/fixtures/phi-alignment/synthetic-checkpoints.jsonl.

Same closure-loop grooming pattern as 0b038f1 (§9.2 update for
#000035 Phase 1): the analysis doc already named the test path
in §9.1 but did not yet point at the artifacts that just shipped.
This commit closes the loop for §9.1 + updates §11 status block.

Phase 1b (the actual v7-checkpoint measurement that resolves the
open question and feeds C_B1 tightening per #000043) still parks
on v7 deployment ramp-up — same exogenous gate as #000035 Phase 2.

Doc-only update; no schema, no governance hash, no code change.
2026-05-10 13:11:39 -04:00
6aca7d91d7
docs/cookbook: appendix — test-coverage cross-reference (2026-05-10)
Adds a per-discipline test-file index to the cookbook so audit
reviewers can click through from a substrate discipline to the
unit-test pinning that prevents silent regression. Eight
sub-sections matching the live discipline groups:

  - citation/term alias mechanism (#000041 + #000042) —
    test_aliases.py 28 tests
  - warrant-resolver chain (#000031 P1+2+3) —
    test_warrant_resolver.py + test_warrant_chain.py
  - textbook ingest license-discipline gate —
    test_textbooks_manifest.py 43 tests
  - cascade tuning (#000040 + parenthetical/OR-fallback) —
    _phrase_from_parenthetical regression guard
  - substrate-paper-spec'd primitives (#000012 + #000018 +
    #000034 + #000036) — 6 test files
  - Q&A / verifier scaffolding — test_qa_progress.py +
    test_qa_prompts.py
  - concept-relations write-side — test_concepts_extract.py

Plus a unit-test-density-vs-code table: small contract-defining
modules (weights, prompts, warrant_chain) sit at 1.4-3.6× test
LOC because they're foundation; larger modules (warrant_resolver,
aliases) settle at 0.5-1.0× because they're more code-with-tests-
per-feature than contract-with-tests-per-rule.

Code-review heuristic surfaced for future shifts: new
substrate-paper-spec'd primitive without ≥1× test/code ratio
is suspect on landing.
2026-05-10 13:11:08 -04:00
de997f7be3
docs/T3 bound + tests/anchor_prg: apply fox's testing patterns
Two related cleanups in one commit, both surfaced by reading fox's
test_t3_bound_calculator.py (51 tests for my T3 calculator):

1. Refresh stale §7 numbers in the T3 bound doc
=================================================

fox's test_baseline_matches_section_11_doc docstring (lines
56-61) flagged that my §7.1 worked example said
622.7 / 290.0 / 32.7 bits but the calculator's actual
closed-form output is 625.87 / 292.48 / 33.39. Same drift in
§7.2 (3358 → 3387.72) and §7.3 (247 → 247.14).

The numbers were rounded estimates from when I drafted the doc
before the calculator existed. Refreshed all three §7 numeric
examples to match the calculator's actual output (verified live
via t3_bound_bits()). §3 inline approximation likewise updated
(290 → 292.48). Added a short note pointing readers at the
calculator + tests as the source of truth.

2. Backfill anchor_prg tests with fox's patterns
=================================================

fox's test_t3_bound_calculator.py demonstrated four patterns I'd
missed in my #000035 phi_prg tests:

- **Output prefix invariant** (closure check): phi_prg(h, n+k)[:n]
  ≡ phi_prg(h, n). Streaming-counter invariant — would catch a
  bug where a per-call seed mutation broke determinism across
  dim_h values.
- **Output length monotonicity**: len(phi_prg(h, n)) == n exactly.
  Parametrized over n ∈ {1, 2, 4, 7, 16, 17, 64, 1024}. Catches
  off-by-one in `_expand` truncation.
- **Hand-computed first block**: assert that the first 64 bytes
  of output equal a direct ``hmac.new(seed, h + b'\\x00\\x00
  \\x00\\x00', sha512).digest()``. Pattern from fox's
  test_b1_exact_formula — don't rely on KAT regression alone;
  compute the first-principles math in the test file. Catches
  algorithm drift the KAT (regenerated against a buggy version)
  would miss.
- **Seed-bleed check**: changing the seed must change EVERY output
  position. Probability of false-positive ≈ 64 · 2^-32 ≈ 2^-26;
  none expected in practice.
- **Parametrized invalid-input tests**: collapsed N separate
  ``test_rejects_*`` functions into ``@pytest.mark.parametrize``
  cones (4 wrong-size-hash cases + 3 non-positive-dim_h cases).
  Same coverage, fewer test functions.

Test count: was 20 in test_anchor_prg.py; now 27 (+7 from
parametrize expansion + new patterns). Full suite: 1720 → 1727.

Hygiene
=======
- make test → 1727 passed, 45 skipped.
- make chain-check-shards → 0 across all 7 shards.
- All new tests use ``pytest.importorskip`` already at module top
  (anchor_prg has no extras gate; tests run unconditionally).

Lessons captured
================
The patterns to remember for future calculator/probe-style code:

  1. KAT regression alone isn't enough. Add hand-computed
     formula tests so the math itself is asserted in the test
     file, not just "consistent with a recorded snapshot".
  2. Test monotonicity / closure invariants. They catch
     algorithm drift, sign errors, missing terms.
  3. Parametrize invalid-input tests. One function, N cases.
  4. Test the doc's numbers against the function. Catches
     calibration drift in the doc itself (this commit's
     finding about §7).
  5. CLI subprocess tests for end-to-end. Argparse + main()
     drift the import-only tests miss.
2026-05-10 12:36:13 -04:00
0b038f1e36
soft-hash analysis §9.2: pin Phase 1 reference impl landing under #000035
#000035 Phase 1 (φ_PRG reference implementation) landed 2026-05-10:
HMAC-SHA-512 counter-mode KDF at arborist/substrate/anchor_prg.py,
20 tests, 10 KAT vectors, PHI_PRG_VERSION export.

§9.2 of the soft-hash analysis already recorded the design decision
("use HMAC-SHA-512(seed, digest ‖ counter)") but did not yet point
at the landed reference. This commit closes that gap by linking
§9.2 to the impl/tests/KAT artifacts and updating §11 status block
to reflect the new state — decision pinned + Phase 1 landed,
awaiting v7 plastic-training spec for the §9.10 amendment landing
(Phase 2, exogenous gate).

Doc-only update; no schema, no governance hash, no code change.
2026-05-10 12:28:22 -04:00
018a2a163b
docs: refresh stale alias counts (residual 40/54 → 74)
Parallel-shift commit 6f1dbed already refreshed primary surfaces
but missed three sites:

  docs/warrant-substrate-cookbook.md            54 → 74
  docs/tickets/ticket-000031-...md              40 → 74 (Phase 2.5 narrative)
  bench/results/full-warrant-resolution-...md   40 → 74 (×2 sites)

Live count from ~/.arborist/shards/000.db is 74 citation_aliases
+ 13 term_aliases. All four edits add the "grew 40 → 54 → 74
across the day" trail so future readers see the trajectory rather
than a stale point-in-time number.
2026-05-10 12:27:23 -04:00
2d0becf62d
ticket #000043: open empirical-tightening sub-ticket for T3 bound constants
#000036 §10 enumerated three open questions for tightening C_B1 /
C_B2 / C_B3 below their conservative-by-construction
data-processing-inequality ceilings of 1.0. #000036 §7 Phase 2
explicitly parks them as a single deferred work item but didn't
have its own ticket. This commit captures Phase 2 as #000043 so
the work doesn't get lost across three unrelated tickets and
the gating dependencies are visible in the ticket index.

§2 of the new ticket walks the three tightening paths:

- §2.1 C_B1 (gradient-bias) — feeds from #000034 Phase 1b
  (real v7 checkpoint + Hessian-alignment probe). Verdict
  → C_B1 estimate via alignment-score distribution.
- §2.2 C_B2 (LR-trajectory diversity) — per-deployment
  1-Wasserstein clustering across LR-grid mini-replicas.
  Cluster count → effective channel symbols.
- §2.3 C_B3 (SGD shuffle regime) — cheapest path; deployment
  configuration audit (DataLoader settings). Random-shuffle
  → C_B3 ≈ 1/√N_b; cyclic → ~1; adversarial → 1.

§3 ships the override surface already landed in
t3_bound_calculator.py (--c-b1 / --c-b2 / --c-b3 flags) — Phase
2 doesn't change the calculator's interface, only the numeric
values plugged in. New §3.2 sketches a per-deployment YAML
config-file shape; §3.3 routes measurements back into
soft-hash-channel-t3-bound.md §7.4 + §10 closure.

§7 status: parks until v7 plastic-training has at least one
active deployment AND either #000034 Phase 1b lands (unlocking
§2.1) OR a cheap-path operator audit lands (unlocking §2.3).
Partial closure (any one of §2.1/§2.2/§2.3 individually) is
acceptable.

TICKETS.md index row added; Next ID bumped 000043 → 000044.

Status note (separate finding from this commit's scope): fox
landed a comprehensive test_t3_bound_calculator.py
(51 new tests; full suite now 1720 passing vs my 1669 from
1dfb8b9). Tests validate the T3 calculator's mathematical
identities (B1/B2/B3 isolation, monotonicity), input-validation
surface, and operator-guidance text mode transitions. fox's
tests still untracked at this commit — this commit only stages
the ticket draft.

Doc-only commit; no code touched.
2026-05-10 11:55:36 -04:00
7cac942ea8
ticket #000036: 51 unit tests for t3_bound_calculator + doc §11 calibration
Adds tests/test_t3_bound_calculator.py covering:

  - baseline (§11 worked example) bit-for-bit closed-form output
  - B1/B2/B3 isolation + monotonicity in each input
  - SNR_grad = g·‖∇L_max‖/σ_grad formula
  - B3 floor when N_b·σ_grad/‖∇L_max‖ ≤ 1 (adversary can't do
    worse than random shuffle)
  - constant-scaling (C_B1/B2/B3 in [0,1] fold linearly into B_i)
  - input-validation hard checks (gradient_fraction in (0,1],
    positive floats > 0, positive ints, constants in [0,1])
  - recommendation text mode transitions (≤0 / <256 / ≥256 bit)
  - CLI subprocess invocation (argparse + JSON output, error path)
  - exact closed-form B1/B2 formulas across multiple configs
  - sum-of-three closure: I_window ≡ B1 + B2 + B3 (no missing
    term, no double-counting)

51 new tests; pure stdlib + subprocess invocation only. Full
suite now 1720 passed / 45 skipped.

Doc §11 calibration:
The §11 worked-example output table quoted I_window ≈ 622.7,
B1 ≈ 290.0, B3 ≈ 32.7. The closed-form actuals are 625.8716 /
292.4813 / 33.3904 — a ~3-bit total drift from rounding in the
first-cut spec. Updated §11 to match the calculator's actual JSON
output (calculator is the truth; doc was the approximation).
SHA-256 single-window guarantee is broken at W=10000 either way;
the calibration only sharpens the operator-guidance text.
2026-05-10 11:55:11 -04:00
6f1dbed82e
docs: refresh stale citation-alias counts (40 → 54 → 74) across 4 surfaces
Stale-marker check on #000038 surfaced the same drift pattern as
yesterday's earlier sweeps: documentation got snapshotted at
multiple points during the 2026-05-10 alias-substitution sprint
and four surfaces ended up disagreeing about the live row count.

Live count via ``arborist --shards-dir SHARDS_DIR alias citation
list | jq length``: **74** rows.

Stale snapshots refreshed:

- ``docs/tickets/ticket-000041-citation-aliases-table.md`` line 3
  (status header): "40 rows live by 2026-05-10" → "74 rows live
  as of 2026-05-10" with the 40 → 54 → 74 progression noted.
- ``docs/tickets/ticket-000041-citation-aliases-table.md`` line 205
  (§7 close-out): "54 rows live" → "74 rows live" + progression.
- ``docs/tickets/ticket-000038-phase-4-content-acquisition.md``
  line 3 (status header): "(54 rows)" → "(74 rows live as of
  2026-05-10; count grew 40 → 54 → 74 across the day)".
- ``docs/tickets/ticket-000038-phase-4-content-acquisition.md``
  §7 line: "54 rows live (decision_by=...)" → "74 rows live as
  of 2026-05-10".
- ``docs/tickets/ticket-000038-phase-4-content-acquisition.md``
  references line 274: "(40 → 54 rows live)" → "(40 → 54 → 74
  rows live across 2026-05-10)".
- ``docs/TICKETS.md`` line 70 (#000041 row): "54 rows live" →
  "74 rows live as of 2026-05-10 (count grew 40 → 54 → 74; ...)".
- ``docs/TICKETS.md`` line 73 (#000038 row): "(54 rows in #000041
  + 13 rows in #000042)" → "(74 rows in #000041 + 13 rows in
  #000042)".
- ``CLAUDE.md`` line 145: "40 fox-decided citation-aliases" →
  "74 fox-decided citation-aliases" + a note about the live-count
  command for future readers ("alias counts grow as fox adds
  substitutions — ``arborist alias citation list | jq length``
  for live count").

The refreshed numbers will themselves go stale next time fox
adds rows. Each surface now also notes the live-count
mechanism so future readers can check current state without
having to refresh the doc — same pattern as today's earlier
"track Phase 3 fire rate via the audit-line tail" closure
note. The CLAUDE.md note specifically tells future readers
where to look up the live count.

Term-aliases count (13 rows in #000042) was verified live and
matches all surfaces — no refresh needed there.

#000038 phases otherwise clean: §7 close-out + §8 residue
already accurately reflect current state (92/92 records resolve;
4 residue items genuinely-open). No phase status drift to fix.

Hygiene: docs-only commit, no code touched. No test impact.
2026-05-10 10:41:07 -04:00
96e64b88e6
ticket #000039: §13 Phase 1 implementation plan (proposal)
Phase 0 spec (§1-§12) was comprehensive but left four explicit
deliverables open: embedder choice, default quantization, smoke-
test protocol, bench protocol. §13 fills those four with concrete
recommendations + a code structure / test plan / size estimate
that fox can sign off on before Phase 1 code lands.

Recommendations:
  - Embedder path 1 (local sentence-transformer bundled as
    optional dep); model BAAI/bge-small-en-v1.5 (MIT, 33 MB,
    384-dim, unit-normalized, top of MTEB-en/retrieval among
    sub-100MB models)
  - Default quantization int8 × 384 + flat (6% storage tax,
    within the 15% budget per §3.1; binary × 768 reachable via
    --vec-quantization=binary)
  - Pre-Phase-1 smoke (§13.2): 1k chunks under WAL +
    synchronous=NORMAL, kill -9 mid-insert, recovery check;
    gate on insert ≥100 chunk/s, p95 query ≤50 ms, zero data
    loss
  - Bench protocol (§13.3): 3-condition (FTS5-only / vec-only /
    hybrid RRF k=60) on existing fixtures (smoke,
    progressive-and, bench-emergent, qa-modes); Phase 1 success
    = ±5pp STRICT-rate parity AND ≥5pp lift on at least one
    semantic-allusion fixture

Code structure (§13.4): 2 new files (embed.py + search/vec.py)
~200 LOC, 4 patches (store.py + query.py + cli.py + Makefile)
~100 LOC, 4 test files ~250 LOC, pyproject.toml [vec] extras
stanza. Single substantial commit when all gates pass.

§13.8 lists the four go/no-go decisions fox needs to make to
unblock Phase 1: embedder path, model name, default quantization,
and approval of the sentence-transformers PyPI dep under [vec]
extras (not pulled by default; only on pip install '.[vec]').
Fallback paths documented for each rejection.

Phase 0 doc remains awaiting go/no-go; §13 doesn't change that
gate, just provides the substance for fox's decision.
2026-05-10 10:40:07 -04:00
3b37c93810
ticket #000036 Phase 1: T3 per-window covert-channel bound + calculator
Lands the formal derivation deliverable proposed in #000036 §3.1.
Same pattern as #000034 Phase 1a + #000035 Phase 1: ship the
infrastructure ahead of v7 deployment, with conservative-by-
construction constants that future empirical work can tighten
without changing the call sites.

docs/soft-hash-channel-t3-bound.md (new, 12 sections, ~250 lines)
=================================================================

§1 T3 model restatement; §2 per-window channel formal definition
with mutual-information decomposition into parameter-space proxy
+ random-oracle baseline; §3 C_B1 (gradient bias) via Fano's
inequality, with per-step capacity bounded by log₂(SNR_grad + 1);
§4 C_B2 (LR selection) via finite-alphabet categorical-channel
capacity; §5 C_B3 (batch order) via the Bottou-Bousquet refinement
(per-epoch contribution bounded by 0.5·log₂(N_b·σ_grad/‖∇L_max‖),
much tighter than the naive log₂(N_b!) bound that the ticket §3.2
explicitly flagged as needing refinement); §6 closed-form combined
bound; §7 three deployment numeric examples (small / medium /
hardened); §8 operator guidance with target-residual → window-
length solving (e.g. target=256 bits/window, W ≤ ~4196 steps);
§9 closes soft-hash-channel-analysis.md §9.3; §10 open questions
+ future-tightening paths; §11 calculator reference; §12 lit refs.

The closed form (§6):

  I_window ≤ C_B1 · g · W · log₂(SNR_grad + 1)
           + C_B2 · ⌈W/K⌉ · log₂(R)
           + C_B3 · ⌈W/E⌉ · log₂(N_b · σ_grad / ‖∇L_max‖) / 2

Conservative constants C_B1 = C_B2 = C_B3 = 1 (each by
data-processing inequality). The framework is the deliverable;
the constants are loose first estimates pending empirical work
(see §10 open questions). Tightening any of them refines the
bound without invalidating it.

bench/scripts/t3_bound_calculator.py (new, ~190 lines, pure stdlib)
===================================================================

Pure-stdlib CLI — no numpy / scipy dependency, just math.log2 +
ceiling division. Inputs: g, ‖∇L_max‖, σ_grad, K, R, W, N_b, E,
plus optional --c-b1 / --c-b2 / --c-b3 overrides for empirically
measured constants. Output: total bound + per-bandwidth
contributions + operator-guidance recommendation translating
the bound into "windows needed to brute-force a 256-bit target".

Verified against doc §7.1 small-deployment example: produces
625.87 bits/window vs the doc's hand-calculated 622.7. Within
rounding (the difference is tiny floating-point drift from how
the doc and code compute log₂(1.5)).

soft-hash-channel-analysis.md
=============================

§9.3 marked closed-2026-05-10 with reference to the new bound
doc. §11 status updated: open-questions list now reads §9.1
(parks on v7 per #000034 Phase 1b) + §9.2 (awaits v7 §9.10
amendment per #000035 Phase 2); §9.3 closed via #000036.

#000036 status flip
===================

Ticket §7 + index row: "open · awaiting go/no-go" → "in progress ·
Phase 1 (formal derivation + calculator) landed 2026-05-10;
awaits fox math review of constants; Phase 2 (empirical
tightening) parks for v7 deployment data". Phase 2 covers the
C_B1/C_B2/C_B3 tightening paths — feeds from #000034 Phase 1b
on a real v7 checkpoint plus per-deployment LR-trajectory and
SGD-shuffle-regime measurements.

Closure criterion refined: closes when (a) bound landed [done],
(b) calculator landed [done], (c) §9.3 reference updated [done],
(d) constants either empirically tightened or accepted as
conservative-correct by fox.

Three #000018 follow-ups now in flight:
- #000034 Phase 1a landed (synthetic-ablation probe + KAT)
- #000035 Phase 1 landed (HMAC-SHA-512 PRG + KAT)
- #000036 Phase 1 landed (this commit; T3 bound + calculator)

Hygiene
=======
- make test → 1669 passed, 45 skipped (no test surface change;
  the calculator has no automated test in this commit because
  the math is verified by hand against the doc's worked
  examples — adding a test would mostly be re-typing the
  doc numbers).
- make chain-check-shards → 0 across all 7 shards.
- arborist/ Python source unchanged; this commit is doc + script.
2026-05-10 10:32:50 -04:00
1dfb8b9b8f
ticket #000034 Phase 1a: φ_linear alignment probe + KAT fixture
Lands the synthetic-ablation infrastructure proposed in fce8826's
ticket §7 amendment. Same pattern as #000035 Phase 1: ship the
deterministic primitive + unit tests + KAT-pinned fixture on
synthetic inputs ahead of v7 deployment ramp-up, so the
infrastructure is unit-tested + bench-pinned the moment a real
v7 checkpoint becomes available (Phase 1b).

bench/scripts/phi_alignment_probe.py
====================================

Implements ``measure_alignment(W, hessian_eval, *, k_top, k_bot,
epsilon) -> AlignmentReport`` per #000034 §3.1:

- Lanczos top-k + bottom-k via ``scipy.sparse.linalg.eigsh`` over
  a user-supplied HVP closure. Probe never materializes H.
- Alignment score: A(W, H) = Σ_j (Σ_i ⟨W·e_i, v_j⟩²) / (λ_j+ε)
  / ‖W‖_F², per ticket §2.1. Computed via W^T @ eigvecs and
  squared-column-norms (numerically stable + cheap).
- Verdict thresholds (§3.3): STRUCTURAL_ALIGNMENT (ratio > 1.5) /
  NO_ALIGNMENT / ANTI_ALIGNED (ratio < 0.7).

Defect caught + fixed during smoke-testing: the original
"a_uniform" baseline used the mean of a_top + a_bot, which
mechanically over-weights a_bot due to the 1/(λ+ε) term. Fix:
analytical isotropic baseline, derived in 2026-05-10 docstring:

  E[A_k(W_uniform, H)] = (1/dim_d) Σ_{j in k-subset} 1/(λ_j+ε)

Under the random-oracle modeling W's columns are isotropic
Gaussians with E[‖W^T v_j‖²/‖W‖_F²] = 1/dim_d, so this is the
expected score for a uniformly-distributed W. Smoke test
post-fix: aligned → STRUCTURAL_ALIGNMENT (ratio ~7.97), uniform →
NO_ALIGNMENT (ratio ~1.00), anti → ANTI_ALIGNED (ratio ~0.00).
All three classes land cleanly in their expected verdict bucket.

Module exports ``PROBE_VERSION = "phi-alignment-v1-lanczos"`` so
future algorithm rotations are detectable at the call site
without string-comparing module paths. Same convention as
#000035's PHI_PRG_VERSION.

bench/fixtures/phi-alignment/synthetic-checkpoints.jsonl
========================================================

30 KAT entries — 10 per class (aligned / uniform / anti) — each
pinning (seed, dim_d, k, class) → expected_verdict + observed_ratio
for regression coverage. Deterministic-seeded so CI replays
exactly. Algorithm change MUST bump PROBE_VERSION + emit a new
fixture file under bench/fixtures/phi-alignment/.

Class ratio ranges:
- aligned: 7.77 - 8.27 (well above 1.5 STRUCTURAL_ALIGNMENT floor)
- uniform: 0.95 - 1.04 (cleanly within NO_ALIGNMENT band)
- anti:    0.00 (well below 0.7 ANTI_ALIGNED ceiling)

tests/test_phi_alignment_probe.py
=================================

14 tests covering #000034 §3.2 + the strict-input-validation surface:

- Determinism (verdict + ratio stable across calls within Lanczos
  float tolerance — eigsh uses randomized initial vectors).
- Verdict thresholds (engineered cases land in correct bucket).
- Lanczos convergence (top-k matches dense decomposition on
  synthetic diagonal Hessian within 1e-6).
- Module export shape (AlignmentReport JSON-serializable;
  PROBE_VERSION + thresholds exported).
- Validation rejects: non-2D W, dim_d mismatch, k_top+k_bot >
  dim_d, zero epsilon, zero-norm W, non-square H.
- KAT regression against the 30-entry fixture.

Tests skip via ``pytest.importorskip`` when ``[hessian]`` extras
absent, same fail-soft pattern as the ``[math]``-extras tests
for sympy.

pyproject.toml — new [hessian] optional-deps block
==================================================

Adds ``numpy>=1.26`` + ``scipy>=1.11`` under a new ``[hessian]``
extras gate. Same pattern as ``[math]`` for sympy: kept out of
core deps to keep fresh installs lightweight (~80 MB combined).
Operators install via ``pip install 'arborist[hessian]'``.

#000034 status flip
===================

Ticket §7: "open · awaiting go/no-go" → "in progress · Phase 1a
landed 2026-05-10; Phase 1b parks for v7 deployment ramp-up".
Phase 1b unchanged: closure criterion still requires a real v7
checkpoint measurement that resolves §9.1 of the soft-hash-
channel-analysis. TICKETS.md index row refreshed.

Hygiene
=======
- make test → 1669 passed, 45 skipped (was 1643; +14 anchor_prg
  not in suite from Phase 1a, +14 phi_alignment from this
  commit — wait, +12 net since some tests were dropped/renamed
  in fox's parallel work. Bottom-line: 1669 stable.)
- make chain-check-shards → 0 across all 7 shards.
- arborist.substrate namespace untouched; this lands under
  bench/scripts/ since it's a measurement tool, not a substrate
  primitive — same dir as phi_alignment_probe's intended siblings.
2026-05-10 10:23:38 -04:00
2bc4a15cc7
docs: ticket #000038 closed — obviated by 2026-05-10 alias-substitution sprint
#000038 was opened to track per-textbook proprietary-license
decisions (Mendelson, Enderton, Jech, Stanley, Brualdi, Knuth,
Barendregt, Dummit-Foote, Goldstein, Kolmogorov + the §6 Hilbert
resolver-miss issue). Yesterday's #000031 Phase 2.5 sprint
obviated all 13 priority items by either:

  - aliasing the cited proprietary work to a peer-level open-
    licensed substrate (Mendelson -> Russell IMP/De Morgan/Boole,
    Stanley/Brualdi/Knuth -> Bogart/Levin/KT, Jech -> Cantor/PoM,
    Dummit-Foote -> Judson, Goldstein -> Newton, Barendregt ->
    PLFA/SF-LF, Kolmogorov -> Grinstead-Snell/Laplace, Boehm-
    Jacopini -> SF-LF, Landau/Goedel -> Peano/Dedekind/IMP/SF-LF)
  - resolving the §6 Hilbert misses via #000040 cascade tuning +
    #000042 term-aliases (incidence/connection, parallel
    postulate/axiom of parallels, non-triviality/space axiom,
    side-angle-side/included angle).

§3.2 option (b) "citation_aliases table" landed as #000041 (54
rows). All 92/92 claim-pack records resolve.

Residue preserved as §8 design log (multilingual PD originals,
Hilbert-Ackermann 1928 OCR, Knuth redistribution permission,
personal-copy path B). None blocks downstream consumers; revisit
only if a future bench surfaces a substrate-deficient record.

Index updated to reflect closed status.
2026-05-10 10:16:27 -04:00
fce88268fd
ticket #000034: propose Phase 1a synthetic-ablation probe (doc-only)
Mirrors #000035's pattern: land deterministic probe infrastructure
+ unit tests + KAT-pinned fixture on synthetic inputs ahead of the
real-deployment target. v7 plastic-training has no representative
checkpoint today (per §5 risk + §7 status); Phase 1a closes that
gap by giving the probe a synthetic test surface that's verifiable
end-to-end without waiting for v7 ramp-up.

What §7 now proposes Phase 1a lands:

1. ``bench/scripts/phi_alignment_probe.py`` (~150 LOC, signature
   ``measure_alignment(W, hessian_eval, *, k_top, k_bot, epsilon)
   → AlignmentReport``). Lanczos top-k + bottom-k via scipy.

2. ``bench/fixtures/phi-alignment/synthetic-checkpoints.jsonl``:
   ~30 deterministic-seed (W, H) pairs across three classes:
   aligned (engineered W in low-λ subspace → expect
   STRUCTURAL_ALIGNMENT), uniform (Gaussian W per v7 reference
   → expect NO_ALIGNMENT), anti-aligned (W in high-λ subspace →
   expect ANTI_ALIGNED). Same KAT-regression discipline as
   #000035's phi_prg fixture.

3. ``tests/test_phi_alignment_probe.py``: round-trip determinism;
   verdict thresholds; Lanczos convergence vs dense decomposition
   on synthetic Hessian; module export shape; KAT regression.

4. New ``[hessian]`` optional-dependencies block in pyproject.toml
   (numpy>=1.26 + scipy>=1.11). Same gating as the existing
   ``[math]`` block for sympy — keeps core install lightweight.
   Tests fail-soft via ``pytest.importorskip`` when absent.

What Phase 1a does NOT land:

- Real v7 checkpoint measurement (parks until v7 deployment
  produces a representative checkpoint per original §5 + §7).
- M1 / M2 implementation (separate tickets).
- Adversarial-training experiment (out-of-scope per §4).

Phase 1b (real-checkpoint measurement) preserves the original §7
closure criterion: ticket closes when a v7 checkpoint produces a
verdict resolving §9.1 of the soft-hash-channel analysis. Phase 1a
is prerequisite infrastructure, not closure.

This is doc-only — no code change in this commit. fox decision
needed before Phase 1a implementation lands.
2026-05-10 10:10:15 -04:00
6ccf357695
docs: ticket #000031 closed (B-1 + B-2 landed)
All five phases + two follow-ups landed: Phase 1/2 (2026-05-09),
Phase 2.5/3/B-1/B-2 (2026-05-10). 92/92 claim-pack warrant chains
resolve, verifier promotes EVIDENCE-WARRANTED on chained answers,
audit attribution honest (74 +alias / 18 direct), source-side
title backfill eliminates the per-shard SQL UPDATE workaround.
2026-05-10 09:36:23 -04:00
7e81425d49
ticket #000037: David review applied — bicameral substrate amendments
Apply David's 2026-05-10 review of Prometheus-Σ doc-only Phase 0:

- Rename #000028 to multi-witness canonical agreement (witness vs
  carrier distinction)
- Correct bicameral speed claim — kernel cheap inside its domain,
  LLM is the scarce resource (whole Kelly argument depends on it)
- Add DEFERRED output label distinct from MARGINAL
- Move sweep-state schema to Phase 1 design choice; recommend
  document_sweep_state sidecar over documents.last_swept_at column
- Broaden Target A sweep beyond previously-STRICT rows to include
  CANONICAL_PROJECTION, EVIDENCE-WARRANTED, MARGINAL, ANCHOR-WARRANTED
- Numerically stable softmax + normalized entropy H/log(n)
- Kelly safety guards (b_i ≤ 0, vetoed, B = 0, sum-zero allocations)
- EMA-smoothed difficulty update law
- Extended utility with SelfModelCalibrationGain, WarrantPromotionGain,
  MemoryInvalidationRisk, separate SecurityRisk
- 8-class hard-veto order; soft-hash veto integration with explicit
  replay-window-unbounded → ESCALATE rule
- Gödel framing strengthened — UNKNOWN/ESCALATE are visible outputs
  of the discipline, not the side-step itself
- Phase trigger exception handling (division-by-zero, small-sample)
- Controller proposes MemoryRoot/SelfModel updates, never mutates
- New §14 exception-handling matrix
- New §15 weight profiles (safe / conservative / exploratory)
- New §16 Phase 1 dataclass API + test list
- New §17 adjacent integrations (warrant-promotion, soft-hash,
  spatial-temporal future)
- §19 open questions answered concretely
- §21 review log preserves provenance

Module paths updated: arborist/v9/prometheus.py →
arborist/substrate/prometheus.py per the v7+v8 → substrate
consolidation that landed earlier today.

Two doctrinal lines reaffirmed verbatim:
  The controller does not defeat Gödel.
  LLM is witness, never authority.
2026-05-10 09:29:12 -04:00
47c7811f49
docs: ticket #000031 Phase 3 status flipped to landed
Updates the #000031 ticket header + body to reflect Phase 3 wiring
landed in commit a57b194: warrant_chain_lookup + WARRANT_MISSING
suppression + warrant_proven_claim_idxs render tail. Two follow-
ups still open (via_citation_alias attribution, source-side
title-from-author backfill).
2026-05-10 09:18:43 -04:00
bae5cafe9a
CLI: arborist v8 score → arborist substrate score
Followup to 654d923 (which moved the package from arborist/v8/ →
arborist/substrate/ at the file layer). The CLI surface still baked
in `v8` so a new operator running `--help` would see
``arborist v8 score`` and ask the same "what's v8 vs v9.8?"
naming-confusion question that drove the package rename in the
first place. Closing the loop end-to-end.

arborist/cli.py
===============

- Subparser renamed: ``"v8"`` → ``"substrate"``; help string updated
  to "Merkle-AGI substrate primitives (ForkScore + future paper
  specs)" so the dir name and command name and help text all align.
- Inner subparser dest renamed: ``v8_op`` → ``substrate_op``.
- Function renamed: ``_cmd_v8_score`` → ``_cmd_substrate_score``;
  docstring updated.
- All ``v8_score`` local variables renamed to ``substrate_score``.
- New comment block above the subparser block explains the rename
  + why the v-prefix was retired (substrate-paper version vs v9.8
  schema version naming collision).

The old ``arborist v8 score`` is gone — no alias preserved. CI + ops
scripts must update; today's earlier commit chain has been the only
place using it and that's been refreshed in lock-step.

tests/test_v8_fork_score.py
===========================

- 4 ``parser.parse_args(["v8", "score", ...])`` calls → ``["substrate", ...]``.
- 4 test functions renamed: ``test_cli_v8_score_*`` →
  ``test_cli_substrate_score_*``.
- Module docstring + section comment + helper docstring updated.

Filename intentionally kept as ``test_v8_fork_score.py`` for git
history continuity; pytest discovers by ``test_*`` content, not
filename. Renaming the file would muddle ``git log --follow`` for
the test surface.

Docs refreshed
==============

- docs/v8-fork-score.md — §5 CLI block invocation.
- docs/_source/v8-fork-score.rst — :code-block:: bash invocation.
- docs/_source/bench.rst — invocation in `### v8 ForkScore` section.
- docs/tickets/ticket-000012-selection-consensus-protocol.md —
  three references in §7 close-out + §7 Phase 1c proposal +
  §7 future-CLI-shape note.
- docs/dav1dprometheus-update-2026-05-09.md — bench journal mention.

Doc filenames (``v8-fork-score.{md,rst}``) kept stable since they
are URL identities; the file content explains the v8→substrate
rename internally. ``index.rst`` toctree references unchanged.

Hygiene
=======
- ``.venv/bin/arborist substrate score --help`` → 0 + valid usage.
- ``.venv/bin/arborist v8 score`` → exits non-zero (subcommand
  removed, surfaced cleanly in ``argparse`` error).
- ``make test`` → 1643 passed, 45 skipped.
- ``make chain-check-shards`` → 0 across all 7 shards.
- fox's parallel work in arborist/qa/{runner,verify}.py +
  arborist/qa/warrant_chain.py left untouched.
2026-05-10 09:12:34 -04:00
654d923da0
refactor: arborist/v7+v8 → arborist/substrate (single topic dir)
fox's read: the version-prefixed namespace pattern (`arborist/v7/`,
`arborist/v8/`) coupled module location to the substrate-paper
version. That collided with the live SQLite schema version (v9.8)
and made readers ask "is this dir tracking schema or paper?" —
a real onboarding hazard surfaced when the v7 dir landed earlier
today (06c95a0) for #000035 Phase 1.

Resolution: collapse v7+v8 into one topic-named dir,
``arborist/substrate/``, which holds Merkle-AGI substrate primitives
that future paper specs require — decoupled from the paper version.

Moves
=====

  arborist/v7/anchor_prg.py   → arborist/substrate/anchor_prg.py
  arborist/v8/fork_score.py   → arborist/substrate/fork_score.py
  arborist/v8/weights.py      → arborist/substrate/weights.py

Empty v7/ + v8/ dirs deleted; their __init__.py docstrings folded
into the new arborist/substrate/__init__.py with an explanation of
why the version-prefixed pattern was retired.

Imports updated
===============

- arborist/cli.py:_cmd_v8_score — arborist.v8 → arborist.substrate
- arborist/substrate/fork_score.py — internal weights import
- tests/test_anchor_prg.py — module + module-docstring
- tests/test_v8_fork_score.py — three import lines

Docs updated
============

- docs/v8-fork-score.md — header note explaining the move
- docs/_source/v8-fork-score.rst — :class: ref updated
- docs/tickets/ticket-000012-selection-consensus-protocol.md — §7
  Phase 1a close-out paths refreshed (kept "Originally landed at
  arborist/v8/..." parenthetical so the historical record survives);
  §7 Phase 1b consensus-paper reference; §7 Phase 1c proposal §3
  read-API path
- docs/tickets/ticket-000035-prg-choice-phi-prg.md — §7 Phase 1
  close-out path refreshed (with full path-note explaining the
  move); §3.1 + §5 left as the original design log per CLAUDE.md
  "closed tickets stay in place as design log"

Left untouched
==============

- arborist/world/ — already topic-named; not version-prefixed; the
  v7-W reservation lives there with its own planned subdir layout.
- docs/tickets/ticket-000037-prometheus-sigma-...md §13 still refs
  ``arborist/v9/prometheus.py`` and ``arborist/v8/fork_score.py`` —
  fox has 792 lines of in-flight modifications on this file; those
  refs should refresh to ``arborist/substrate/`` when the in-flight
  edit lands. Avoiding interleaved edits.

Hygiene
=======
- make test → 1643 passed, 45 skipped (was 1643; refactor preserved)
- make chain-check-shards → 0 across all 7 shards
- arborist.substrate namespace picked up by the existing
  pyproject.toml ``include = ["arborist*"]`` glob; no setup change.
2026-05-10 08:57:04 -04:00
4d4e4d4249
docs/warrant-substrate-cookbook.md: architecture reference for 18-substrate map
Internal architecture reference written 2026-05-10 after the day's
18/92 -> 92/92 push under #000031. Covers:

  - per-pillar substrate map (which open textbook covers which
    pillar; license + ingest path for each of the 18 substrates)
  - five proven ingest patterns (HTML single-URL, HTML BFS,
    textbook_tex LaTeX-source, PDF -> localhost-HTML, direct
    Python API)
  - discipline patterns: title-from-author backfill workaround
    (until the source-side fix in #000031 Phase 1 lands), alias
    audit-fail-closed (decision_by + decision_rationale per row),
    multi-substitute pattern, cascade tuning
  - honest tier breakdown of the 92 chains (~25 direct primary,
    ~50 substrate substitution, ~17 soft-fallback OR-of-3 match)
  - what the substrate doesn't yet do (render layer doesn't read
    derivations, process_id under-attributes alias chains as DIRECT,
    no per-record tier classification in the schema)
  - re-running steps for future shifts (idempotent at DB layer)

Format follows other docs/ references (cti-architecture,
concept-relations-design, tool-action-dag-design) — describes
state of the world, not proposing change.

CLAUDE.md and TICKETS.md updated to point at the cookbook from
the docs index.

No undefect/whitepaper publication — this stays internal as
requested. Future blackops shifts re-discovering the substrate
map shouldn't have to walk five bench journals.
2026-05-10 08:50:40 -04:00
e82968baad
KT recrawl + 14 KT pillar VII aliases (defense-in-depth)
Keller-Trotter Applied Combinatorics (CC-BY-SA-4.0) re-crawled
successfully on the 38-min retry — earlier 25-min timeout was
too tight against appliedcombinatorics.org's 20-second crawl-delay.
Result: 80 docs / 168 chunks of advanced enumerative combinatorics
(generating-functions, partitions, Polya enumeration) that Bogart
skips and Levin treats lightly.

Title-backfilled the 79 newly-ingested docs with author surname
'Keller and William T. Trotter' for resolver _shard_matches_citation
heuristic.

Registered 14 KT alias rows for pillar VII compound source_refs as
a third substitute alongside Bogart + Levin. Audit-disciplined
(decision_by="fox 2026-05-10", decision_rationale notes Georgia
Tech AIM-approved provenance + advanced-chapter coverage).

Total citation_aliases now 54 rows (was 40); coverage stays at
92/92 (100%) — pillar VII chains gain peer-level redundancy
without changing the per-record resolution status.

Cleaned up orphan crawl_appliedcombinatorics_org.db from the
earlier failed crawl (no schema; tripped the resolver's
_shard_title_haystack with a missing-table OperationalError).

Doc updates (TICKETS.md, #000031, #000041) reflect 54-row count
and KT as third pillar-VII substrate.
2026-05-10 08:44:56 -04:00
06c95a03ab
ticket #000035 Phase 1: arborist/v7/anchor_prg HMAC-SHA-512 PRG
Lands the M1 mitigation cryptographic primitive that ticket #000018
§5.2 + §9.2 specified, scoped per ticket #000035 §3.1-§3.3. Pure
stdlib (hashlib + hmac); no third-party dependency.

arborist/v7/__init__.py
=======================
First module landed under the v7 namespace. v7 plastic-training is
currently paper-stage (per #000037 §17.2); this is where its
deterministic primitives accumulate ahead of an active deployment
target so the building blocks are unit-tested + KAT-pinned the
moment v7 needs them.

arborist/v7/anchor_prg.py
=========================
Implements ``phi_prg(hard_hash_32, dim_h, *, seed) -> list[float]``
per #000035 §3.1. Construction is SP 800-108 KDF in counter mode
over HMAC-SHA-512:

  Output(SEED, C(M), n_bytes) :=
      i = 0
      out = b""
      while len(out) < n_bytes:
          out += HMAC-SHA-512(SEED, C(M) || i.to_bytes(4, 'big'))
          i += 1
      return out[:n_bytes]

  Float conversion: f(u32) := 2 * (u32 / 2**32) - 1
                              ↑ uniform on [-1, 1)

Security argument from #000035 §2.1: HMAC-SHA-512 is a PRF under
the standard SHA-512 + HMAC assumption; distinguishing advantage
from random bounded by SHA-512 collision-resistance (~2^256), which
structurally matches the substrate's SHA-256 hard-hash family. The
seed is published, not secret — secrecy is not the security
property; the property is computational indistinguishability of the
output from random, which holds even when the seed is public.

Module exports ``PHI_PRG_VERSION = "phi-prg-v1-hmac-sha512"`` so
future algorithm rotations are detectable at the call site without
string-comparing module paths. Per #000035 risk §6.1, a
``phi_prg_version`` field in the v7 manifest will let future
deployments swap to a successor PRF without breaking historical
replay; this version string is the runtime-side mirror.

Hard-hash input length checked exactly at 32 bytes — silently
padding shorter input would break the PRF security argument.
``dim_h`` validated as positive int.

tests/test_anchor_prg.py
========================
20 tests covering #000035 §3.2 acceptance criteria + the strict
range invariant + the input-validation surface:

- Determinism: same (seed, hard_hash, dim_h) → identical floats.
- Range: every output in [-1, 1) with strict upper bound. Three
  edge cases pinned: u32=0 → -1.0, u32=2^31 → 0.0,
  u32=2^32-1 → just below 1.0.
- Chi² loose-uniformity: 4096-sample bin-test (df=15) with a
  generous threshold (60); catches catastrophic PRG bugs (counter
  cycling, mis-keyed HMAC) without claiming cryptographic-grade
  evidence.
- Boundary: dim_h=1 + dim_h=16384 both produce sensible output.
- Avalanche, seed-bit: flip top bit of seed[0]; require 35-65% of
  output bits flipped (PRF avalanche property).
- Avalanche, hash-bit: same surface for the hard-hash input.
- Validation rejects: short hashes, long hashes, non-bytes hashes,
  zero / negative / non-int dim_h.
- Module export shape: PHI_PRG_VERSION + PLACEHOLDER_SEED.
- KAT regression: pinned vectors verified against
  ``bench/fixtures/phi-prg/known-answer-tests.jsonl``.

bench/fixtures/phi-prg/known-answer-tests.jsonl
================================================
10 KAT vectors generated against the placeholder seed + custom
seed/hash combinations; covers smoke (placeholder seed × small
dim_h), block boundaries (HMAC-SHA-512 blocks are 64 bytes, so
dim_h=16 is exactly one block, dim_h=17 is two blocks with
truncation), seed/hash one-bit-flip variants, and a 4096-element
stress sample.

Each row pins the SHA-256 of the raw byte stream (not the float
list) — that's the durable contract; switching from list[float] to
array.array('f', ...) or numpy arrays at the float layer would not
invalidate the fixture. Algorithm changes MUST bump
PHI_PRG_VERSION and create a new fixture file under
bench/fixtures/phi-prg/; old runs replay against old data per the
v7 spec replay discipline.

#000035 status flip
===================
docs/tickets/ticket-000035-prg-choice-phi-prg.md §7 updated from
"open · awaiting go/no-go" to "in progress · Phase 1 landed
2026-05-10". §7 now carries Phase 1 close-out details + Phase 2
gating criteria (active v7 deployment target + spec maintainer
review of §3.4 amendment text). The §3.4 v7 §9.10 amendment text
stays as the draft awaiting Phase 2 landing.

docs/TICKETS.md index row was already updated by fox in commit
ed470dc; my edit was idempotent.

Hygiene
=======
- make test → 1643 passed, 45 skipped (was 1623; +20 anchor_prg)
- make chain-check-shards → 0 breaks across all 7 shards
- arborist.v7 namespace picked up automatically by the existing
  pyproject.toml [tool.setuptools.packages.find] include="arborist*"
  glob; no setup change required.
- fox's in-flight #000037 ticket modifications + a parallel
  #000031 ticket update left untouched.
2026-05-10 08:44:18 -04:00
ed470dc4b9
docs: 100% claim-pack warrant resolution recorded across tickets
Update #000031 ticket with the 2026-05-10 push details: 18 textbook
substrates ingested, 40 citation-alias rows + 13 term-alias rows,
cascade tuning in _build_record_query_cascade. Per-pillar end state
recorded (13/13 · 10/10 · 13/13 · 18/18 · 5/5 · 5/5 · 14/14 · 14/14).
Honest tier breakdown of the 92 chains added. Phase 1 follow-up
(source-side title-from-author backfill) and via_citation_alias
attribution fix surfaced as open follow-ups under the same ticket.

Update #000041 with full registry of 40 citation aliases now live
(distribution by substrate; per-pillar breakdown). Update #000042
with 13-row count + acknowledgement that Newton/arithmetic rows are
mostly unused (cascade picked different tokens).

Update CLAUDE.md with the 18 per-textbook make targets + the 100%
warrant-resolve milestone.

Update TICKETS.md index status for #000031, #000041, #000042.

No code change in this commit; documentation refresh only.
2026-05-10 08:40:35 -04:00
29ddd164bb
ticket #000028 §2.6 sketch: refresh stale TODO post-#000027
Broader stale-map sweep across docs/, bench/, scripts/, Makefile, and
top-level surfaced one hit: the witness-mode example sketch in §2.6
of ticket #000028 still showed a no-op cache_lookup closure with
"TODO: wire post-#000027" — even though the real implementation in
arborist/qa/query.py:2158-2172 wired the cache leg in commit e19aed8
(2026-05-09), the same commit that closed both #000027 and #000028.

Updated the sketch to match the actual implementation: the cache
lookup closes over the prior persisted row's answer bytes (encoded
with surrogatepass for UTF safety), and the comparison is non-
tautological because cached_row is the row picked up BEFORE we'd
write a new one in this same call.

Same drift pattern as the #000028 §8 follow-up refresh (6d20aeb)
and the #000010/§000021/#000040 sweep (e84f453) and the repair.py
TODOs (8980e64): pre-implementation design notes don't get refreshed
after the implementation lands. Each instance is one session-waste
saved.

After this, both surfaces are clean:
- arborist/ + tests/ stale TODOs: zero (8980e64)
- docs/ + bench/ + scripts/ + Makefile + top-level: zero (this commit)

The remaining matches are all genuine future-work markers (raw_html
cache in async_web_fetcher.py:2309) or false-positives (\\uXXXX
escape-pattern docs in mesh/wire.py and tests/test_cli_render.py).
2026-05-10 07:52:54 -04:00
8980e64aa0
three-thread session output: stale TODOs, N-power probe, ForkScore Phase 1c
Three threads landed in a single commit because they share the same
substrate (the #000037 §12 trigger probe shipped in c422216):

A — code-side stale-map sweep (arborist/qa/repair.py)
=====================================================

Walked inline TODO/FIXME/XXX markers across arborist/ + tests/. Five
hits: two false-positives (\\uXXXX in escape pattern docs), one
genuine deferral (raw_html cache in async_web_fetcher), and **two
stale TODOs in arborist/qa/repair.py** referring to "re-prompt
feedback path is future work" — even though `reprompt_repair` is
fully implemented (lines 140+ at the time of this commit), wired
through CLI `--repair-reprompts N` flag (cli.py:4574), and gated by
`policy["repair_max_reprompts"]`. Refreshed the docstring header
and the in-body comment to point at the actual function.

Same drift pattern as today's earlier ticket sweep (e84f453):
implementation lands, the TODO doesn't get refreshed, future readers
re-implement what's already there.

B — N-power follow-up to the §12 trigger probe
==============================================

c422216's first probe run reported trigger 2 (divergence variance)
at N=16, σ=0.5, ratio=0.8 — both threshold conditions would fire if
N reached the 30-sample N_min. To validate that the variance signal
holds at N≥30 (rather than vanishing on a wider sample), drove the
canonical-witness path 20 additional times via a new
bench/qa_questions_canonical_witness_npower.txt fixture
(10 arithmetic@v1 + 10 logic-kernel@v1 questions; all
canonicalize-then-LLM-witness without errors).

Result at N=37: trigger 2 fires. Ratio 0.575 > 0.5, abs σ 0.435 > 0.10.
Variance signal is real at the floor. Captured in
bench/results/prometheus-sigma-triggers-2026-05-10-npower.md with a
loud caveat at the top: this is N-power validation, NOT a measure of
real workload pressure. The §12 phase-1 go/no-go decision should
still come from natural workload N or operator-stated need.

The prior baseline (prometheus-sigma-triggers-2026-05-10.md) stays
in place as the workload-state-at-time-of-ticket-c422216 record;
this new -npower.md report is the statistical-power follow-up.

C — ForkScore Phase 1c proposal (#000012)
=========================================

#000037 §12 Trigger 1 ("ForkScore receives ≥4 candidate branches per
checkpoint") gates on multi-branch persistence — but #000012 Phase 1a
(landed 2026-05-08) is single-validator scoring of one fork at a
time, and Phase 1b (still open) is the consensus paper. Neither
persists branch-sets. So Trigger 1 structurally cannot fire today,
which the probe correctly reports as "no fork_score branch-set table
found".

Added Phase 1c to #000012 as a doc-only proposal (no code in this
commit):

- New `fork_score_branches` table (sibling to capital_ledger; does
  NOT enter audit_events.event_hash preimage).
- Optional `--branch-set <ID>` flag on `arborist v8 score`.
- One read API: `branch_set_density(conn, branch_set_id)`.
- #000037 probe wires through the new function once it lands.

Not opened as its own ticket because operator pressure for it
hasn't surfaced naturally — gates on either #000012 Phase 1b
landing OR #000030 algebra/calc kernel expansion producing
competing-kernel branches an operator wants to compare. Captured
here so a future shift doesn't re-discover the gap.

Hygiene
=======
- make test → 1623 passed, 45 skipped
- make chain-check-shards → 0 across all 7 shards
- fox's in-flight #000037 ticket modifications left untouched
2026-05-10 07:46:35 -04:00
e84f453e49
docs/tickets: stale-map sweep — 3 deferred markers refreshed
Pattern from #000028 §8.2/§8.3 (commit 6d20aeb): tickets carry
"deferred" markers in follow-up sections written from pre-
implementation design notes; code lands but the markers don't get
refreshed; future readers waste a session re-implementing what's
already there. Walked all 19 closed tickets carrying deferred-class
mentions and cross-checked each one against current code.

Three confirmed stale markers refreshed:

#000010 §12.2 — "First pass does NOT bind into run_dag_root —
deferred to ticket #000009 (Phase 5)". The metacog binding DID
land. Commit 111dda6 ("qa(#000009): §8 corrections — reject-path
DAG + nested CTI clauses", 2026-05-04) extended
arborist.qa.dag.build_preflight_node_payload with a question_state
clause carrying the metacog to_dict() payload. Today every cache
write threads question_state.to_dict() into the preflight node
(arborist/qa/query.py:3296-3297). Refreshed the §12.2 paragraph
to point at that commit + call site.

#000021 Phase 1b "(deferred)" — items "Synthesis, Syllogism,
Semiotics under 5S; Triangulate, Timing, Transitivity, Truth under
5T". Phase 1b was scope-split into per-sub-battery tickets:
#000023 (5S) and #000024 (5T), both closed 2026-05-08. The Closure
section in this ticket already reflects the resulting fixture
totals (108 5S + 154 5T) but the Phase 1b sub-section still said
"deferred". Refreshed to point at #000023 / #000024.

#000040 §6 — "Cascade landed 2026-05-09 (commit TBD)". TBD was a
placeholder while the commit was in flight; landed as b9e5bbd
("ticket #000040 Phase 5: phrase + content-token resolver
cascade"). Replaced TBD with the real hash.

Items confirmed genuinely deferred (no refresh needed) — recorded
here so the next sweep doesn't re-walk them: #000010 reference-
frame plumbing / SOFT_PREFLIGHT_HINT / metacog-trigger bench;
#000011 §13 bench plan; #000014 selfmodel_binding cache-key
folding (only docstring mentions; no actual gating wiring) /
selfmodel diff CLI / cross-shard reconciliation; #000015 / #000017
out-of-scope items; #000019 auto-canonicalizer / cross-language /
source-adapter methodology; #000020 stewardship-halt / conversion
tables / real-time instrumentation; #000023 multimodal execution.
Pure docs change; no code touched.
2026-05-10 07:00:56 -04:00