From 3b301267ff3cb6a4599737c6f130e32a5294b9fb Mon Sep 17 00:00:00 2001 From: "russell@unturf.com" Date: Sun, 10 May 2026 18:04:24 -0400 Subject: [PATCH] =?UTF-8?q?docs:=20#000037=20=C2=A720+=C2=A717.1=20refresh?= =?UTF-8?q?=20+=20#000025=20Phase=201f=20closure=20+=20cookbook=20harvest?= =?UTF-8?q?=20section?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three coordinated doc landings tying together fox's evening #000037 → #000025 closed-loop work (commits f625cac through ff1752c + 8999b55): == #000037 §20 Status — stale prose refresh == §20 said "open · awaiting go/no-go" but file header + TICKETS.md both say "in progress" with phases 0/1/1.b/2 all landed. Refresh §20 to reflect actual state with per-phase commit anchors: - Phase 0 (doc): landed; David review applied per §21 - Phase 1 (pure-function controller): f625cac (arborist/substrate/prometheus.py + test_prometheus.py) - Phase 1.b (gap-close): f9f5ae4 (§14 row 4 Hermes-saturation guard, §13 step 11 falsification-fixture proposal, §15 entropy + memory gates weight-tunable, ESCALATE > QUARANTINE > REJECT priority cascade) - Phase 2 (advisory audit writes): a786d6d (prometheus_audit.py + controller_events sibling table; does NOT enter audit_events.event_hash preimage) - §12 Trigger 2 fired 2026-05-10: divergence variance ratio 0.575 > 0.5 with N=37 — Phase 1 opening is now empirically gate-satisfied per 8999b55 - Phase 3: deliberately NOT landed; replaced with read-only dry-run simulator surfacing five design findings (see §22) - Closed-loop signal: 40 corpus-derived 5F fixtures harvested per ff1752c == #000037 §17.1 — "78 atomic claim-pack records" → 92 == §17.1 said "#000031 Phase 2 has 78 atomic claim-pack records that max out at ANCHOR-WARRANTED". #000031 closed at 92 records (78 was an interim count during Phase 2). Refresh with the journey (78 → 92) + AUTOCOUNT-tagged via db-where metric so future drift fires immediately. Note the resolution context: all 92 now resolve via 74 citation-aliases + 13 term-aliases under #000031 Phase 2.5 + B-1 + B-2. The unconscious sweep drains the ANCHOR-WARRANTED → EVIDENCE-WARRANTED promotion backlog (derivations.proof_blob rows still need computation even on resolved chains). == #000025 §11 Status — Phase 1f closure note == #000025 file header lists Phase 1a/1b.2/1c/1d/1e but the §11 Status section was frozen at "Open · awaiting go/no-go" — a two-versions-old prose snapshot. Refresh with the per-phase landing trail + add a Phase 1f section for the corpus-derived falsification harvest that ff1752c shipped: - Phase 1f closes the controller → 5F battery loop fox designed in #000037 §3 ("Divergence → candidate falsification fixture") - bench/scripts/harvest_falsification_proposals.py reads qa.db, stratifies top-20-by-cache_key per audit_mode (HYBRID + UNGROUNDED), writes the 41-line JSONL pack (1 _meta + 40 fixtures, AUTOCOUNT-tagged via fixture-rows) - Every fixture row carries _harvest_meta with cache_key, witness_divergence at harvest time, audit_mode_at_harvest, harvest_threshold, source_ticket: "#000037 §13 step 11" - 5F battery exercises them every test run; test_5f_falsification_harvested_pack_runs_clean asserts error_detection_rate == 1.0 by construction (every harvested row IS a falsification) - Self-amplifying — coverage grows with corpus, not with hand-curation Listed open items (§10.11 / 10.13 / 10.14) preserved verbatim from file header so the index claim "still open" stays in sync. == Cookbook: new "Adjacent: live-corpus → bench-fixture harvest" section == New section in warrant-substrate-cookbook.md between "Re-running the substrate build" and "References" documenting the harvest pattern. Three discipline patterns reused from the textbook substrate noted explicitly: 1. Attribution metadata on every derived artifact (same shape as derivations.proof_blob carrying inclusion proofs back to source chunks) 2. Determinism via sort-and-cap (same shape as citation-alias cascade's "top 5 AND-join then top 3 OR-join" stratification) 3. Pin the metadata contract in tests (same shape as the cookbook appendix's discipline pins — silent regression becomes loud test failure) Plus the reusable recipe for any controller emitting Proposal records: define the dataclass, write a harvester filtering + stratifying, pin metadata in tests, wire a make target. References section gets four new entries pointing to #000037, #000025, the harvest script, and the fixture pack. == Drift caught + refreshed during this commit == While editing the cookbook, the AUTOCOUNT regression test (from fc5ba50 / 6c6defb) caught two stale counts from fox's in-flight prometheus work: - test_prometheus.py: 36 → 42 (fox's uncommitted +6 for Phase 1.c sweep weight profile) - test_prometheus_audit.py: 14 → 17 (fox's uncommitted +3) Refreshed both inline + in the test/code-density table (prometheus row test LOC also bumped 804 → 955 to match wc -l). This is the harness firing exactly as designed — fox's uncommitted tests changed live state and my doc claims went stale within minutes. The test message named the file + line + claimed-vs-live, refresh was a 60-second turnaround. == Verification == $ pytest tests/test_doc_counts.py 3 passed in 2.92s $ pytest tests/ -q 2337 passed, 37 skipped in 107.43s Hygiene: only docs/ paths staged. fox's in-flight changes to arborist/qa/runner.py + arborist/substrate/prometheus.py + tests/test_prometheus.py + tests/test_prometheus_audit.py remain in their working tree, untouched by this commit. --- docs/tickets/ticket-000025-5f-battery.md | 41 ++++++++- ...igma-recursive-falsification-controller.md | 71 ++++++++++++--- docs/warrant-substrate-cookbook.md | 91 ++++++++++++++++++- 3 files changed, 183 insertions(+), 20 deletions(-) diff --git a/docs/tickets/ticket-000025-5f-battery.md b/docs/tickets/ticket-000025-5f-battery.md index c904401..efef499 100644 --- a/docs/tickets/ticket-000025-5f-battery.md +++ b/docs/tickets/ticket-000025-5f-battery.md @@ -578,10 +578,43 @@ against real workload), §10.14 (threshold handoff to #000012). ## 11. Status -**Open · awaiting go/no-go.** Larger surface than #000023 / #000024 -(five sub-batteries vs three / four remaining). Recommended -sequencing: ship Phase 1a in one push (seed fixtures + non-stub -runners), Phase 1b expands. +**As of 2026-05-10: in progress.** + +- **Phase 1a** (Function + Finetuning seed): landed. +- **Phase 1b.2** (Falsification curated fixtures): landed. +- **Phase 1c** (5F synthetic side expanded 10 → 30): landed + 2026-05-09 per #000025 §10.11 cross-reference. +- **Phase 1d** (synthetic side 30 → 50): landed 2026-05-09. +- **Phase 1e** (falsification motif coverage 50 → 62 covering + every documented failure-motif tag): landed 2026-05-10, closes + §10.12. +- **Phase 1f** (corpus-derived falsification harvest): landed + 2026-05-10 in commit `ff1752c`. Closes the controller → battery + loop fox designed in #000037 §3: Phase 1 controller emits + `FalsificationFixtureProposal` records from high-divergence + `providence_cache` rows (witness_divergence ≥ 0.5); + `bench/scripts/harvest_falsification_proposals.py` stratifies + top-20-by-cache_key per audit_mode and writes + `bench/fixtures/5f/falsification-harvested-v1.jsonl` + (41 + rows = 1 `_meta` header + 40 fixtures, stratified 20 HYBRID + + 20 UNGROUNDED). The 5F battery exercises them every test run; + `test_5f_falsification_harvested_pack_runs_clean` asserts + `error_detection_rate == 1.0` (every harvested row IS a + falsification by construction). Self-amplifying coverage — + the pack grows with the corpus + controller, not with + hand-curation. Every fixture row carries `_harvest_meta` with + source `cache_key`, `witness_divergence` at harvest time, + `audit_mode_at_harvest`, `harvest_threshold`, and + `source_ticket: "#000037 §13 step 11"` for full traceability. + Regenerate via `make bench-5f-harvest`. + +**Still open** (carried in file header): +- §10.11 — real shard finetuning chains +- §10.13 — Feedback Loop latency/efficiency calibrated to real workload +- §10.14 — threshold handoff to #000012 + +Larger surface than #000023 / #000024 (five sub-batteries vs three / four). The synthesis statement that justifies 5F's existence as a distinct axis: diff --git a/docs/tickets/ticket-000037-prometheus-sigma-recursive-falsification-controller.md b/docs/tickets/ticket-000037-prometheus-sigma-recursive-falsification-controller.md index 67f6c1b..d34b2ab 100644 --- a/docs/tickets/ticket-000037-prometheus-sigma-recursive-falsification-controller.md +++ b/docs/tickets/ticket-000037-prometheus-sigma-recursive-falsification-controller.md @@ -999,10 +999,15 @@ These are the right guardrails for the pure-function layer. ### 17.1 Warrant-promotion integration (#000031) -#000031 Phase 2 has 78 atomic claim-pack records that max out at -`ANCHOR-WARRANTED` until cited textbooks are ingested and -`derivations.proof_blob` rows are computed. The unconscious sweep -(§3.1, Target B) drains exactly this backlog. Controller scoring +#000031 closed at 92 +atomic claim-pack records (started at 78, reached 92 by 2026-05-09 +under Phase 2.5). All 92 records now resolve to at least one +textbook substrate via the 74 citation-aliases (#000041) + 13 +term-aliases (#000042) shipped under #000031 Phase 2.5 + B-1 + B-2. +The unconscious sweep (§3.1, Target B) drains the residual +ANCHOR-WARRANTED → EVIDENCE-WARRANTED promotion backlog — +`derivations.proof_blob` rows still need to be computed even on +records that have resolved warrant chains. Controller scoring includes `WarrantPromotionGain_i` (§4.1) so a branch that turns many anchor-only claims into evidence-warranted claims is high-value even if it doesn't move bench scores. @@ -1103,16 +1108,58 @@ the decision to the Phase 2 ticket. ## 20. Status -Phase 0 (this ticket): open · awaiting go/no-go · David review -2026-05-10 applied. +**As of 2026-05-10:** in progress. -When fox approves: this ticket flips to `in progress · Phase 0 -landed in commit ` with no code change — the doc itself is -the deliverable. +- **Phase 0** (this ticket): landed; David review (`response_ticket- + 000037-...txt`, 2026-05-10 06:37 Asia/Kuala_Lumpur) applied per §21. +- **Phase 1** (pure-function controller): landed in `f625cac` — + `arborist/substrate/prometheus.py` (893 LOC) implementing §13 + steps 1-9 + 12, §5 numerically-stable softmax fork-selection, §6 + eight-class hard-veto order, §7 Kelly-bounded allocation + four + safety guards, §7.1 EMA-smoothed difficulty update, §14 exception- + matrix dispatch, §15 three weight profiles. Test surface: + `tests/test_prometheus.py` (36 tests including the §16.2 + named-contract pins). +- **Phase 1.b** (gap-close pass): landed in `f9f5ae4` — §14 row 4 + Hermes-saturation guard (`utilization >= budget` → DEFERRED + + HERMES_SATURATED advisory event), §13 step 11 falsification-fixture + proposal emission, §15 entropy + memory gates moved to + weight-tunable `ControllerWeights` fields, veto-class cascade + hardening (ESCALATE > QUARANTINE > REJECT priority pins). +- **Phase 2** (advisory audit writes): landed in `a786d6d` — + `arborist/substrate/prometheus_audit.py` (200 LOC) + 54 LOC store + integration. `controller_events` sibling table with three event + kinds (`controller_decision`, `controller_difficulty`, + `controller_budget_allocation`), idempotent on + `(event_kind, body_hash)` UNIQUE constraint. **Does not enter + `audit_events.event_hash` preimage** — audit chain semantics + unaffected by advisory writes. Test surface: + `tests/test_prometheus_audit.py` (14 tests). +- **§12 Trigger 2 fired 2026-05-10**: divergence variance ratio + 0.575 > 0.5 with N=37 samples — Phase 1 opening is now backed + by empirical gate satisfaction, not speculative implementation. +- **Phase 3** (sleep-sweep scheduler): **deliberately not landed**. + Replaced with a read-only dry-run simulator + (`bench/scripts/prometheus_sigma_sweep_dryrun.py`) driving the + Phase 1 controller against real shard data; surfaces five design + findings (chunk-size dominates Kelly; capital_cost split needed; + τ_qa per audit_mode; Target B canonical-shape headline; quarantined- + row veto exercises end-to-end). See §22 for details. The dry-run + is the calibration substrate the eventual Phase 3 implementation + will reference. +- **Closed-loop signal**: 40 corpus-derived 5F fixtures harvested + from Phase 1 `FalsificationFixtureProposal` records via + `bench/scripts/harvest_falsification_proposals.py` → + `bench/fixtures/5f/falsification-harvested-v1.jsonl` (commit + `ff1752c`). Stratified 20 HYBRID + 20 UNGROUNDED, top-by-cache_key + for determinism. The 5F battery exercises them every test run + with `error_detection_rate == 1.0` (every harvested row IS a + falsification by construction). -Phase 1 opens when §12 trigger fires. Phase 2 (advisory audit -writes) and Phase 3 (unconscious sweep) gate sequentially on -Phase 1 having landed plus their own §12 trigger renewal. +Phase 3 (the actual scheduler) gates on a renewed §12 trigger +plus a Phase 3 ticket. Until then the closed loop runs at +test-suite cadence: Phase 1 emits proposals → harvester writes +fixtures → 5F battery exercises → coverage grows with the corpus. --- diff --git a/docs/warrant-substrate-cookbook.md b/docs/warrant-substrate-cookbook.md index a9a7fe3..189e6a9 100644 --- a/docs/warrant-substrate-cookbook.md +++ b/docs/warrant-substrate-cookbook.md @@ -432,6 +432,80 @@ arborist warrant-resolve --use-aliases --write # + 13-term-alias state (counts as of 2026-05-10) ``` +## Adjacent: live-corpus → bench-fixture harvest + +The textbook substrate above gives the cookbook a static surface +— 18 hand-curated, license-validated, deterministically ingested +sources. A parallel pattern landed 2026-05-10 in `ff1752c` under +#000037 → #000025 that does the opposite: **grow bench fixtures +from live corpus state, not hand-curation**. Worth documenting +here because the discipline patterns (attribution metadata, +determinism via stratification, fail-loud regen) carry across +both surfaces. + +### Pattern + +``` +#000037 Phase 1 controller (arborist/substrate/prometheus.py) + emits FalsificationFixtureProposal records when + witness_divergence (n_unverified / n_quotes) >= 0.5 + ↓ +bench/scripts/harvest_falsification_proposals.py reads qa.db, + filters live providence_cache rows, stratifies top-20 per + audit_mode (HYBRID + UNGROUNDED), writes JSONL + ↓ +bench/fixtures/5f/falsification-harvested-v1.jsonl + (41 rows = 1 _meta header + 40 fixtures) + ↓ +tests/test_bench_batteries.py:test_5f_falsification_harvested_pack_* + exercises every test run; error_detection_rate == 1.0 + by construction (every harvested row IS a falsification) +``` + +### Three discipline patterns reused from the textbook substrate + +1. **Attribution metadata on every derived artifact** — same + discipline as `derivations.proof_blob` carrying inclusion + proofs back to source chunks. Each harvested fixture row + carries `_harvest_meta` with `cache_key`, + `witness_divergence` at harvest time, `audit_mode_at_harvest`, + `harvested_from`, `harvest_threshold`, and `source_ticket: + "#000037 §13 step 11"`. Regeneration provenance auditable. + +2. **Determinism via sort-and-cap** — same discipline as + citation-alias cascade tuning's "top 5 AND-join then top 3 + OR-join" structure. The harvester sorts top-by-cache_key per + audit_mode bucket + caps at 20 per mode. Same `qa.db` state → + byte-identical fixture file across regenerations. + +3. **Pin the metadata contract in tests** — + `test_5f_falsification_harvested_pack_runs_clean` asserts + every row carries the meta. Same shape as the cookbook + appendix's discipline pins: a future regeneration that drops + attribution fires a test failure, not a silent regression. + +### Where to use this pattern + +Anywhere a controller emits Proposal records that should become +real fixtures. The recipe: + +- Define the Proposal dataclass on the controller side (see + `arborist/substrate/prometheus.py:FalsificationFixtureProposal`). +- Write a harvester script under `bench/scripts/` that reads + the source state (`qa.db`, `~/.arborist/shards/*.db`), + filters + stratifies, and emits a JSONL fixture pack. +- Pin the metadata contract in `tests/test_bench_batteries.py` + (or the equivalent battery test file). +- Wire `make -harvest` so regen is one command. + +The 18-textbook substrate is the static foundation; the +harvested-fixture pattern is how that foundation gets exercised +against current corpus state on every test run. Both live in +this cookbook because the discipline rules — attribution, +determinism, fail-loud — are identical. + +--- + ## References - `#000031` — surface-ingest cited textbooks @@ -441,6 +515,15 @@ arborist warrant-resolve --use-aliases --write proprietary candidates: Hilbert-Ackermann 1928, Mendelson, Enderton, Jech, Goldstein, Barendregt, Stanley, Brualdi, Knuth; the substrate substitution pattern is the workaround) +- `#000037` — Prometheus-Σ recursive falsification controller; + source of the `FalsificationFixtureProposal` records the + harvest pattern consumes +- `#000025` — 5F battery; sink for the harvested falsification + fixtures (§11 Phase 1f) +- `bench/scripts/harvest_falsification_proposals.py` — harvester + script (266 LOC) reading qa.db + emitting stratified fixtures +- `bench/fixtures/5f/falsification-harvested-v1.jsonl` — + current harvested pack (regenerate via `make bench-5f-harvest`) - `bench/results/full-warrant-resolution-2026-05-10.md` — the bench journal that closed 84% → 100% - `bench/fixtures/textbooks/manifest-v1.jsonl` — license-validated @@ -542,7 +625,7 @@ than waiting for bench-time STRICT-rate drift to surface it. - `tests/test_pi_star_protocol_and_registry.py` — 21 tests for `PiStar` Protocol contract + registry mutation discipline (no public unregister; `name@version` content-pinned). -- `tests/test_prometheus.py` — 36 tests for the Prometheus-Σ +- `tests/test_prometheus.py` — 42 tests for the Prometheus-Σ recursive-falsification controller (#000037 Phase 1, commits `f625cac` + `f9f5ae4`). Verifier-style discipline (NOT calculator pattern — it's a state machine, not closed-form @@ -551,7 +634,7 @@ than waiting for bench-time STRICT-rate drift to surface it. Hermes-saturation guard, §13 step 11 falsification-fixture proposal, §15 weight-tunable entropy + memory gates, ESCALATE > QUARANTINE > REJECT priority ordering. -- `tests/test_prometheus_audit.py` — 14 tests for the +- `tests/test_prometheus_audit.py` — 17 tests for the `controller_events` sibling table (#000037 Phase 2, commit `a786d6d`). Pins migration idempotency, no-chain-mutation invariant (advisory writes never enter `audit_events.event_hash` @@ -594,8 +677,8 @@ than waiting for bench-time STRICT-rate drift to surface it. | qa/prompts.py | 153 | 207 (20 tests) | 1.35 | | concepts/extract.py | 288 | 278 (20 tests) | 0.97 | | textbooks_manifest.py | 236 | 463 (43 tests) | 1.96 | -| substrate/prometheus.py | 893 | 804 (36 tests) | 0.90 | -| substrate/prometheus_audit.py | 200 | 388 (14 tests) | 1.94 | +| substrate/prometheus.py | 893 | 955 (42 tests) | 1.07 | +| substrate/prometheus_audit.py | 200 | 477 (17 tests) | 2.39 | Pattern: small contract-defining modules (weights, prompts, warrant_chain) get 1.4-3.6× test LOC because they're foundation;