diff --git a/docs/tickets/ticket-000037-prometheus-sigma-recursive-falsification-controller.md b/docs/tickets/ticket-000037-prometheus-sigma-recursive-falsification-controller.md index 9522c93..3fa9fce 100644 --- a/docs/tickets/ticket-000037-prometheus-sigma-recursive-falsification-controller.md +++ b/docs/tickets/ticket-000037-prometheus-sigma-recursive-falsification-controller.md @@ -1,19 +1,19 @@ # Ticket #000037 — Prometheus-Σ recursive falsification controller (bicameral substrate) -**Status:** open · awaiting go/no-go (doc-only Phase 0) +**Status:** open · awaiting go/no-go (doc-only Phase 0; David review 2026-05-10 applied — see §21) **Opened:** 2026-05-09 **Scope:** Spec a named control law that unifies the recursive- falsification machinery already shipped across #000012 (ForkScore), #000014 (SelfModel), #000017 (MemoryRoot), #000018 (soft-hash threat model), #000020 (capital ledger), #000025 (5F battery), #000027 (canonical-projection persistence), and #000028 (multi- -modality witness) into a Shannon-entropy + Nash-veto + Kelly- -allocated difficulty controller. Frame the deterministic kernel -and the probabilistic LLM as a **bicameral pair** that probes each -other — left brain / right brain, both witness-only authority — -with an unconscious nightly falsification sweep during ingestion -lulls. Phase 0 is doc-only; implementation phases gate on a -measured-pressure trigger, not a calendar date. +witness canonical agreement) into a Shannon-entropy + Nash-veto + +Kelly-allocated difficulty controller. Frame the deterministic +kernel and the probabilistic LLM as a **bicameral pair** that +probes each other — left brain / right brain, both witness-only +authority — with an unconscious nightly falsification sweep +during ingestion lulls. Phase 0 is doc-only; implementation +phases gate on a measured-pressure trigger, not a calendar date. **Audience:** fox + future blackops shifts + downstream Merkle-AGI v9.x authors + anyone proposing a unified mutation-selection layer. **Hard constraint:** doc-only in Phase 0 — **no** schema bump, @@ -25,7 +25,9 @@ existing rule from #000028: kernel is ground truth on canonical shapes; LLM is witness, never authority. The controller does not defeat Gödel — it side-steps the engineering trap by replacing global completeness with **local, versioned, replayable -admissibility**. +admissibility**. Controller may **propose** updates to MemoryRoot +/ SelfModel; existing write paths commit them only after their +own validation. Controller never mutates either directly. --- @@ -115,8 +117,8 @@ Three load-bearing observations from the thread: 2. **Unconscious falsification.** A nightly / ingestion-lull sweep that walks committed nodes and re-runs witness comparison. Distinct from inference-time witness fan-out (#000028) — runs - on **already-cached** answers, and writes new 5F fixtures only - when previously-strict rows now diverge. + on **already-committed** state, and writes new 5F fixtures + when re-checked rows disagree with their original verdict. 3. **Compute is the bottleneck.** Hermes serves ~4 concurrent requests. This is the binding constraint that forces the controller to be Kelly-bounded (allocate compute to high- @@ -132,9 +134,11 @@ yet — fox's "slow me down in the short term" gates Phase 1. ```text ┌──────────────┐ - │ canonical │ deterministic kernel - │ π* witness │ (left brain — exact, slow, - │ (kernel) │ unhallucinable) + │ canonical │ π* kernel + │ π* witness │ exact, deterministic, + │ (kernel) │ cheap inside supported + │ │ domains, narrow / domain- + │ │ bounded └──────┬───────┘ │ │ agreement matrix @@ -142,8 +146,9 @@ yet — fox's "slow me down in the short term" gates Phase 1. │ ┌──────┴───────┐ │ LLM witness │ probabilistic LLM - │ (Hermes) │ (right brain — fluent, - │ │ fast, hallucination-prone) + │ (Hermes) │ fluent, broad, hallucination- + │ │ prone, expensive / scarce + │ │ relative to kernels └──────┬───────┘ │ v @@ -153,14 +158,24 @@ yet — fox's "slow me down in the short term" gates Phase 1. └──────────────┘ ``` +Note on the speed/cost asymmetry: the older draft of this diagram +called the kernel "slow" and the LLM "fast." That inverted the +real cost surface. **Kernels are cheap inside their supported +domains** — pure-Python arithmetic, propositional logic, and +combinatorics calls return in microseconds with no concurrency +limit. **The LLM is the scarce resource** — Hermes serves ~4 +concurrent requests at a single endpoint. The whole Kelly- +allocation argument (§7) depends on Hermes being the binding +constraint, not the kernel. Anywhere else in this doc that +mentions "fast / slow" is downstream of this correction. + The bicameral pair is **already implemented** by #000028 — the ticket's contribution is to (a) name it, (b) commit to "kernel is ground truth on canonical shapes, LLM is witness never authority" as a doctrine (already enforced by the verifier; this writes it down), and (c) add the **unconscious sweep** as a Phase 1 -deliverable that walks committed `providence_cache` rows during -ingest lulls and re-runs the witness fan-out on rows older than a -configurable threshold. +deliverable that walks committed state during ingest lulls and +re-runs the witness fan-out on stale rows. The unconscious sweep is the new mechanical contribution. Today #000028 fan-out runs at inference time only. The sweep amortizes @@ -184,15 +199,29 @@ The unconscious sweep is where that probe runs. Two distinct sweep targets, both processed during ingest lulls: **Target A — `providence_cache` rows older than threshold τ_qa.** -Re-runs the multi-modality witness fan-out (#000028) on cached Q&A -answers. Catches drift: a Hermes update changes the LLM's -divergence pattern; a kernel bump changes canonical bytes; a peer's -mesh sync delivered new evidence that would have changed the -answer. Threshold τ_qa default: 7 days. +Re-runs the multi-witness fan-out (#000028) on cached Q&A answers. +The pool to sweep is **not just previously-STRICT rows** — it +includes any row whose verdict could change under re-check or whose +warrant could be promoted: + +- `CANONICAL_PROJECTION` rows (kernel could have bumped versions) +- `STRICT-WITNESSED` rows (LLM could now diverge; cache could have + drifted) +- `EVIDENCE-WARRANTED` rows whose claims have canonical-shaped + content (would benefit from canonical re-probe) +- `MARGINAL` rows queued for re-evaluation +- `ANCHOR-WARRANTED` claim-pack records awaiting warrant promotion + (target of #000031's surface-ingest substrate) + +A row does not need to have been STRICT to deserve re-witnessing. +Some of the highest-value sleep work is upgrading marginal / +anchor-only records or detecting document-level canonical +divergence. Threshold τ_qa default: 7 days. **Target B — `documents` rows ingested without meta-cognition.** -Walks `documents` where `last_swept_at` is NULL or older than τ_doc, -runs whichever cross-checks apply to the document's content: +Walks `documents` whose state indicates "never swept" or +"swept-but-stale," runs whichever cross-checks apply to the +document's content: - **Canonical-projection probe.** If chunks contain canonical-shape language (math expressions, propositional logic, time-series JSON @@ -210,25 +239,97 @@ runs whichever cross-checks apply to the document's content: upstream changed get marked stale; docs whose upstream vanished get marked gone. Provides drift signal without paying full re-fetch cost. -- **Witness fan-out on document content.** For documents whose - body contains canonical-shape statements (e.g., a Wikipedia - paragraph asserting "0.1 + 0.2 = 0.3"), run the kernel + LLM - + cache witness pair AS IF the document were a Q&A answer - citing itself. Produces high-value falsification fixtures - for documents the corpus *wrote down* but never *probed*. +- **Witness fan-out on candidate-statement chunks only.** For + documents whose body contains canonical-shape statements (e.g., + a Wikipedia paragraph asserting "0.1 + 0.2 = 0.3"), run the + kernel + LLM + cache witness pair AS IF the document were a + Q&A answer citing itself. **Do not run LLM witness over whole + documents** — only over detected canonical-shape candidate + statements. Cheap prefilters (math operator regex, logic + token regex, truth-table pattern, JSON array detector, + source_reference presence, citation marker presence, high + dependency fanout, recently-ingested textbook/claim-pack + source) gate the LLM call. -**State tracking.** A new column on `documents`: -`last_swept_at INTEGER`. Updated when the sweep runs cross-checks -on a doc. Schema migration is small (one ALTER TABLE), but lives -behind the §12 phase trigger so it doesn't bump the cache_key -invariant prematurely. +**Sweep state tracking — Phase 1 design choice, not Phase 0 +schema.** Per-document sweep status needs persistence so re-checks +are resumable. Phase 0 explicitly does **not** decide the schema. +Phase 1 picks one of two options: -**Why this matters.** Without target B, the substrate's "I have -this content in my Merkle tree" claim is honest at the byte -level but silent on whether the content was ever tested against -the kernel ground truth. Sleep is when you do the tests you -deferred to keep the day moving. The bicameral pair's -unconscious half is exactly where this catch-up happens. +```text +Option A (column-on-documents): + ALTER TABLE documents ADD COLUMN last_swept_at INTEGER + +Option B (recommended — sidecar): + CREATE TABLE document_sweep_state ( + document_root BLOB NOT NULL, + sweep_policy_hash BLOB NOT NULL, + last_swept_at INTEGER, + last_sweep_status TEXT NOT NULL, + last_sweep_event_root BLOB, + canonical_probe_count INTEGER NOT NULL DEFAULT 0, + divergence_count INTEGER NOT NULL DEFAULT 0, + warrant_promotion_count INTEGER NOT NULL DEFAULT 0, + freshness_status TEXT, + error_message TEXT, + PRIMARY KEY (document_root, sweep_policy_hash) + ); +``` + +Why sidecar (Option B) is cleaner: + +```text +multiple sweep policies can coexist (sweep_policy_hash is keyed) +no mutation to core document identity +no premature migration of the documents table +clean re-sweep under changed controller policy +clearer audit separation between ingest and sleep +``` + +The bicameral pair's unconscious half is exactly where this +catch-up happens. Without target B, the substrate's "I have this +content in my Merkle tree" claim is honest at the byte level but +silent on whether the content was ever tested against the kernel +ground truth. Sleep is when you do the tests you deferred to +keep the day moving. + +**Sweep audit-event tags** (when Phase 2+ writes them): + +```text +document_canonical_probe +document_warrant_resolution +document_freshness_probe +document_witness_fanout +document_sleep_sweep +``` + +All of these are advisory event tags on `audit_events`, not new +preimage fields. They do not bump `event_hash`. + +### 3.2 Lineage — earlier metacognition layers + +Prometheus-Σ is not a new philosophy; it's the next layer of an +existing series. + +| Layer | Ticket | Role | +|---|---|---| +| Pre-answer difficulty control | #000008 | broad-quantifier preflight guard | +| Pre-answer epistemic boundary control | #000010 | M0 / MCTL (false premises, contradictions, time sensitivity, reference-frame ambiguity) | +| Post-answer / post-ingest recursive falsification | #000037 (this ticket) | controller layer over witness, ledger, memory | + +#000008 already framed broad-quantifier handling as a difficulty +/ homeostasis controller: query shape + model profile + answer +mode + benchmark constraint determine allowable claim budget and +output contract. #000010 then defined M0 / MCTL as a runtime +epistemic control layer that classifies the question before +generation. #000037 is the third member of the same family — the +post-answer recursive layer that decides whether to re-probe what +the system already committed to. + +This continuity matters because the design discipline carries +forward: **fail visibly, locally, reproducibly, and non-reusably** +(per #000010). Prometheus-Σ does not invent that discipline; it +inherits it. --- @@ -237,27 +338,38 @@ unconscious half is exactly where this catch-up happens. ### 4.1 Inputs ```text -battery deltas: 5S, 5T, 5F, 5R (per candidate branch) +battery deltas: Δ5S, Δ5T, Δ5F, Δ5R (per candidate branch) witness stats: agreement_rate, divergence_rate, kernel_error capital ledger: compute_cost, latency_cost, storage_cost, witness_cost memory state: stale_claims, invalidation_frontier selfmodel state: calibration_delta -risk flags: soft_hash_signal, unsupported_carrier, - source_warrant_level +risk inputs: soft_hash_signal, unsupported_carrier, + source_warrant_level, security_risk +gain inputs: selfmodel_calibration_gain, + warrant_promotion_gain budget: B (concurrent-Hermes ceiling ≈ 4) +hermes_utilization: current concurrent in-flight LLM calls ``` +`security_risk` is split out from the generic regression-penalty +bucket because the soft-hash analysis (#000018) treats it as a +hard veto class with a different threshold, not a smooth penalty. +`warrant_promotion_gain` is a first-class input because Target B +sweep work (anchor → evidence-warranted) is high-value even when +it doesn't move bench scores. + ### 4.2 State ```text Σ_t = { Ω_t, # organism state (#000017 + #000014 roots) - H_t, # branch-distribution entropy + H_t, # branch-distribution entropy (normalized) ForkScore_t, # latest verdict map (#000012) divergence_rate_t, # rolling window from #000028 sweep capital_budget_t, # remaining compute under B - difficulty_t # named knob this ticket introduces + difficulty_t, # named knob this ticket introduces + difficulty_ema_t # smoothed knob; avoids oscillation } ``` @@ -267,43 +379,91 @@ budget: B (concurrent-Hermes ceiling ≈ 4) ACCEPT — branch passes all hard vetoes, score positive, entropy bounded MARGINAL — useful but uncertain; queue for unconscious sweep +DEFERRED — branch worth evaluating but compute budget or + Hermes utilization blocks it now REJECT — negative score OR hard-regression flag -QUARANTINE — integrity issue (carrier, soft-hash, schema) +QUARANTINE — integrity issue (carrier, soft-hash, schema, drift) UNKNOWN — bounded verifier returned no decision ESCALATE — strictly stronger verifier or human required ``` -`UNKNOWN` and `ESCALATE` are the labels that side-step Gödel. See §10. +`MARGINAL` vs `DEFERRED` — the distinction matters for sleep-time +scheduling: + +```text +MARGINAL: branch was evaluated; result is useful but uncertain. +DEFERRED: branch has not been evaluated because budget / priority + prevented it. +``` + +`UNKNOWN` and `ESCALATE` are visible outputs of the Gödel +discipline; they are **not** the side-step itself. The actual +side-step is the entire policy of typed/verifier-bounded domains, +versioned axiom sets, local admissibility labels, replayable +audit chains, and refusal of global completeness. See §10. + +### 4.4 Update authority + +The controller never mutates MemoryRoot or SelfModel directly. +It **emits update proposals**; existing write paths +(`arborist.store.append_audit`, the SelfModel falsification path +in #000014, the MemoryRoot belief-update path in #000017) commit +them only after their own warrant validation. Treating the +controller as a privileged mutator would skip those gates and +turn it into an unsafe shortcut for hard-hash chain mutation. --- ## 5. Shannon-entropy fork-selection -For each candidate branch `b_i ∈ B`: +For each candidate branch `b_i ∈ B`, utility: ```text -U_i = α·5S_i + β·5T_i + γ·5F_i + ρ·5R_i - − λ·C_i − μ·X_i − ν·D_i +U_i = + α · Δ5S_i + + β · Δ5T_i + + γ · Δ5F_i + + ρ · Δ5R_i + + σ · SelfModelCalibrationGain_i + + τ · WarrantPromotionGain_i + − λ · CapitalCost_i + − μ · RegressionPenalty_i + − ν · WitnessDivergence_i + − ξ · SecurityRisk_i + − ω · MemoryInvalidationRisk_i ``` -where `C_i` is capital cost, `X_i` is regression / security -penalty, `D_i` is witness divergence. Branch distribution: +Numerically stable softmax (required — naive `exp(η · U_i)` +overflows on large `U`): ```text -p_i = exp(η · U_i) / Σ_j exp(η · U_j) -H(p) = − Σ_i p_i · log p_i +z_i = η · U_i +p_i = exp(z_i − max(z)) / Σ_j exp(z_j − max(z)) ``` -The difficulty knob `difficulty_t` directly modulates `η`, search -depth, witness sampling rate, and branch count: +Entropy and **normalized** entropy (better for cross-branch-count +comparison): ```text -high H → widen search, raise witness sampling, lower acceptance - confidence, allocate more compute -low H → narrow search, reduce witness sampling, conserve - compute -↑ divergence → raise difficulty, require more verification -↓ divergence → drop difficulty, allow fast path +H(p) = − Σ_i p_i · log p_i +H_norm(p) = H(p) / log(n) where n = |branches| +``` + +Use `H_norm` for gating: + +```text +H_norm > H_high → widen search, raise witness sampling, + lower acceptance confidence, allocate more + compute +H_norm < H_low → narrow search, reduce witness sampling, + fast path +``` + +If no branch survives hard vetoes (§6), return: + +```text +UNKNOWN | REJECT (depending on veto class) +reason = ALL_BRANCHES_VETOED ``` This is the homeostatic loop: divergence-rate is the thermostat; @@ -326,35 +486,104 @@ benchmark: wants performance ``` A branch is acceptable iff **no critical validator vetoes** and -the equilibrium is stable under the scoring rule. Hard vetoes: +the equilibrium is stable under the scoring rule. + +Hard-veto order — apply **before** probabilistic scoring: ```text -unsupported carrier → QUARANTINE -verifier failure → REJECT -hard regression → REJECT -cache drift detected → QUARANTINE -soft-hash signal above threshold → QUARANTINE -memory invalidation > κ → ESCALATE +1. unsupported carrier → QUARANTINE +2. verifier failure → REJECT +3. hard regression → REJECT +4. cache drift detected → QUARANTINE +5. soft-hash signal above threshold → QUARANTINE +6. memory invalidation > κ → ESCALATE +7. schema or canonicalization mismatch → QUARANTINE +8. source-warrant downgrade below required floor → REJECT ``` +**Soft-hash veto integration (#000018).** The closed soft-hash +threat-model analysis bounds chosen-input and chosen-state +attacks under SHA-256 assumptions, with a per-checkpoint nonce +mitigation; replay-window T3 still open as #000036. Translated +into controller rules: + +```text +If soft_hash_signal > threshold: + QUARANTINE + +If replay_window_unbounded (T3 open or per-checkpoint nonce missing): + ESCALATE (or require stricter witness policy) + +Otherwise: + fold soft_hash signal into SecurityRisk_i for smooth penalty. +``` + +This turns the soft-hash analysis from a doc into an actual +Nash-veto component. + This is the formal version of "balance computation with tractability": no objective wins by destroying the others. --- -## 7. Kelly-bounded compute allocation +## 7. Kelly-bounded compute allocation with safety guards -With `p_i` from §5, payoff `b_i`, and `q_i = 1 − p_i`: +Naive Kelly: ```text -f_i = max(0, (p_i · b_i − q_i) / b_i) -allocation_i ∝ f_i · exp(η · U_i) -Σ_i allocation_i ≤ B +f_i = max(0, (p_i · b_i − q_i) / b_i) where q_i = 1 − p_i ``` -`B` is the binding Hermes-concurrency constraint (~4 today). High- -upside, high-confidence branches get most compute; low-payoff or -noisy branches get little or none; vetoed branches get zero. +This needs explicit safety guards or it will divide-by-zero, +allocate to vetoed branches, or saturate when payoffs go negative. +Required guards: + +```text +if vetoed_i: + f_i = 0 +elif b_i <= 0: + f_i = 0 +else: + f_i = max(0, (p_i · b_i − q_i) / b_i) +``` + +Then: + +```text +raw_allocation_i = f_i · exp(η · U_i) +allocation_i = B · raw_allocation_i / Σ_j raw_allocation_j + +if Σ_j raw_allocation_j == 0: + allocation_i = 0 for all i + return DEFERRED (or UNKNOWN if all vetoed — see §5) +``` + +`B` is the binding Hermes-concurrency constraint (~4 today). +High-upside, high-confidence branches get most compute; low- +payoff, noisy, or vetoed branches get zero. `B = 0` (Hermes +unavailable / saturated) returns `DEFERRED` for everything — +sleep-time scheduling can pick the work back up. + +### 7.1 Difficulty update law + +```text +difficulty_raw = + clamp( + difficulty_t + + a · H_norm + + b · divergence_rate + + c · hard_regression_rate + − d · witness_agreement_rate + − e · capital_pressure, + min_difficulty, + max_difficulty + ) + +difficulty_ema_{t+1} = (1 − r) · difficulty_ema_t + r · difficulty_raw +``` + +The EMA smoothing avoids oscillation under bursty input. `r` +defaults to `0.2` (Phase 1 weight profile §15). --- @@ -363,11 +592,13 @@ noisy branches get little or none; vetoed branches get zero. Build `A[i, j]` over (mutation × chain) with chains: ```text -j=1: 5S delta j=6: memory consistency -j=2: 5T delta j=7: selfmodel calibration -j=3: 5F delta j=8: inverse capital cost -j=4: 5R delta j=9: inverse security risk -j=5: witness agreement j=10: inverse regression penalty +j=1: Δ5S j=7: selfmodel calibration gain +j=2: Δ5T j=8: warrant promotion gain +j=3: Δ5F j=9: inverse capital cost +j=4: Δ5R j=10: inverse regression penalty +j=5: witness agreement j=11: inverse witness divergence +j=6: memory consistency j=12: inverse security risk + j=13: inverse memory invalidation risk ``` Solve: @@ -375,8 +606,8 @@ Solve: ```text choose i maximizing A_i · w subject to: - no hard veto - Σ capital cost_i ≤ B + no hard veto (§6 list applied first) + Σ_i capital cost_i ≤ B regression_i ≥ −ε witness divergence_i ≤ δ memory invalidation_i ≤ κ @@ -404,15 +635,51 @@ over a committed cognitive state-space** is the engineering name. LLMs propose candidate branches not enumerated in current state. The controller does not trust them until they pass canonicalization, verification, witness comparison, and benchmark scoring. Every -proposed branch is a witness, not an authority — same rule as -§3. +proposed branch is a witness, not an authority — same rule as §3. --- ## 10. Gödel discipline — local, versioned, replayable admissibility -The controller does **not** defeat Gödel. It side-steps the -engineering trap by refusing single-system closure. Concretely: +The controller does **not** defeat Gödel. + +A single sufficiently expressive consistent formal system cannot be +both complete and prove all its own truths from within itself. +Arborist avoids relying on one closed system. It uses typed +kernels, versioned axiom bundles, source warrants, empirical +evidence, witness channels, local labels, and replayable audit +chains. + +The controller must never infer: + +```text +"not proved false" → true (open-world is not closed-world) +"LLM agrees" → true (LLM is witness, not authority) +"kernel arithmetic" → empirical claim (warrant types do not cross) +``` + +The admissibility labels are the side-step: + +```text +CANONICAL_PROJECTION +EVIDENCE-WARRANTED +ANCHOR-WARRANTED +UNKNOWN +UNSUPPORTED +UNDECIDED_UNDER_CURRENT_AXIOMS +ESCALATE +QUARANTINE +DEFERRED +``` + +The system is powerful **because it can stop**. UNKNOWN and +ESCALATE are visible outputs of that discipline; they are not the +discipline itself. The discipline is the whole policy: typed +domains, versioned axioms, local labels, replayable audit, refusal +of global completeness. + +This continues #000010's principle: not "never be trapped," but +fail visibly, locally, reproducibly, and non-reusably. | Technique | Gödel response | |---|---| @@ -422,12 +689,6 @@ engineering trap by refusing single-system closure. Concretely: | separation of empirical / formal / source warrants | a kernel result does not warrant an empirical claim and vice versa — no false closure across types | | canonical bytes + audit chain + replayable fixtures | every admissibility decision is reproducible; consistency is local-and-versioned, not global | -This is what fox's thread named: **local, versioned, replayable -admissibility** instead of global completeness. The doctrine is -already implicit across #000005 (label ladder), #000027 (canonical -projections), and the verifier's trichotomy. This ticket commits -it to writing as the controller's design substrate. - --- ## 11. Bottleneck: Hermes ~4 concurrent requests @@ -439,8 +700,10 @@ binding `B` in §7. Implications: - Witness fan-out (#000028) can not run uncontrolled; the controller throttles it via Kelly allocation. - Unconscious sweep (§3) **must not** compete with inference-time - traffic — runs only when concurrent-Hermes utilization is low, - yields to inference traffic on contention. + traffic. Phase 1 sweep yields cooperatively: if live Hermes + utilization ≥ `B − 1`, sleep work yields immediately. Strict + priority queues are a Phase 2 escalation if cooperative + yielding underperforms. - Adding deterministic kernels (the "left brain") buys extra effective concurrency at zero LLM cost: kernel calls are pure- Python and parallelize freely. Encourages expanding π* domain @@ -455,20 +718,45 @@ controller would crowd out fox's actual queries. ## 12. Phase gating — measured trigger, not calendar date Phase 0 (this ticket) is doc-only. Phase 1 opens **only** when one -of these triggers fires: +of these triggers fires. Each trigger has explicit edge-case +handling so it doesn't false-fire on small samples or undefined +quantities. -1. ForkScore (#000012 Phase 1a) regularly receives ≥ 4 candidate - branches per checkpoint and the operator wants probability- - weighted compute allocation across them. -2. #000028 witness sweep observes divergence-rate variance large - enough that fixed witness sampling starts wasting budget on - stable rows or under-sampling unstable rows. Concrete signal: - rolling divergence-rate stddev / mean > 0.5 over a 7-day - window. -3. Capital ledger (#000020) shows witness-cost > 30 % of total - compute spend on a real-shard workload (#000026 baseline). -4. Operator-stated mission need for the unconscious sweep — e.g. - nightly re-validation of an ingested textbook (#000031). +**Trigger 1 — branch density.** ForkScore (#000012 Phase 1a) +regularly receives ≥ 4 candidate branches per checkpoint AND the +operator wants probability-weighted compute allocation across +them. + +**Trigger 2 — divergence variance.** #000028 witness sweep +observes divergence-rate variance large enough that fixed +witness sampling is wasting budget on stable rows or under- +sampling unstable rows. Concrete signal: + +```text +sample_count >= 30 (N_min) +AND ( + stddev(divergence) / max(mean(divergence), ε) > 0.5 + OR absolute stddev > 0.10 +) +``` + +The `max(mean, ε)` guard keeps the trigger from dividing by +zero when divergence is uniformly low; the absolute-stddev +fallback catches the case where mean is near zero but variance +is still large enough to matter. `N_min = 30` events over the +window keeps tiny samples from triggering on noise. + +**Trigger 3 — witness cost share.** Capital ledger (#000020) +shows `witness_cost / total_compute > 0.30` on a real-shard +workload (#000026 baseline). If the denominator is zero, the +trigger does not fire (do not compute the percentage). +Requires #000026 baseline to exist; if missing, run that +ticket first. + +**Trigger 4 — operator mission need.** Operator-stated need for +the unconscious sweep (e.g., nightly re-validation of an +ingested textbook per #000031). Bypasses the measured-pressure +gates because it is an explicit go. Until then, the controller exists on paper. ForkScore, 5F, and witness sweep continue to operate as independent modules with no @@ -486,115 +774,445 @@ Algorithm: Prometheus-Σ Recursive Falsification Controller Inputs: Ω_t, B_t (candidate branches), W_t (witnesses), M_t (memory + selfmodel), C_t (capital ledger), - P_t (policy constraints) + P_t (policy constraints), B (compute budget), + hermes_utilization -Step 1: for each b_i, run available deterministic checks -Step 2: build A[i, j] cross-chain matrix -Step 3: apply hard vetoes (carrier, verifier, regression, cache - drift, security, memory) -Step 4: compute U_i -Step 5: p_i = softmax(η · U_i) -Step 6: H(p) -Step 7: adjust difficulty_t: - ↑ H → wider search - ↑ divergence → stricter witness - ↑ cost → narrower search -Step 8: Kelly-bounded allocation under B -Step 9: pick branch: - ACCEPT if positive, bounded risk, low entropy - MARGINAL if useful but uncertain (queue for sweep) - REJECT if negative or vetoed +Step 1: for each b_i, run available deterministic checks +Step 2: build A[i, j] cross-chain matrix +Step 3: apply hard vetoes in §6 order; emit veto reasons + if all vetoed → return UNKNOWN | REJECT, + reason=ALL_BRANCHES_VETOED +Step 4: compute U_i (extended utility, §5) +Step 5: stable softmax: p_i = exp(z_i − max z) / Σ exp(z_j − max z) +Step 6: H_norm(p) = H(p) / log(n) +Step 7: update difficulty_ema (§7.1) — smoothed +Step 8: Kelly-bounded allocation under B with safety guards (§7). + If sum-of-raw-allocations == 0 → return DEFERRED +Step 9: pick branch / label: + ACCEPT if positive, bounded risk, low entropy + MARGINAL if useful but uncertain (queue for sleep) + DEFERRED if budget blocks evaluation + REJECT if negative or vetoed QUARANTINE if integrity issue -Step 10: commit decision to audit chain (#000017 + #000014 roots) + UNKNOWN | ESCALATE per §10 +Step 10: emit ADVISORY audit-event row (controller_decision / + controller_difficulty / controller_budget_allocation). + Sibling tag, no event_hash preimage change. Step 11: emit 5F falsification fixtures from divergences -Step 12: update MemoryRoot / SelfModel if warranted +Step 12: emit MemoryRoot / SelfModel update PROPOSALS (not + direct mutations — existing write paths commit them + only after their own validation). See §4.4. -Output: Ω_{t+1}, decision, audit event, new fixtures, updated - difficulty_t +Output: Ω_{t+1}, decision, advisory audit event, new fixtures, + updated difficulty_ema. ``` -Phase 1 implementation lives in `arborist/v9/prometheus.py` (new -namespace; sibling to `arborist/v8/fork_score.py`). No schema +Phase 1 implementation lives in `arborist/substrate/prometheus.py` +(sibling to the existing `arborist/substrate/fork_score.py`, +post-`arborist/v7+v8 → arborist/substrate` consolidation). No schema changes — the controller is a pure function over already-committed -state. +state. Phase 2+ adds optional advisory-audit-event writes; Phase 3 +adds the unconscious sweep. Each phase gates on §12 triggers +plus the prior phase having landed. --- -## 14. Scope boundaries — what this ticket does NOT do +## 14. Exception-handling matrix -- **Does not** bump `governance_policy_hash` or any cache_key dim. - The controller reads existing state and emits decisions; its - outputs land as audit-event rows, not as cache_key inputs. +Constraint-optimization, not just mathematical elegance. + +| Failure / edge case | Correct behavior | Output | +|---|---|---| +| No candidate branches | No compute allocation; record no-op decision | `UNKNOWN` | +| All branches hard-vetoed | Emit veto reasons; no allocation | `REJECT` or `QUARANTINE` | +| Budget `B = 0` | Do not run LLM; queue sleep if useful | `DEFERRED` | +| Hermes unavailable | Kernel-only checks; mark LLM witness unavailable | `DEFERRED` / `MARGINAL` | +| Kernel error on canonical shape | Quarantine canonical path; do not trust LLM fallback | `QUARANTINE` | +| Cache drift | Quarantine row; schedule chain-check | `QUARANTINE` | +| Missing MemoryRoot | Run stateless scoring; no memory update proposal | `MARGINAL` | +| Missing SelfModel | Do not compute calibration gain | `MARGINAL` | +| Capital ledger unavailable | Conservative mode; cap allocations | `DEFERRED` | +| Soft-hash signal > threshold | Do not score normally | `QUARANTINE` | +| Replay-window unbounded (#000036 still open) | Stricter witness policy required | `ESCALATE` | +| Unsupported carrier | Fail explicitly; no silent fallback | `QUARANTINE` | +| LLM canonicalization ambiguous | Record ambiguity; do not count as agreement | `UNKNOWN` / falsification candidate | +| Entropy NaN / overflow | Use stable softmax; if still invalid, reject scoring | `UNKNOWN` | +| Weight config invalid | Fall back to safe defaults; record config warning | `MARGINAL` | +| Sleep sweep collides with inference traffic | Yield immediately | `DEFERRED` | +| Mean divergence = 0 (Trigger 2) | Use absolute stddev path; do not divide | trigger does not fire | +| Sample count < N_min (Trigger 2) | Do not trigger on variance | trigger does not fire | +| Witness-cost denominator = 0 (Trigger 3) | Skip percentage computation | trigger does not fire | + +--- + +## 15. Weight default profiles + +Three named profiles. Do not bake mission-language priorities into +code as scattered constants — register them as named profiles and +switch via config field. The profile name folds into the +governance-policy hash if Phase 2 enables policy-hash binding; +Phase 1 keeps profiles advisory. + +### 15.1 Safe default (recommended Phase 1 starting point) + +```json +{ + "alpha_5s": 1.0, + "beta_5t": 1.0, + "gamma_5f": 1.25, + "rho_5r": 1.0, + + "sigma_selfmodel_calibration": 0.75, + "tau_warrant_promotion": 0.75, + + "lambda_capital_cost": 1.0, + "mu_regression_penalty": 2.0, + "nu_witness_divergence": 1.5, + "xi_security_risk": 3.0, + "omega_memory_invalidation": 2.0, + + "eta_softmax_temperature": 1.0, + "ema_smoothing_r": 0.2 +} +``` + +Rationale: + +```text +5F slightly upweighted: + falsification and feedback are the recursive core. +security and hard regression heavily penalized: + avoid selecting dangerous or destabilizing branches. +witness divergence penalized but not infinitely: + divergence is also useful falsification data. +capital cost penalized but not enough to suppress high-value + proof work. +``` + +### 15.2 Conservative profile (real-shard production) + +```json +{ + "gamma_5f": 1.0, + "lambda_capital_cost": 1.5, + "xi_security_risk": 4.0, + "eta_softmax_temperature": 0.75 +} +``` + +Lower temperature concentrates allocation on top-confidence +branches; higher security weight further suppresses risky picks. + +### 15.3 Exploratory profile (research sweeps) + +```json +{ + "gamma_5f": 1.5, + "nu_witness_divergence": 0.75, + "lambda_capital_cost": 0.5, + "eta_softmax_temperature": 1.5 +} +``` + +Higher temperature widens the distribution; lower divergence +penalty admits speculative branches; lower cost penalty allows +more compute spend on exploration. + +--- + +## 16. Phase 1 implementation prerequisites + +### 16.1 Pure-function dataclass API + +Phase 1 must be implementable without a database, without a +Hermes endpoint, and without a background scheduler — the +controller is an advisory pure function over already-committed +state. Phase 2 adds optional audit-event writes; Phase 3 adds +the sweep scheduler. + +```python +@dataclass(frozen=True) +class ControllerBranch: + branch_id: str + deltas: BatteryDeltas + witness_divergence: float + capital_cost: float + regression_penalty: float + security_risk: float + memory_invalidation: float + selfmodel_calibration_gain: float = 0.0 + warrant_promotion_gain: float = 0.0 + hard_vetoes: tuple[str, ...] = () + +@dataclass(frozen=True) +class ControllerInput: + organism_root: str + branches: tuple[ControllerBranch, ...] + budget: int + hermes_utilization: int + weights: ControllerWeights + difficulty: float + +@dataclass(frozen=True) +class ControllerDecision: + selected_branch_id: str | None + label: Literal[ + "ACCEPT", "MARGINAL", "REJECT", + "QUARANTINE", "UNKNOWN", "ESCALATE", "DEFERRED" + ] + allocations: dict[str, float] + difficulty_next: float + veto_reasons: dict[str, tuple[str, ...]] + entropy: float + notes: tuple[str, ...] +``` + +This lets tests run with no DB, no LLM, no background jobs. + +### 16.2 Phase 1 test list (write now even though Phase 0 is doc-only) + +```text +test_all_vetoed_returns_reject_or_quarantine +test_zero_budget_returns_deferred +test_stable_softmax_no_overflow +test_negative_payoff_gets_zero_allocation +test_unsupported_carrier_quarantines +test_cache_drift_quarantines +test_memory_invalidation_above_threshold_escalates +test_high_entropy_increases_difficulty +test_low_entropy_decreases_or_preserves_difficulty +test_divergence_increases_witness_sampling_recommendation +test_no_llm_call_in_controller +test_controller_does_not_modify_cache_key_inputs +test_controller_outputs_advisory_event_only +test_zero_mean_divergence_does_not_trigger_phase_1 (#12 Trigger 2) +test_small_sample_does_not_trigger_phase_1 (#12 Trigger 2) +test_propose_not_mutate_memory_root (§4.4) +test_propose_not_mutate_self_model (§4.4) +``` + +These are the right guardrails for the pure-function layer. + +--- + +## 17. Adjacent integrations + +### 17.1 Warrant-promotion integration (#000031) + +#000031 Phase 2 has 78 atomic claim-pack records that max out at +`ANCHOR-WARRANTED` until cited textbooks are ingested and +`derivations.proof_blob` rows are computed. The unconscious sweep +(§3.1, Target B) drains exactly this backlog. Controller scoring +includes `WarrantPromotionGain_i` (§4.1) so a branch that turns +many anchor-only claims into evidence-warranted claims is +high-value even if it doesn't move bench scores. + +### 17.2 Spatial-temporal carrier (#000013, future) + +#000013's v7-W landed as paper / frontier catalog / namespace +stub: a `frame` Merkle node committing +`(time_ns, pose, grid_cells, observations)` and four ε-frontiers +(`pose_integration`, `observation_update`, `object_logits`, +`relation_logits`). Not code yet beyond namespace reservation. + +When v7-W frames become active, the controller can treat frame +transitions as branches: + +```text +Ω_t(frame_k) → Ω_{t+1}(frame_{k+1}) +``` + +with potential future vetoes: + +```text +pose ε-frontier exceeded +relation_logits divergence +observation_update inconsistency +world-frame cache drift +``` + +**Do not implement now.** Phase 0 captures the integration +surface so future v7-W work doesn't have to retro-fit it. + +--- + +## 18. Scope boundaries — what this ticket does NOT do + +- **Does not** bump `governance_policy_hash` or any cache_key dim + in Phase 0. Phase 2+ may fold `controller_*` advisory tags into + policy hash; Phase 0/1 explicitly does not. - **Does not** introduce LLM-as-judge anywhere. All judgment is deterministic — entropy, Kelly fraction, ForkScore verdict, hard-veto checks. The LLM remains a witness only. - **Does not** replace ForkScore (#000012). ForkScore is the single-branch verdict; Prometheus-Σ is the multi-branch allocator that calls ForkScore on each candidate. -- **Does not** implement the unconscious sweep in Phase 0. Phase 1 - spec'd; Phase 1 implementation gated by §12. +- **Does not** implement the unconscious sweep in Phase 0/1. + Phase 1 is the pure function. Phase 3 spec'd; Phase 3 + implementation gated by §12. +- **Does not** mutate MemoryRoot / SelfModel directly. §4.4: + controller emits update **proposals**; existing write paths + commit them only after their own validation. +- **Does not** add `documents.last_swept_at` or + `document_sweep_state` in Phase 0. §3.1 names both options; + Phase 1 picks one. - **Does not** claim completeness, claim Gödel-defeat, or claim AGI. It implements local-versioned-replayable admissibility, full stop. See §10. - **Does not** modify `audit_events.event_hash` preimage — sibling - audit-event tags (`controller_decision`, `controller_difficulty`) - per #000020's advisory-ledger pattern. + audit-event tags (`controller_decision`, `controller_difficulty`, + `controller_budget_allocation`, `document_*` sweep tags) per + #000020's advisory-ledger pattern. --- -## 15. Open questions for go/no-go +## 19. Open questions — answered -1. **`α, β, γ, ρ, λ, μ, ν, η` weights.** The synthesis names them - but does not pick values. Phase 0 should ship a default weight- - set tied to fox's mission priority (intellectual capital ≫ - computational throughput ≫ social capital, per BLACKOPS shard). - Phase 1 may make weights pluggable per operator. -2. **Where do divergence-rate windows live?** New table, or fold - into `audit_events` with a query-time aggregation? Trade-off: - new table = clean separation; aggregation = fewer schema - surfaces. Lean toward aggregation — DRY in context. -3. **Unconscious sweep priority vs inference traffic.** Cooperative - yielding via a concurrent-Hermes utilization probe, or a - strict-priority queue? Probe is simpler; queue gives stronger - guarantees. Probe wins for Phase 1; revisit if it under-yields. -4. **Sympathetic naming.** "Prometheus-Σ" comes from the synthesis. - Internal-arborist convention is `arborist/vN/.py`. Plan: - keep "Prometheus-Σ" as the **doctrine name** (used in tickets, - papers, mesh wire docs), but the module lives at - `arborist/v9/prometheus.py` and the table tags are - `controller_*`. Name does not enter cache_key. +These were open in the original draft. David's review supplies +concrete answers; questions remain only as marked. + +**Q1. Weight values.** Three named profiles in §15. Phase 1 ships +the **safe default** (§15.1) as the baseline. Conservative and +exploratory are pluggable via config; profile name is logged on +each decision but does not (yet) enter `governance_policy_hash`. + +**Q2. Where do divergence-rate windows live?** Phase 1: aggregate +from `audit_events` at query time. **No new table.** Doc-only +Phase 0 + minimal Phase 1 + DRY-in-context all argue for +derivation over materialization. Phase 2 may add materialized +summaries if the query becomes slow under real workload. + +**Q3. Sweep priority vs inference traffic.** Cooperative yielding +in Phase 1: when live Hermes utilization ≥ `B − 1`, sleep work +yields. Strict priority queues are a Phase 2 escalation only if +the cooperative scheme underperforms in the real-shard baseline. + +**Q4. Naming.** Doctrine name **Prometheus-Σ** stays in tickets, +papers, mesh wire docs. Module path: `arborist/substrate/prometheus.py` +(under the unified `arborist/substrate/` topic dir; v7/v8 namespaces +were consolidated 2026-05-10). +Audit-event tag prefix: `controller_*`. Doctrine name does not +enter cache_key or schema enums. + +**Q5 (open).** Should `controller_*` advisory audit events fold +into `governance_policy_hash` once Phase 2 lands writes? Lean +toward yes (consistency with #000020's ledger pattern), but defer +the decision to the Phase 2 ticket. --- -## 16. Status +## 20. Status -Phase 0 (this ticket): open · awaiting go/no-go. +Phase 0 (this ticket): open · awaiting go/no-go · David review +2026-05-10 applied. When fox approves: this ticket flips to `in progress · Phase 0 landed in commit ` with no code change — the doc itself is the deliverable. -Phase 1 opens when §12 trigger fires. +Phase 1 opens when §12 trigger fires. Phase 2 (advisory audit +writes) and Phase 3 (unconscious sweep) gate sequentially on +Phase 1 having landed plus their own §12 trigger renewal. --- -## 17. References +## 21. Review log + +David review (`response_ticket-000037-...txt`, 2026-05-10 +06:37 Asia/Kuala_Lumpur). Verdict: **Go for Phase 0, with +amendments.** Changes applied to ticket text: + +1. `#000028` renamed `multi-modality witness` → + `multi-witness canonical agreement` throughout + (witness vs carrier distinction). +2. Bicameral diagram speed claim corrected — kernel is + cheap inside its supported domain; LLM is the scarce + resource (§3 note). +3. `DEFERRED` output label added (§4.3); distinguishes + "not evaluated due to budget" from "evaluated but + uncertain" (`MARGINAL`). +4. Sweep-state schema explicitly Phase 1 (§3.1); sidecar + `document_sweep_state` recommended over column-on- + `documents`. Phase 0 commits to neither. +5. Sweep targets broadened from "previously-strict" to + include CANONICAL_PROJECTION, STRICT-WITNESSED, + EVIDENCE-WARRANTED with canonical content, MARGINAL, + ANCHOR-WARRANTED claim-pack records (§3.1 Target A). +6. Numerically stable softmax mandated (§5). +7. Normalized entropy `H_norm = H / log(n)` for cross- + branch-count gating (§5). +8. Kelly safety guards: `b_i ≤ 0`, vetoed, `B = 0`, sum-zero + raw-allocations (§7). +9. Difficulty update law with EMA smoothing (§7.1). +10. Extended utility formula with SelfModelCalibrationGain, + WarrantPromotionGain, MemoryInvalidationRisk, separate + SecurityRisk (§5). +11. Hard-veto order expanded to 8 classes; soft-hash veto + integration with explicit replay-window-unbounded → + ESCALATE rule (§6). +12. Gödel framing strengthened — UNKNOWN/ESCALATE are + visible outputs, not the side-step itself (§10). The + side-step is the whole policy. Plus the "controller + must never infer" rules. +13. Phase triggers gain exception handling (Trigger 2 + division-by-zero + small-sample guards; Trigger 3 + zero-denominator guard; §12). +14. Step 12 of the algorithm now reads "emit + MemoryRoot/SelfModel update **proposals**" — controller + does not mutate hard-hash chain state directly (§4.4, + §13). +15. Concrete weight profiles (safe / conservative / + exploratory) replace the placeholder Q1 in old §15 + (new §15). +16. Exception-handling matrix added (§14). +17. Phase 1 dataclass API + test list specified (§16). +18. Lineage to #000008 (broad-quantifier preflight) and + #000010 (M0 / MCTL) named — Prometheus-Σ is the + post-answer recursive layer of an existing series + (§3.2). +19. Adjacent integrations section: warrant-promotion + (#000031), spatial-temporal carrier (#000013) (§17). + +Two doctrinal commitments preserved verbatim from the original +ticket and reaffirmed by the review: + +```text +The controller does not defeat Gödel. +LLM is witness, never authority. +``` + +--- + +## 22. References - This ticket: design-substrate doc, no code yet. +- #000008 (broad-quantifier preflight): pre-answer difficulty + control — earliest member of the metacognition series. +- #000010 (M0 / MCTL): pre-answer epistemic boundary control — + middle member of the series. - #000012 (ForkScore): single-branch verdict layer the controller consumes. -- #000028 (multi-witness): the bicameral pair, pre-named. +- #000013 (spatial-temporal substrate): future v7-W carrier + (§17.2). +- #000018 (soft-hash threat model): the security-veto input (§6). +- #000020 (capital ledger): the `B` budget input. +- #000026 (real-shard workload baseline): denominator for + Trigger 3. +- #000028 (multi-witness canonical agreement): the bicameral + pair, pre-named. - #000027 (canonical projections): committed objects to falsify. - #000025 (5F battery): falsification fixture sink. - #000017 (MemoryRoot), #000014 (SelfModel): persistent state the - controller updates. -- #000020 (capital ledger): the `B` budget input. -- #000018 (soft-hash threat model): the `X_i` security penalty - input. + controller proposes updates to (never directly mutates). - #000029 (claim-packs): versioned axiom bundles for §10 admissibility. +- #000031 (cited textbooks): warrant-promotion substrate (§17.1). +- #000036 (T3 per-window covert-channel budget): replay-window + closure tracker referenced in §6 soft-hash integration. - `docs/v8-fork-score.md`: ForkScore reference. - BLACKOPS shard "Patch the Planet" / "Balance All Workstations": - the mission anchor for the weight-set in §15 Q1. + the mission anchor for the named-profile choice in §15. - 2026-05-09 fox thread: bicameral hunch, Gödel side-step phrase, Hermes ~4 concurrent bottleneck. +- 2026-05-10 David review (Asia/Kuala_Lumpur): see §21 review + log.