#000046 + #000047 — open the two #000025 §10.14 follow-up tickets

The 5F threshold calibration (#000025 §10.14) surfaced two loose ends
that #000012's v8 acceptance protocol owns; opening them as proper
proposals so they don't get lost.

#000046 — Harder 5S/5T/5F fixture tier (below-ceiling baselines).
Every canonical 5S/5T/5F pack runs at rate 1.0 at HEAD, so fork_score's
α·Δ5s+β·Δ5t+γ·Δ5f terms are inert (a child can only move them ≤ 0).
The embedded packs can't be made "harder" (their evaluator is the
gold); a harder tier is necessarily a live-path tier where a real,
imperfect arborist surface produces wrong/partial output. Recommends
Option A scoped narrow: a `falsification-hard-v1.jsonl` built around
the verify_quotes entity-strategy gap already documented by
5f-fal-live-003, confirm the rate sits below 1.0 and is stable, pin it
via `make bench-fork-baseline-hard`, demonstrate a planted surface fix
lifting the rate. Status: open · awaiting go/no-go.

#000047 — ForkScore _delta_* aggregator (mean vs max vs sum). _delta_*
means over a battery's 5 sub-batteries, so a single-sub gain is worth
a fifth of face value (the 5× dilution from the §10.14 calibration:
+0.06 on one 5F sub → MARGINAL, same +0.06 on all five → ACCEPT).
Recommends Option D: parameterize delta_aggregator on WeightSet,
default "mean", record it in the breakdown + fork_score_branches; bump
ESTIMATOR_VERSION only if the default changes. Defer implementation
until #000046 lands a below-ceiling baseline so the aggregator choice
is benchable, not guessed. Status: open · awaiting go/no-go; parks
behind #000046.

TICKETS.md index rows + Next ID bump (000046 → 000048) landed in
8599ce3 (swept in by a concurrent commit; content is correct).
ticket-000012 §8 §3/§4 updated to name #000047 / #000046.

docs/tickets/ticket-000012-selection-consensus-protocol.md

@@ -554,7 +554,10 @@ protocol must account for:
    +`SIGNAL_FLOOR` across every sub of all three batteries scores
    ≈ 0.15. If v8 wants single-sub improvements to weigh equally,
    switch `_delta_*` from mean to a max/sum aggregator — a design
-   call this ticket owns, not a calibration finding.
+   call this ticket owns, not a calibration finding. Tracked as
+   **#000047** (recommends parameterizing `delta_aggregator`, default
+   `mean`; parks until #000046 produces a below-ceiling baseline to
+   bench the choice against).
 
 4. **Ceiling saturation.** Every 5S/5T/5F pack is at rate 1.0 at
    HEAD, so `α·Δ5s + β·Δ5t + γ·Δ5f` can only be ≤ 0 — an unchanged
@@ -565,5 +568,6 @@ protocol must account for:
    `feedback_efficiency`) + the non-bench terms
    (`selfmodel_calibration_gain`, `audit_completeness`,
    `validator_diversity`, …), not by bench Δ-rate. Adding the
-   harder tier is tracked as a #000025 follow-up, not a #000012
-   blocker.
+   harder tier is tracked as **#000046** (a #000025 follow-up,
+   recommends a narrow live-path hard tier starting with 5F
+   Falsification), not a #000012 blocker.

docs/tickets/ticket-000046-harder-5sf-fixture-tier.md

@@ -0,0 +1,188 @@
+# Ticket #000046 — Harder 5S/5T/5F fixture tier (below-ceiling baselines)
+
+**Status:** open · awaiting go/no-go
+**Opened:** 2026-05-11
+**Scope:** Give the 5S/5T/5F bench packs *headroom* — a fixture tier
+the runtime does **not** pass at rate 1.0 — so the ForkScore Δ-rate
+terms (`α·Δ5s + β·Δ5t + γ·Δ5f` in `arborist/substrate/fork_score.py`)
+can be strictly positive for a genuine improvement. Doc-only proposal:
+options + a narrow recommended start; no fixtures authored in this
+ticket.
+**Audience:** fox + #000025 follow-up readers + #000012's v8
+acceptance-protocol author + future bench authors.
+**Hard constraint:** the existing `*-v1.jsonl` packs keep their pinned
+fixture digests — a harder tier is a **new** pack (`*-hard-v1.jsonl`
+or a `tier` field), never an edit to a pinned one. No schema /
+governance-hash / canonicalization-version change. No LLM-as-judge in
+any runner (the discipline that lets the bench be a scoreboard, not a
+vibe check).
+
+---
+
+## 1. Problem statement
+
+The #000025 §10.14 ForkScore threshold calibration
+(`bench/scripts/fivef_threshold_calibration.py` →
+`bench/results/5f-threshold-calibration-2026-05-11.md`, summarised in
+`ticket-000012` §8) surfaced a structural fact: **every canonical 5S,
+5T, and 5F pack runs at rate 1.0 at HEAD.** Consequence in
+`fork_score`:
+
+- `_delta_5s / _delta_5t / _delta_5f` are means of per-sub-battery
+  Δ-rates. A child at ceiling vs a parent at ceiling ⇒ every Δ is 0
+  ⇒ the three battery terms contribute exactly 0. An *unchanged*
+  child therefore scores 0 → MARGINAL (not ACCEPT — which is the
+  right behaviour, but it means the bench terms are inert).
+- A child can only move those terms *negative* (regress a pack) — the
+  bench Δ-rate has no upside at the current pack difficulty.
+- So fork *acceptance* today is carried entirely by the
+  efficiency-aware bonuses (`adaptation_efficiency` /
+  `feedback_efficiency` deltas) and the non-bench terms
+  (`selfmodel_calibration_gain`, `audit_completeness`,
+  `validator_diversity`, …). The "did the organism's measurable
+  capabilities improve?" signal is dormant.
+
+For the bench Δ-rate to carry positive signal, *something the runtime
+gets partially wrong* has to be in the corpus. The embedded packs
+can't be that something: the embedded evaluators are deterministic
+and the gold answers are constructed to match them — a fixture the
+embedded evaluator "fails" would be a fixture with a wrong gold, not
+a hard problem. So a harder tier is necessarily a **live-path** tier
+— inputs fed to a real, imperfect arborist surface (`parse_claims`,
+`verify_quotes`, `store_snapshot`, the retrieval pipeline) where the
+surface produces wrong / partial / over-confident output and the
+fixture's `expected` records the *correct* outcome the surface
+misses. Then rate < 1.0, and a future improvement to that surface
+moves the rate up — real positive Δ.
+
+---
+
+## 2. Options
+
+### 2.1 Option A — live-path "hard" tier (recommended, narrow start)
+
+Add a `*-hard-v1.jsonl` pack to one sub-battery where the live
+surface most plausibly fails on difficult input, confirm the rate
+sits below 1.0 *and is stable across runs* (the live surfaces are
+deterministic — imperfect, not random — so rate must be a fixed
+fraction), then a `make bench-fork-baseline-hard` that pins it as a
+below-ceiling parent. Iterate to other sub-batteries only if the
+loop proves out.
+
+Two strong candidates for the first hard pack:
+
+- **5F Falsification (`verify_quotes`)** — there's already a
+  documented soft-signal gap: `5f-fal-live-003` records that the
+  entity-strategy verifier soft-matches "Insulin / Penicillin" as one
+  claim because both name Alexander Fleming. A hard pack of near-miss
+  quote pairs that *should* land UNGROUNDED but the entity strategy
+  over-grounds would put the Falsification rate below 1.0 today; a
+  future tightening of the entity strategy (or the title-relevance
+  hard check picking up more) raises it. Directly bench-able.
+- **5F Formulate (`parse_claims`)** — genuinely messy prose where the
+  claim-lattice parser produces the wrong claim count / pointer-set /
+  order. The `match_policy` is already structural (sorted-approx
+  claim text + exact-set pointer IDs + required order), so a hard
+  pack just needs inputs the parser mis-segments.
+
+Cost: ~10-20 hard fixtures for the first pack + a make target + a
+test pinning "rate is below 1.0 and equals X". Risk: low — the live
+surfaces are deterministic; the only failure mode is "the surface is
+actually perfect on the inputs we picked", which just means picking
+harder ones.
+
+### 2.2 Option B — deliberately-degraded parent baseline
+
+Ship a pinned `bench/results/baseline-degraded-v1.json` with
+knocked-down rates (e.g. 0.85 uniformly), so a real child run (at
+1.0) shows +0.15 → ACCEPT. Cheap (one JSON file, no fixtures). But
+the "improvement" is fictional — the child didn't get better, the
+parent was artificially handicapped. This is useful for *exercising
+the ForkScore mechanics* (which the §10.14 calibration already does
+in-memory via `_degraded_parent`), not for real fork selection. It
+does **not** make the bench Δ-rate carry real signal; it just makes
+the synthetic demo persist as a file. Probably not worth a ticket on
+its own — note it here as the cheap alternative we explicitly chose
+*not* to pursue as the primary path.
+
+### 2.3 Option C — do nothing; revisit when headroom surfaces organically
+
+Accept that the bench Δ-rate is inert at the current pack difficulty,
+keep the §8 note in #000012, and add a hard tier only when one shows
+up for free: a new π* domain whose kernel is imperfect, an
+LLM-backed sub-battery, or a real regression that drops a pack below
+ceiling. Lowest cost; the cost is that the dormant signal stays
+dormant indefinitely.
+
+---
+
+## 3. Recommendation
+
+**Option A, scoped to one sub-battery first — Falsification.** It has
+a pre-documented gap (`5f-fal-live-003`), the live surface
+(`verify_quotes`) is deterministic, and a hard pack there immediately
+gives ForkScore a below-ceiling 5F sub-battery whose rate a future
+verifier improvement can lift. If the loop proves out (rate stable
+below 1.0, a planted verifier improvement demonstrably raises it,
+ForkScore shows a positive `γ·Δ5f` term), extend to Formulate, then
+to whichever 5S/5T sub-batteries have an imperfect live surface
+(retrieval-backed ones are the obvious next look). Treat Option B as
+a documented non-choice and Option C as the fallback if A's first
+pack turns out perfect on every input we can author.
+
+Defer **#000047** (`_delta_*` mean-vs-max-vs-sum aggregation) until at
+least one hard pack lands — the aggregator choice is empirically
+testable only once there's a below-ceiling baseline to bench against.
+
+---
+
+## 4. Out of scope
+
+- **Authoring the hard fixtures** — this ticket picks the strategy
+  and the first sub-battery; the fixture pack lands in the
+  implementation commit chain this ticket gates.
+- **Changing `fork_score`** — the scorer already consumes whatever
+  rates the packs produce. A hard tier doesn't touch
+  `fork_score.py`; it changes the *inputs*.
+- **Re-pinning the existing `*-v1.jsonl` packs** — hard off-limits;
+  those digests are referenced across closed tickets.
+- **An LLM-backed sub-battery** — a different (larger) way to get
+  headroom; out of scope here, ticket-able separately if it ever
+  becomes the path.
+- **`_delta_*` aggregator choice** — #000047.
+
+---
+
+## 5. Status
+
+**Open · awaiting go/no-go.** Doc-only proposal. Opens when fox picks
+a strategy (recommendation: Option A / Falsification first). The
+implementation commit chain it gates: (1) author the first hard pack,
+(2) confirm rate < 1.0 and stable + add the pinning test, (3) `make
+bench-fork-baseline-hard`, (4) demonstrate a planted surface
+improvement lifting the rate + a positive ForkScore Δ term, (5)
+extend to the next sub-battery if the loop holds. Closure: ≥ 1 hard
+pack live with a below-ceiling stable rate AND a worked example
+showing the ForkScore bench-Δ term going positive on a real
+improvement.
+
+---
+
+## 6. References
+
+- `bench/scripts/fivef_threshold_calibration.py` +
+  `bench/results/5f-threshold-calibration-2026-05-11.md` — the
+  calibration that surfaced the ceiling-saturation problem.
+- `docs/tickets/ticket-000012-selection-consensus-protocol.md` §8 —
+  the handoff section recording "ceiling saturation → bench Δ-rate
+  inert; track a harder tier as a #000025 follow-up".
+- `docs/tickets/ticket-000025-5f-battery.md` — parent; this is the
+  follow-up the §10.14 closure flagged.
+- `bench/fixtures/5f/falsification-live-v1.jsonl` (`5f-fal-live-003`)
+  — the pre-documented entity-strategy soft-signal gap the first
+  hard pack would build around.
+- `arborist/qa/verify.py` (`verify_quotes`, the entity strategy) /
+  `arborist/qa/parse_claims.py` — the imperfect live surfaces a hard
+  tier exercises.
+- `docs/tickets/ticket-000047-fork-score-delta-aggregator.md` —
+  sibling; parks until this ticket produces a below-ceiling baseline.

docs/tickets/ticket-000047-fork-score-delta-aggregator.md

@@ -0,0 +1,203 @@
+# Ticket #000047 — ForkScore `_delta_*` aggregator: mean vs max vs sum
+
+**Status:** open · awaiting go/no-go (parks until #000046 produces a
+below-ceiling baseline to bench against)
+**Opened:** 2026-05-11
+**Scope:** Decide how `arborist/substrate/fork_score.py:_delta_5s /
+_delta_5t / _delta_5f` should aggregate the per-sub-battery Δ-rate
+vector into the single `Δ5{s,t,f}` term — keep the current
+**mean**, switch to **max-of-positives** / **sum**, or expose it as a
+policy parameter. Doc-only proposal: trade-offs + recommendation; no
+code change in this ticket.
+**Audience:** fox + #000012's v8 acceptance-protocol author +
+maintainers of `arborist/substrate/fork_score.py` /
+`arborist/substrate/weights.py`.
+**Hard constraint:** `ForkScore` constants/aggregators are *estimator
+parameters* (pinned by `ESTIMATOR_VERSION`), **not**
+`governance_policy_hash` inputs — changing the default bumps
+`ESTIMATOR_VERSION` and stales `fork_score_branches` rows written
+under the prior estimator, but it does **not** invalidate any cache
+record. No schema / canonicalization change. The hard-regression
+side (`HARD_REGRESSION_FLOOR` catching a single-sub drop) is
+unaffected — this ticket is only about how *positive* per-sub deltas
+combine.
+
+---
+
+## 1. Problem statement
+
+`_delta_5{s,t,f}` currently returns the **arithmetic mean** of the
+per-sub-battery Δ-rates (5 subs per battery). The #000025 §10.14
+calibration (`bench/results/5f-threshold-calibration-2026-05-11.md`,
+summarised in `ticket-000012` §8) made the consequence concrete — the
+**5× averaging dilution**:
+
+- A child that improves **one** 5F sub-battery by +0.06 scores
+  `1.0 · (0.06 / 5) = 0.012` on the `γ·Δ5f` term → **MARGINAL**.
+- A child that improves **all five** 5F subs by +0.06 scores
+  `1.0 · 0.06 = 0.06` → **ACCEPT** (boundary).
+- Each battery contributes a *separate* term at weight 1.0, so a
+  uniform +`SIGNAL_FLOOR` across every sub of all three batteries
+  scores ≈ 0.15.
+
+So "a fork must improve by `SIGNAL_FLOOR` (0.05)" really means *one
+of*: ≈ 0.25 on a single sub-battery, ≈ 0.05 uniform across one
+battery's five subs, or ≈ 0.017 uniform across the whole 15-sub
+suite. A fork that **specializes** — doubles one capability, leaves
+the rest flat — barely moves the score. Is that the intended
+sensitivity, or should a real single-sub gain weigh closer to face
+value?
+
+---
+
+## 2. Options
+
+### 2.1 Option A — keep `mean` (status quo)
+
+`Δ5{s,t,f} = mean(per-sub Δ-rates)`.
+
+- **Pro:** rewards broad, balanced improvement; no single-sub spike
+  dominates; a single-sub *regression* is averaged down on the
+  positive side but caught regardless by `HARD_REGRESSION_FLOOR` on
+  the flag side — so the asymmetry is deliberate (regressions are
+  zero-tolerance per-sub; improvements need breadth).
+- **Pro:** robust to a noisy sub-battery — one flaky pack can't swing
+  the whole battery term.
+- **Con:** heavily damps specialization gains. A fork that genuinely
+  advances one capability is told it barely improved.
+- **Con:** with packs at ceiling (until #000046), the term is
+  inert regardless of aggregator — the dilution only bites once
+  there's headroom, which is *why this ticket parks behind #000046*.
+
+### 2.2 Option B — `max(0, per-sub Δ)` (best-of-the-improved)
+
+`Δ5{s,t,f} = max(0, max_i Δ_i)` — the largest single-sub gain, floored
+at 0.
+
+- **Pro:** a single-sub +0.06 gain weighs at face value → ACCEPT.
+  Rewards specialization; "the fork improved *something*" ⇒ positive
+  term.
+- **Con:** silent on the rest of the vector. A fork that improves one
+  sub by +0.05 and *regresses* four by −0.01 each (net-negative on
+  that battery, but each below `HARD_REGRESSION_FLOOR`) scores
+  positive on the `max` — the regressions are invisible to this term
+  and slip under the hard-regression flag. Would need the
+  `regression_penalty` term re-weighted to compensate.
+- **Con:** discards how broad an improvement is — a uniform +0.05 and
+  a single +0.05 score identically.
+
+### 2.3 Option C — `sum` of per-sub Δs
+
+`Δ5{s,t,f} = Σ_i Δ_i`.
+
+- **Pro:** linear, total-improvement semantics; a single +0.06 scores
+  0.06 and a uniform +0.06 across five scores 0.30 — breadth *and*
+  magnitude both count.
+- **Con:** over-rewards broad tiny gains: five subs each +0.011 (each
+  below the 5pp bench noise floor — i.e. statistical noise, not real
+  signal) sums to 0.055 ≥ `SIGNAL_FLOOR` → ACCEPT on what is plausibly
+  zero real improvement. The mean is *deliberately* sub-noise-floor
+  in that case.
+- **Con:** the `SIGNAL_FLOOR` / `HARD_REGRESSION_FLOOR` framing in
+  `docs/bench-maxing.md` is per-rate (5pp on *a* rate); a summed term
+  no longer maps cleanly onto "5pp" — the floors would need
+  re-derivation.
+
+### 2.4 Option D — parameterize it (recommended)
+
+Add `delta_aggregator ∈ {"mean", "max", "sum"}` to `WeightSet` (or a
+sibling policy struct), default `"mean"`. `_delta_*` dispatches on it;
+the chosen aggregator is recorded in `ScoredFork.breakdown` (so a
+reader can replay the verdict) and in `fork_score_branches`'
+`breakdown_blob`. `ESTIMATOR_VERSION` bumps **only if the default
+changes** — a per-call override at the non-default value is recorded
+but doesn't restate the estimator generation for everyone.
+
+- **Pro:** #000012's v8 acceptance protocol picks per-deployment
+  without a code change; the default stays `"mean"` so nothing breaks
+  silently; the choice is auditable.
+- **Con:** another knob (Five-step §2 "delete the part" pushes back —
+  but this knob has a real, named requester: #000012's protocol
+  author wanting to tune fork sensitivity, and the alternative is
+  re-litigating mean-vs-max every time someone wants the other).
+
+---
+
+## 3. Recommendation
+
+**Option D — parameterize, default `mean`** — but **do not implement
+until #000046 lands a below-ceiling baseline.** Rationale:
+
+1. The dilution only matters once a pack has headroom; until then any
+   choice is synthetic (the §10.14 calibration already shows the
+   mechanics in-memory). Implementing now would be guessing.
+2. Once #000046's hard pack exists, the aggregator choice becomes a
+   bench question, not an opinion: pin a parent below ceiling, hand-
+   craft a "broad small improvement" child and a "single big
+   improvement" child, run all three aggregators, see which one's
+   ACCEPT/MARGINAL/REJECT verdicts match the intuition fox wants. The
+   result decides the default (still recording the others as
+   selectable).
+3. `mean` stays the default regardless — it's the conservative,
+   noise-robust, asymmetric-with-the-regression-side choice that
+   matches `docs/bench-maxing.md`'s per-rate floor framing.
+
+So this ticket parks behind #000046, then opens as a small
+implementation (≈ 30 LOC + a `WeightSet` field + tests + an
+`ESTIMATOR_VERSION` note if the default moves, which it shouldn't).
+
+---
+
+## 4. Out of scope
+
+- **Changing the default away from `mean`** — that needs the #000046
+  bench data; this ticket recommends *keeping* `mean` as default and
+  only adding the knob.
+- **Re-weighting `regression_penalty`** — Option B would need it; if
+  fox ever picks `max` as default, that's part of *that* change, not
+  a separate ticket.
+- **Touching `HARD_REGRESSION_FLOOR` / `SIGNAL_FLOOR`** — #000025
+  §10.14 already recommended keeping both at 0.05; this ticket
+  doesn't reopen that.
+- **The harder-fixture tier itself** — #000046.
+- **`bench/scripts/fivef_threshold_calibration.py`** — it already
+  exercises the mechanics; if the aggregator becomes a parameter,
+  the calibration script gains a column showing each aggregator's
+  verdict (small follow-on, part of the implementation).
+
+---
+
+## 5. Status
+
+**Open · awaiting go/no-go; parks until #000046 produces a
+below-ceiling baseline.** Doc-only spec for the aggregator decision.
+Opens when: #000046 lands ≥ 1 hard pack with a stable below-ceiling
+rate (so the choice can be benched), OR fox decides the parameter is
+worth adding now regardless. Closure: `delta_aggregator` is a
+`WeightSet`/policy field with `mean` default, `_delta_*` dispatches on
+it, the value is recorded in `breakdown` + `fork_score_branches`, and
+the bench data behind the default choice is captured (in a
+`bench/results/` note or this ticket).
+
+---
+
+## 6. References
+
+- `arborist/substrate/fork_score.py` (`_delta_5s` / `_delta_5t` /
+  `_delta_5f`, `SIGNAL_FLOOR`, `HARD_REGRESSION_FLOOR`,
+  `ESTIMATOR_VERSION`) — the code this ticket would touch.
+- `arborist/substrate/weights.py` (`WeightSet`) — where
+  `delta_aggregator` would live.
+- `bench/results/5f-threshold-calibration-2026-05-11.md` (§3 sanity
+  checks, §4 "5× averaging dilution") + `ticket-000012` §8 — the
+  calibration finding that motivates this ticket.
+- `docs/bench-maxing.md` — the per-rate 5pp signal-floor framing the
+  `mean` aggregator preserves and `sum` would break.
+- `docs/tickets/ticket-000046-harder-5sf-fixture-tier.md` — the
+  prerequisite; the aggregator choice is benchable only once a hard
+  pack exists.
+- `docs/tickets/ticket-000012-selection-consensus-protocol.md` (§5
+  fork choice, §8 calibration handoff) — the v8 protocol that would
+  consume `delta_aggregator`.
+- `docs/tickets/ticket-000025-5f-battery.md` §10.14 — where this
+  loose end was flagged.