# Ticket #000047 — ForkScore `_delta_*` aggregator: mean vs max vs sum

**Status:** **closed · 2026-05-11** — Option D landed: `delta_aggregator`
∈ {`mean`, `max`, `sum`} on `WeightSet` (default `mean`, unchanged →
no `ESTIMATOR_VERSION` bump); `fork_score._delta_5{s,t,f}` dispatch via
`_aggregate`; recorded in `ScoredFork.weights["delta_aggregator"]`;
the bench data behind keeping `mean` as default is in
`bench/results/5f-threshold-calibration-2026-05-11.md` §5 (run on the
#000046 below-ceiling pack). All §5 closure criteria met. See §7.
**Opened:** 2026-05-11
**Scope:** Decide how `arborist/substrate/fork_score.py:_delta_5s /
_delta_5t / _delta_5f` should aggregate the per-sub-battery Δ-rate
vector into the single `Δ5{s,t,f}` term — keep the current
**mean**, switch to **max-of-positives** / **sum**, or expose it as a
policy parameter. Doc-only proposal: trade-offs + recommendation; no
code change in this ticket.
**Audience:** fox + #000012's v8 acceptance-protocol author +
maintainers of `arborist/substrate/fork_score.py` /
`arborist/substrate/weights.py`.
**Hard constraint:** `ForkScore` constants/aggregators are *estimator
parameters* (pinned by `ESTIMATOR_VERSION`), **not**
`governance_policy_hash` inputs — changing the default bumps
`ESTIMATOR_VERSION` and stales `fork_score_branches` rows written
under the prior estimator, but it does **not** invalidate any cache
record. No schema / canonicalization change. The hard-regression
side (`HARD_REGRESSION_FLOOR` catching a single-sub drop) is
unaffected — this ticket is only about how *positive* per-sub deltas
combine.

---

## 1. Problem statement

`_delta_5{s,t,f}` currently returns the **arithmetic mean** of the
per-sub-battery Δ-rates (5 subs per battery). The #000025 §10.14
calibration (`bench/results/5f-threshold-calibration-2026-05-11.md`,
summarised in `ticket-000012` §8) made the consequence concrete — the
**5× averaging dilution**:

- A child that improves **one** 5F sub-battery by +0.06 scores
  `1.0 · (0.06 / 5) = 0.012` on the `γ·Δ5f` term → **MARGINAL**.
- A child that improves **all five** 5F subs by +0.06 scores
  `1.0 · 0.06 = 0.06` → **ACCEPT** (boundary).
- Each battery contributes a *separate* term at weight 1.0, so a
  uniform +`SIGNAL_FLOOR` across every sub of all three batteries
  scores ≈ 0.15.

So "a fork must improve by `SIGNAL_FLOOR` (0.05)" really means *one
of*: ≈ 0.25 on a single sub-battery, ≈ 0.05 uniform across one
battery's five subs, or ≈ 0.017 uniform across the whole 15-sub
suite. A fork that **specializes** — doubles one capability, leaves
the rest flat — barely moves the score. Is that the intended
sensitivity, or should a real single-sub gain weigh closer to face
value?

---

## 2. Options

### 2.1 Option A — keep `mean` (status quo)

`Δ5{s,t,f} = mean(per-sub Δ-rates)`.

- **Pro:** rewards broad, balanced improvement; no single-sub spike
  dominates; a single-sub *regression* is averaged down on the
  positive side but caught regardless by `HARD_REGRESSION_FLOOR` on
  the flag side — so the asymmetry is deliberate (regressions are
  zero-tolerance per-sub; improvements need breadth).
- **Pro:** robust to a noisy sub-battery — one flaky pack can't swing
  the whole battery term.
- **Con:** heavily damps specialization gains. A fork that genuinely
  advances one capability is told it barely improved.
- **Con:** with packs at ceiling (until #000046), the term is
  inert regardless of aggregator — the dilution only bites once
  there's headroom, which is *why this ticket parks behind #000046*.

### 2.2 Option B — `max(0, per-sub Δ)` (best-of-the-improved)

`Δ5{s,t,f} = max(0, max_i Δ_i)` — the largest single-sub gain, floored
at 0.

- **Pro:** a single-sub +0.06 gain weighs at face value → ACCEPT.
  Rewards specialization; "the fork improved *something*" ⇒ positive
  term.
- **Con:** silent on the rest of the vector. A fork that improves one
  sub by +0.05 and *regresses* four by −0.01 each (net-negative on
  that battery, but each below `HARD_REGRESSION_FLOOR`) scores
  positive on the `max` — the regressions are invisible to this term
  and slip under the hard-regression flag. Would need the
  `regression_penalty` term re-weighted to compensate.
- **Con:** discards how broad an improvement is — a uniform +0.05 and
  a single +0.05 score identically.

### 2.3 Option C — `sum` of per-sub Δs

`Δ5{s,t,f} = Σ_i Δ_i`.

- **Pro:** linear, total-improvement semantics; a single +0.06 scores
  0.06 and a uniform +0.06 across five scores 0.30 — breadth *and*
  magnitude both count.
- **Con:** over-rewards broad tiny gains: five subs each +0.011 (each
  below the 5pp bench noise floor — i.e. statistical noise, not real
  signal) sums to 0.055 ≥ `SIGNAL_FLOOR` → ACCEPT on what is plausibly
  zero real improvement. The mean is *deliberately* sub-noise-floor
  in that case.
- **Con:** the `SIGNAL_FLOOR` / `HARD_REGRESSION_FLOOR` framing in
  `docs/bench-maxing.md` is per-rate (5pp on *a* rate); a summed term
  no longer maps cleanly onto "5pp" — the floors would need
  re-derivation.

### 2.4 Option D — parameterize it (recommended)

Add `delta_aggregator ∈ {"mean", "max", "sum"}` to `WeightSet` (or a
sibling policy struct), default `"mean"`. `_delta_*` dispatches on it;
the chosen aggregator is recorded in `ScoredFork.breakdown` (so a
reader can replay the verdict) and in `fork_score_branches`'
`breakdown_blob`. `ESTIMATOR_VERSION` bumps **only if the default
changes** — a per-call override at the non-default value is recorded
but doesn't restate the estimator generation for everyone.

- **Pro:** #000012's v8 acceptance protocol picks per-deployment
  without a code change; the default stays `"mean"` so nothing breaks
  silently; the choice is auditable.
- **Con:** another knob (Five-step §2 "delete the part" pushes back —
  but this knob has a real, named requester: #000012's protocol
  author wanting to tune fork sensitivity, and the alternative is
  re-litigating mean-vs-max every time someone wants the other).

---

## 3. Recommendation

**Option D — parameterize, default `mean`** — but **do not implement
until #000046 lands a below-ceiling baseline.** Rationale:

1. The dilution only matters once a pack has headroom; until then any
   choice is synthetic (the §10.14 calibration already shows the
   mechanics in-memory). Implementing now would be guessing.
2. Once #000046's hard pack exists, the aggregator choice becomes a
   bench question, not an opinion: pin a parent below ceiling, hand-
   craft a "broad small improvement" child and a "single big
   improvement" child, run all three aggregators, see which one's
   ACCEPT/MARGINAL/REJECT verdicts match the intuition fox wants. The
   result decides the default (still recording the others as
   selectable).
3. `mean` stays the default regardless — it's the conservative,
   noise-robust, asymmetric-with-the-regression-side choice that
   matches `docs/bench-maxing.md`'s per-rate floor framing.

So this ticket parks behind #000046, then opens as a small
implementation (≈ 30 LOC + a `WeightSet` field + tests + an
`ESTIMATOR_VERSION` note if the default moves, which it shouldn't).

---

## 4. Out of scope

- **Changing the default away from `mean`** — that needs the #000046
  bench data; this ticket recommends *keeping* `mean` as default and
  only adding the knob.
- **Re-weighting `regression_penalty`** — Option B would need it; if
  fox ever picks `max` as default, that's part of *that* change, not
  a separate ticket.
- **Touching `HARD_REGRESSION_FLOOR` / `SIGNAL_FLOOR`** — #000025
  §10.14 already recommended keeping both at 0.05; this ticket
  doesn't reopen that.
- **The harder-fixture tier itself** — #000046.
- **`bench/scripts/fivef_threshold_calibration.py`** — it already
  exercises the mechanics; if the aggregator becomes a parameter,
  the calibration script gains a column showing each aggregator's
  verdict (small follow-on, part of the implementation).

---

## 5. Status

**Closed · 2026-05-11 — Option D landed.** Receipt:

- **`WeightSet.delta_aggregator: str = "mean"`** (`arborist/substrate/weights.py`)
  — a categorical field, validated by `__post_init__` against
  `DELTA_AGGREGATORS = ("mean", "max", "sum")`. `from_dict` takes it
  as a string (doesn't `float()` it). The default is **unchanged**
  (`mean`), so existing `ScoredFork` output is byte-identical and
  `fork_score.ESTIMATOR_VERSION` does **not** bump.
- **`fork_score._aggregate(deltas, how)`** — the dispatch helper:
  `mean` = arithmetic mean, `max` = `max(0.0, max_i Δ_i)`, `sum` =
  `Σ Δ_i`; empty vector → 0.0 under all three. `_delta_5s / _delta_5t
  / _delta_5f` take an `aggregator` arg (default `"mean"`); the 5F
  efficiency bonus is added *after* the aggregated base, so it's
  aggregator-independent. `fork_score` passes `weights.delta_aggregator`.
- **Per-sub regression flags are aggregator-independent** — the
  `HARD_REGRESSION_FLOOR` check fires on each per-sub Δ before
  aggregation, so a single-sub regression still forces REJECT under
  `max`/`sum` (`test_hard_regression_flag_independent_of_aggregator`).
- **Recording:** the chosen aggregator is in
  `ScoredFork.weights["delta_aggregator"]` (via `WeightSet.as_dict()`,
  which `fork_score` already returns). `fork_score_branches`
  traceability is via the opaque `weights_id` (the caller sets it to
  reflect the weights+aggregator combo) — not a new column, no schema
  migration.
- **Bench data behind the default choice:**
  `bench/scripts/fivef_threshold_calibration.py` gained §5 — runs the
  #000046 below-ceiling pack (`5f/falsification` at 0.333) and shows
  the verdict / `γ·Δ5f` under each aggregator;
  `bench/results/5f-threshold-calibration-2026-05-11.md` §5 is the
  captured record. Reading: `mean` dilutes a single-sub verifier fix
  1/5 (rewards breadth over specialization); `max`/`sum` weigh it at
  face value. Default stays `mean` — the conservative, noise-robust,
  regression-symmetric choice that matches `docs/bench-maxing.md`'s
  per-rate floor framing.
- **Tests:** 8 in `tests/test_fork_score.py` (default-is-mean,
  `_aggregate` helper, bad-aggregator-rejected, `from_dict`,
  single-sub-gain face value under max/sum vs diluted under mean,
  sum-vs-max diverge on a broad gain, aggregator-recorded-in-weights,
  hard-regression-flag-independent) + 1 anchor in
  `tests/test_fivef_threshold_calibration.py` (§5 present, all three
  aggregators named) + the `as_dict` field-set test in
  `tests/test_weights.py` updated to include `delta_aggregator`.

The original §1–§4 below stays as the design log; §3's Option D is
what shipped.

---

## 6. References

- `arborist/substrate/fork_score.py` (`_delta_5s` / `_delta_5t` /
  `_delta_5f`, `SIGNAL_FLOOR`, `HARD_REGRESSION_FLOOR`,
  `ESTIMATOR_VERSION`) — the code this ticket would touch.
- `arborist/substrate/weights.py` (`WeightSet`) — where
  `delta_aggregator` would live.
- `bench/results/5f-threshold-calibration-2026-05-11.md` (§3 sanity
  checks, §4 "5× averaging dilution") + `ticket-000012` §8 — the
  calibration finding that motivates this ticket.
- `docs/bench-maxing.md` — the per-rate 5pp signal-floor framing the
  `mean` aggregator preserves and `sum` would break.
- `docs/tickets/ticket-000046-harder-5sf-fixture-tier.md` — the
  prerequisite; the aggregator choice is benchable only once a hard
  pack exists.
- `docs/tickets/ticket-000012-selection-consensus-protocol.md` (§5
  fork choice, §8 calibration handoff) — the v8 protocol that would
  consume `delta_aggregator`.
- `docs/tickets/ticket-000025-5f-battery.md` §10.14 — where this
  loose end was flagged.
