# Ticket #000060 — H-ABCDEFG same-model substrate-delta harness (+ jaggedness tensor + performance curvature)

**Status:** in progress · v1 deterministic jaggedness instrument
landed 2026-05-21 (`bench/jaggedness.py` + `make bench-jaggedness`);
A-vs-C answer-quality arm already existed under #000057
(`control_ab.py` / `control_sweep.py`) — NOT rebuilt. Curvature +
LLM-arm jaggedness delta still open (see §8).
**Opened:** 2026-05-20
**Scope:** The report's "decisive proof" — a harness that runs the
SAME base model with and without the arborist substrate over
long-horizon / adversarial / non-jagged batteries, and reports the
delta. Produces two new metrics — a jaggedness tensor and discrete
performance curvature — as first-class outputs.
**Audience:** fox + a Dav1d de-novo review of the empirical bet + bench
maintainers (`bench/`, `docs/benchmarks.md`).
**Hard constraint:** deterministic where possible; gate numbers come
from bench-qa pipeline output, never contrived fixtures (CLAUDE.md
bench-maxing rule); n=3, 5pp signal floor. Mined fixtures complement,
never replace, the curated adversarial set.

---

## 1. Origin

Dav1dPrometheus, *Protocol-Layer AGI, Recursive Falsification, and the
Final Bet* (2026-05-20 working report; held outside the repo per fox).
Across **§26, §50, §82-84** the report's single most-repeated claim is
that the architecture is *theoretically strong but not empirically
settled*, and the decisive next step is:

> same model · same compute · same tasks · with and without substrate
> · measure H-ABCDEFG performance.

That is a measurement gap, not a strategy gap — squarely in arborist's
remit and the cheapest high-value thing to build.

## 2. What exists vs the gap

**Exists:** the benchmark surface (5S/5T/5F/5R batteries, deterministic
fixtures, no LLM-as-judge); `bench/mine_questions.py` +
`bench/recall_at_k.py` (deterministic retrieval recall@k); `bench-qa`
QA-quality sweep; the cost/quality matrix work (#000057).

**Gap:** none of these run the **same base model in a substrate-on /
substrate-off A/B** and report the delta. The report's systems:

```
A: base model + standard agent scaffold
B: base model + lab-style memory/tools/evals
C: base model + Arborist / PROMETHEUS-Σ substrate
D: base model + open proof mesh        (out of scope here — #000012/#000016)
```

For arborist the tractable, honest cut is **A vs C** (and optionally B):
toggle the substrate, hold the model + tasks fixed, measure.

## 3. The two new metrics (report §73, §5)

- **Jaggedness tensor (§73).** For a task family with semantically
  nearby variants, `J = variance of performance across variants`,
  normalized by expected difficulty (`J_norm`). A system with high
  benchmark score but high jaggedness fails the Hassabis non-jagged
  bar. Testable prediction the harness checks: *same base model +
  substrate should reduce `J_norm` vs the model alone.* This rides the
  existing `mine_questions` / perturbation tooling — generate nearby
  variants, measure variance.
- **Performance curvature (§5.2).** Discrete second difference
  `κ_t = P_t − 2·P_{t-1} + P_{t-2}` over upgrade cycles, with the
  honest bound correction (§5.3, Erratum 5): performance is bounded in
  `[0,1]`, so **no global-convexity claim** — target positive slope +
  non-destructive curvature in the unsaturated regime, route to new
  constraint surfaces when curvature decays.

## 4. Coupled fold (NOT a new ticket): curvature-aware ForkScore → #000012

The report's **§11 Priority 3 / §7.3** wants ForkScore extended with
ΔHABCDEFG, κ, jaggedness-delta, falsifier-coverage-delta. That is an
**extension of an in-progress ticket, not a new one** — `weights.py`
already reserves the slots (`iota` SecurityRisk, `kappa` Complexity).
Per don't-proliferate, this folds into **#000012** (Selection &
consensus / ForkScore) as a new phase, consuming this harness's metric
outputs. Cross-link both ways; do not spawn a sibling.

Suggested extension (recorded for #000012, hard rejects unchanged —
hard regression / NEG_INF_REGRESSION / rollback-missing / audit-break
all still reject):

```
ExtendedForkScore = ForkScore
  + a·ΔHABCDEFG + b·κ_HABCDEFG + c·ΔFalsifierCoverage
  + d·ΔJaggednessReduction + e·ΔPoisonSurvivalReduction
  − f·LatencyPenalty − g·SecurityRisk
```

## 5. Scope boundaries

- A-vs-C (substrate off/on) is the spine; B (lab-style memory/tools) is
  optional; D (proof mesh) is explicitly OUT (consensus/mesh =
  #000012/#000016).
- Budget discipline: control-arm LLM calls route to Hermes-3-8B +
  Qwen, never Opus, without an explicit fox go (CLAUDE.md budget rule).
  Heavy off-device passes run on the GPU box, never in arborist's
  python+sqlite3 core.
- Deterministic retrieval-recall arms (no LLM) fan out in parallel;
  only same-fixture LLM A/B that toggles state needs serialization.
- ABCDEFG self-validation caveat (report §27.2, Erratum 4 / Anomaly 4):
  include external held-out / mechanism-agnostic variants so the
  harness does not reward the substrate by construction.

## 6. Done when

A `make` target runs the same base model substrate-off vs substrate-on
over the batteries and reports per-battery delta + `J_norm` + curvature,
with gate numbers from bench-qa pipeline output (n≥3); the jaggedness
and curvature outputs are consumable by the #000012 ForkScore
extension. `docs/benchmarks.md` gains the new harness + metric schema.
