Two paired changes addressing fox's "learn this hyperparameter
from model use, not hard coding" + the 1M-context-window
caveat:
(1) Log-scale prompt-size buckets extend from 8KB through 1M+:
<8KB / 8-16KB / 16-32KB / 32-64KB / 64-128KB / 128-256KB /
256-512KB / 512K-1M / >=1M
The same bench harness now covers 8B-class models (Hermes 82K
context, max useful prompt ~32-64KB) through 1M-context models
(Gemini 1.5 Pro, Claude with extended context, Llama 4) without
code change. A model whose context window stops at 82K simply
never populates the giant buckets; a 1M-context model fills
them and finds its own sweet spot.
(2) New "recommended context budget (learned from this bench)"
section — per-mode peak-strict-rate bucket. Operator-driven
landing per the five-step algorithm step 5: surfaced, not
auto-applied. Minimum sample size of 5 runs per bucket so
statistical noise doesn't masquerade as signal. Tie-break on
smaller-bucket-wins so equivalent strict-rates favor the
cheaper choice.
The substrate is now self-tuning at the OBSERVATION layer: bench
records what budget actually grades best per model. Per-model
profile JSON (storing the recommended budget back into
~/.aborist/model_profiles/<model>.json) is the next beat once
this surface is observable in real bench runs.
3 new bench tests cover the recommended-budget section, the
sample-size floor, and the giant-context bucket coverage.
Full suite: 746 passed (was 738, +8).
Connects to D8 (automate after test-pinning): the bench tells us;
we don't guess.
russell@unturf.com
4ba4a815e0