ticket(#000008): mark Phases 0.x-4 landed; commit chain pinned

Updates §8 Status, §9.6 Phase details, §9.8 Test surface, §9.11
commit sequence to reflect actual implementation state:

§8 Status — Phase 0 through Phase 4 all landed 2026-05-03 across
six commits (2ffed005a60e85). 906 tests passing (120 new).
Defaults preserve §10.11.3 dry-run discipline:
  quantifier_guard_apply_caps=False, reminder=False, reject=False.
Six-level disable hierarchy fully wired. Live verification recorded
for both reject-broad-fires (Winners of all major sports?) and
reject-broad-skips (name all members of the Beatles → bounded
universal, NOT rejected).

§9.6 Phase details — each phase tagged LANDED <sha>; description
trimmed to what actually shipped vs the original proposal.

§9.8 Test surface — every check box flipped to [x] with the
matching test file path and test count. The two reject-broad
integration tests remain skipped (exercised by live bench).

§9.11 Implementation commit sequence — actual SHAs replace planned
commit numbers. Notes that DAG/audit binding for quantifier_preflight
node ("commit 7" in plan) is NOT shipped — tracked as Phase 5
follow-up. Optional now; required for §9.5 Merkle-AGI-DAG framing
to fully hold.

Next steps are bench measurement, not code: full bench under dry-run,
classifier review, then flip apply_caps and measure §10.8 deltas.
This commit is contained in:
russell@unturf.com 2026-05-03 08:08:03 -04:00
parent 5a60e8595f
commit c684dc17e1
No known key found for this signature in database

View file

@ -446,14 +446,83 @@ enhancement, not blocking.
## 8. Status
Phase 0 landed (bench harness extension, commit `2ffed00`,
2026-05-02). Phases 14 open · awaiting go/no-go per the §9 phased
plan **as refined by §10**. Bench-first per `docs/bench-maxing.md`
no Phase-2+ implementation lands without a measured delta over
baseline. Where §10 conflicts with §9, §10 wins (specifically:
three-form universal compilation §10.1, entropy framing §10.2,
new violation kinds vs new audit_mode §10.3, three-clause CTI
contract §10.4).
**All four phases landed 2026-05-03. Mechanism complete; defaults
preserve the dry-run discipline.** Bench-first measurement of
§10.8 decision-tree thresholds is the only remaining work.
Implementation commit chain (all on `origin/main`):
| Commit | Phase | Scope | Tests |
|-----------|-------|----------------------------------------------------------|-------|
| `2ffed00` | 0 | bench-harness extension (FC rate, violation kinds) | +5 |
| `5e8d662` | 0.x | bench-row telemetry (pointer/bracket/profile fields) | +5 |
| `926b05e` | 1 | pure quantifier classifier `aborist/qa/quantifier.py` | +61 |
| `84d5b5c` | 2 | model-profile caps `aborist/qa/model_profiles.py` | +19 |
| `6f90f21` | 3 | reminder injection `aborist/qa/quantifier_reminder.py` | +19 |
| `5a60e85` | 4 | CLI flags + 4 violation kinds + reject-broad early-return | +16 |
Total: **906 tests passing (120 new); 36 skipped.**
Defaults respected per §10.11.3 dry-run discipline:
```
quantifier_guard_enabled True # classifier emits telemetry
quantifier_guard_apply_caps False # cap reported, NOT applied
quantifier_reminder_enabled False # reminder mechanism off
quantifier_reject_broad False # reject-broad off
quantifier_guard_modes ["claim_lattice_pointer",
"claim_lattice"]
```
§10.11.2 disable hierarchy fully wired (six levels):
1. Per-test: `policy={"quantifier_guard_enabled": False}`
2. Per-call CLI: `--no-quantifier-guard`, `--allow-broad`,
`--reject-broad`, `--apply-quantifier-caps`
3. Per-phase policy: 7 fields (master, apply_caps, reminder_enabled,
reject_broad, caps_by_intensity, guard_modes, plus Phase 0
format_collapse_check_enabled)
4. Per-mode: `quantifier_guard_modes` (quote opts out by default)
5. Per-model: `aborist/qa/model_profiles.py` PROFILES dict
6. Master via governance: all 7 quantifier policy fields fold into
`_VERIFIER_POLICY_FIELDS` so flipping any of them invalidates
prior cache records on lookup
Live end-to-end verification (post-`5a60e85`):
```
$ aborist query --reject-broad "Winners of all major sports?"
Winners of all major sports?
UNGROUNDED · via BROAD_QUANTIFIER_REJECTED · ALL ("all")
· cap was 8 0/0 0.0s (preflight)
BROAD-QUANTIFIER PREFLIGHT REJECTED · scope unbounded
Question matched ALL intensity ("all") with an under-specified
universe. Narrow ... or run with --allow-broad for exploratory
enumeration.
$ aborist query --reject-broad "name all members of the Beatles"
name all members of the Beatles
UNGROUNDED · via claim_lattice · title mismatch 4/4 20.9s
[Beatles enumerated; scope_bound_hint=bounded → not rejected]
```
Next steps are **measurement, not code**:
- §10.11.3 step 2: full bench (134 questions × 3 modes × n=3) under
dry-run. Inspect per-question `quantifier_intensity` and
`claim_cap_applied`; catch mis-classifications.
- §10.11.3 step 3: fix any classifier drift. Re-run dry-run bench.
- §10.11.3 step 4: flip `quantifier_guard_apply_caps=True` on the
broad-quantifier subset. Measure §10.8 deltas (≥5pp gate per
mode for STRICT-rate / FORMAT_COLLAPSED rate / pointer-loss rate).
- §10.8 decision tree decides whether to flip
`quantifier_reminder_enabled` and `quantifier_reject_broad`
defaults, or keep them opt-in.
Where §10 conflicts with §9, §10 wins (specifically: three-form
universal compilation §10.1, entropy framing §10.2, new violation
kinds vs new audit_mode §10.3, three-clause CTI contract §10.4).
## 9. Architectural review (2026-05-03, Asia/Kuala_Lumpur)
@ -626,9 +695,9 @@ PROMETHEUS-Σ owns the mapping `model_profile → quantifier_intensity
### 9.6 Phase details
#### Phase 0 — Bench harness instrumentation (LANDED)
#### Phase 0 — Bench harness instrumentation (LANDED `2ffed00`)
Closed in commit `2ffed00`. The fields landed:
Three fields per bench row:
```
format_collapsed: bool
@ -636,30 +705,30 @@ violation_kinds: list[str]
answer_brackets: int
```
The §9 review additionally recommends these fields, **not yet
landed**, queued under Phase 0.x:
#### Phase 0.x — Bench-row telemetry expansion (LANDED `5e8d662`)
Eight additional fields covering pointer/bracket/profile/quantifier
slots:
```
raw_claim_line_count: int meaningful_lines from raw_answer
parsed_claim_count: int parser-extracted claim count (post-cap)
claim_cap_applied: int per-call cap that was active
quantifier_intensity: str classifier output (Phase 1+)
quantifier_matched_token: str classifier matched lexical surface
top_violation_kinds: list[str] dominant kinds for this row (top-3)
answer_pointer_count distinct E\d+ ids in raw_answer
answer_chars_with_brackets chars inside [E\d+,...] regions
raw_meaningful_line_count >20-char lines in raw_answer
quantifier_intensity classifier output (Phase 1)
quantifier_matched_token classifier matched lexical surface
scope_bound_hint bounded / unbounded / unknown
claim_cap_applied cap looked up (None until Phase 2)
model_profile_id configured model id verbatim
```
Phase 0.x lands these fields before Phase 2 measurement so
A/B/D bench cells carry full preflight provenance.
Helper `_bracket_diagnostics()` bundles the bracket/pointer/line
extraction; module-level regexes (`_BRACKET_RE`, `_BRACKET_REGION
_RE`, `_POINTER_ID_RE`) avoid per-row recompilation.
#### Phase 1 — Pure quantifier classifier
#### Phase 1 — Pure quantifier classifier (LANDED `926b05e`)
`aborist/qa/quantifier.py` with a single pure function:
```python
def classify_question_quantifier(question: str) -> QuantifierPreflight: ...
```
Returns:
`aborist/qa/quantifier.py` with `classify_question_quantifier()`,
pure function (no I/O, no model call, no retrieval call). Returns:
```json
{
@ -667,70 +736,151 @@ Returns:
"matched_token": "all",
"explicit_count": null,
"is_broad": true,
"operational_shape": "universal_enumeration"
"operational_shape": "universal_enumeration",
"scope_bound_hint": "unbounded",
"classifier_version": "quantifier-v0.1"
}
```
Pure: no I/O, no model call, no retrieval call. Ten-rung ladder
from §2 drives the classification but is not philosophy — it
exists to estimate **expected answer length and model-format
risk**, nothing more.
Wired into `query()` and `runner.ask()` after policy resolution;
both miss-path and cache-hit-path result dicts surface
`quantifier_intensity`, `quantifier_matched_token`,
`scope_bound_hint`, `quantifier_explicit_count` so bench rows stay
column-aligned.
#### Phase 2 — Model-profile claim-budget contract
Highest-intensity-wins arbitration: COMPREHENSIVE strictly stronger
than OPEN_REQUEST (both > ALL). `tell me everything about all wars`
→ COMPREHENSIVE rather than OPEN_REQUEST.
`aborist/qa/model_profiles.py` (new file), per §9.5 PROMETHEUS-Σ
table.
Scope-bound heuristic (§10.1): bounded universals (`all members of
the Beatles`, year-anchored questions) get `scope_bound_hint:
"bounded"`; broad+missing-anchor → `"unbounded"`; everything else
`"unknown"`. Heuristic only — corpus-arity check left for future
refinement.
Policy fields (additive, no schema bump):
#### Phase 2 — Model-profile claim-budget contract (LANDED `84d5b5c`)
```json
{
"quantifier_guard_enabled": true,
"quantifier_caps_by_intensity": {
"ABSENT": 1, "SINGULAR": 1, "PROPORTIONAL": 1,
"FEW": 5, "MANY": 8, "ALL": 8,
"COMPREHENSIVE": 5, "OPEN_REQUEST": 5
},
"quantifier_reminder_enabled": false,
"reject_broad": false
}
`aborist/qa/model_profiles.py` ships two profiles:
- `adamo1139/Hermes-3-Llama-3.1-8B-FP8-Dynamic` — tight caps for
broad intensities (ALL=8, COMPREHENSIVE=5, OPEN_REQUEST=5).
- `default` — large-reasoner-class fallback (ALL=12,
COMPREHENSIVE=15, OPEN_REQUEST=12). Used when `model_profile_id`
has no entry.
`cap_for_intensity()` resolves caps via three-source chain:
```
1. policy_overrides (per-call dict, highest priority)
2. per-model profile from PROFILES
3. "default" profile fallback
```
Folds into `governance_policy_hash` per the §1 hard constraint.
`EXPLICIT_COUNT` sentinel handles SMALL_NUM_EXPLICIT and
COMPARATIVE_BOUND — cap is the question's explicit count, not a
profile-set value. Defensive fallback to MANY cap if classifier
fired the rung without extracting a count.
#### Phase 3 — Prompt reminder (default OFF)
Four policy fields, all in `_VERIFIER_POLICY_FIELDS`:
Hermes already ignores parts of the existing reminder under
enumeration pressure; extra text may not change behavior. Land the
mechanism but keep `quantifier_reminder_enabled: false` until bench
measurement clears the 5pp signal floor.
```
quantifier_guard_enabled master kill (default True)
quantifier_guard_apply_caps dry-run gate (default False
per §10.11.3)
quantifier_caps_by_intensity per-call override (default {})
quantifier_guard_modes per-mode opt-in (default
["claim_lattice_pointer",
"claim_lattice"])
```
Bench sequence:
`effective_max_claims` computed in both `query()` and
`runner.ask()` from the (classifier_intensity, model_profile,
policy_overrides) triple. Dry-run mode keeps it at the policy
default (12); flipping `quantifier_guard_apply_caps=True` shadows
the default with the looked-up cap.
#### Phase 3 — Prompt reminder (LANDED `6f90f21`, default OFF)
`aborist/qa/quantifier_reminder.py` ships
`broad_quantifier_reminder()` with two templates:
- **Bounded universe** (`all members of the Beatles`):
`"This is a broad-quantifier query with a bounded universe.
Return at most N pointer-linked claim lines. Each claim must
cite an evidence id like [E5]; do not write claim lines without
bracket citations."`
- **Unbounded universe** (`winners of all major sports`):
`"This is a broad-quantifier query with an under-specified
scope. Return at most N pointer-linked claim lines. If you
cannot ground N claims with evidence IDs, return fewer grounded
claims. Do not enumerate from training prior. Each claim must
cite an evidence id like [E5]; do not write claim lines without
bracket citations."`
Inserted as a user-turn message between the existing
`grounding_reminder` and the evidence/question payload — same
position the standard reminder uses, where Hermes-3-8B's
most-recent-token attention catches it.
Default OFF (`quantifier_reminder_enabled: false`); operator opts
in per-call. Bench sequence still required:
```
baseline → cap only → reminder only → cap + reminder
```
Only flip the default if reminder-only or cap+reminder beats cap-only
beyond the floor.
beyond the §10.8 5pp floor.
#### Phase 4 — Optional reject-broad (default OFF)
#### Phase 4 — CLI flags + violation tails (LANDED `5a60e85`)
CLI flags:
CLI flags on `aborist query`:
```
--reject-broad reject ALL/COMPREHENSIVE/OPEN_REQUEST before LLM call
--allow-broad bypass all reductions for emergent search
--no-quantifier-guard Level 2 disable: kills guard for one call
--allow-broad emergent search: classifier on, caps off
--reject-broad strict reject: ALL/COMPREHENSIVE/
OPEN_REQUEST + scope_bound_hint=="unbounded"
returns UNGROUNDED before LLM call.
Bounded universals are NOT rejected.
--apply-quantifier-caps flip dry-run gate per-call (path from
dry-run to live cap)
```
Outputs:
Three new soft-demote violation kinds (§10.3) — no new audit_mode
token; each renders a tail on the audit-line:
```
--reject-broad fires UNGROUNDED · BROAD_QUANTIFIER_REJECTED
"Narrow by sport, league, year, country, or
use --allow-broad for exploratory enumeration."
--allow-broad fires EXPLORATORY · unbounded broad query · not
evidence-warranted
BROAD_QUANTIFIER_RUNAWAY "broad runaway" (raw_lines >> brackets)
BROAD_QUANTIFIER_CAP_APPLIED "broad cap N" (cap interpolated)
BROAD_QUANTIFIER_SCOPE_UNBOUND "broad unbounded"
```
Plus one HARD demote (early-return UNGROUNDED):
```
BROAD_QUANTIFIER_REJECTED "broad rejected" (preflight rejection)
```
Reject-broad early-return path in `query()` returns UNGROUNDED
before the LLM call when policy enables `quantifier_reject_broad`
AND the question is broad-unbounded. Result schema mirrors a
normal UNGROUNDED row (answer_text carries rationale + narrowing
hints). `_render_query_human` has a dedicated branch for the new
status so operators see the rejection without `--json`.
Outputs (live-verified):
```
--reject-broad fires:
UNGROUNDED · via BROAD_QUANTIFIER_REJECTED · ALL ("all")
· cap was 8 0/0 0.0s (preflight)
BROAD-QUANTIFIER PREFLIGHT REJECTED · scope unbounded
[actionable narrowing hints]
--allow-broad fires:
EXPLORATORY · unbounded broad query · not
evidence-warranted
```
Default behavior: cap reduction (Phase 2) only. No reminder. No
@ -758,50 +908,93 @@ harness should track BROAD-CAPPED runs separately from
EVIDENCE-WARRANTED so the operator-visible verdict matches what
the system actually committed.
### 9.8 Test surface (in addition to §5 bench plan)
### 9.8 Test surface — pinned 2026-05-03
Classifier (Phase 1):
All check boxes below are now [x] — landed in commits per §8 status
table. Bench-side measurement (§10.8 decision-tree thresholds)
remains open and orthogonal to the test pinning.
Classifier (Phase 1, `tests/test_quantifier_classifier.py` — 61 tests):
```
[ ] "Winners of all major sports?" → ALL
[ ] "complete list of winners..." → COMPREHENSIVE
[ ] "tell me about major sports winners" → OPEN_REQUEST
[ ] "top 3 winners" → SMALL_NUM_EXPLICIT, cap 3
[ ] "five biggest cities" → SMALL_NUM_EXPLICIT, cap 5
[ ] "most winners were..." → PROPORTIONAL
[ ] "who won the Super Bowl?" → SINGULAR
[ ] "which teams did not win..." → ABSENT / polarity flag
[x] "Winners of all major sports?" → ALL
[x] "complete list of winners..." → COMPREHENSIVE
[x] "tell me about major sports winners" → OPEN_REQUEST
[x] "top 3 winners" → SMALL_NUM_EXPLICIT, cap 3
[x] "five biggest cities" → SMALL_NUM_EXPLICIT, cap 5
[x] "most winners were..." → PROPORTIONAL
[x] "who won the Super Bowl?" → SINGULAR
[x] "which teams did not win..." → ABSENT
```
Policy (Phase 2):
Policy + governance (Phase 2, `tests/test_quantifier_caps.py` — 19 tests):
```
[ ] Hermes + ALL → cap 8.
[ ] large model + ALL → cap 12.
[ ] --allow-broad bypasses reduction.
[ ] --reject-broad returns before LLM.
[ ] quantifier policy changes governance_policy_hash.
[ ] schema/canonicalization/chunking versions unchanged.
[x] Hermes + ALL → cap 8.
[x] large model (default profile) + ALL → cap 12.
[x] policy_overrides beats per-model profile.
[x] EXPLICIT_COUNT sentinel resolves from question count.
[x] All four Phase-2 fields in _VERIFIER_POLICY_FIELDS.
[x] governance_policy_hash bumps on every cap-related field flip.
[x] All ten intensity rungs covered by every profile.
```
Bench:
Reminder (Phase 3, `tests/test_quantifier_reminder.py` — 19 tests):
```
[ ] bench JSONL contains format_collapsed. (LANDED)
[ ] bench JSONL contains violation kinds. (LANDED)
[ ] bench JSONL contains bracket count. (LANDED)
[ ] bench JSONL contains quantifier intensity. (Phase 0.x)
[ ] bench summary has quantifier guard section. (Phase 0.x)
[x] Non-broad intensity → no reminder (None return).
[x] Bounded universe → "bounded universe" template, no
"training prior" clause.
[x] Unbounded universe → stricter template with no-prior clause.
[x] Unknown scope → falls back to unbounded template.
[x] Cap N appears verbatim in reminder.
[x] [E\d+] citation rule restated.
[x] quantifier_reminder_enabled in _VERIFIER_POLICY_FIELDS.
[x] governance_policy_hash bumps on flip.
```
Phase 4 + violation tails (`tests/test_quantifier_phase4.py` — 16 tests):
```
[x] BROAD_QUANTIFIER_RUNAWAY/CAP_APPLIED/SCOPE_UNBOUND in soft-demote set.
[x] BROAD_QUANTIFIER_REJECTED NOT in soft-demote set (it's hard).
[x] All three soft kinds → ANCHOR-WARRANTED rung.
[x] Tail rendering: "broad cap N" interpolates the cap value.
[x] Tail combines with existing kinds (title mismatch + broad cap).
[x] End-to-end render through _render_query_human.
[x] quantifier_reject_broad in _VERIFIER_POLICY_FIELDS.
[x] governance_policy_hash bumps on flip.
[ ] (skipped) integration: --reject-broad on broad-unbounded
returns UNGROUNDED before LLM call.
[ ] (skipped) integration: --reject-broad on bounded-universal
does NOT reject.
```
The two skipped integration tests are exercised by the live bench
cycle (and by the manual smoke tests recorded in §8 Status).
Bench harness:
```
[x] bench JSONL contains format_collapsed.
[x] bench JSONL contains violation_kinds.
[x] bench JSONL contains answer_brackets / pointer_count /
chars_with_brackets / raw_meaningful_line_count.
[x] bench JSONL contains quantifier_intensity / matched_token /
scope_bound_hint / claim_cap_applied / model_profile_id.
[x] bench summary has format-collapse + violation-kinds section.
```
No-regression:
```
[ ] FORMAT_COLLAPSED remains downstream catch.
[ ] verifier binary logic unchanged.
[ ] existing pointer-mode checks still pass.
[ ] no LLM verifier introduced.
[ ] claim cap is per-call policy, not schema mutation.
[x] FORMAT_COLLAPSED remains downstream catch (untouched).
[x] verifier binary logic unchanged (cap is a parameter, not a rule).
[x] existing pointer-mode checks still pass (full suite green).
[x] no LLM verifier introduced (preflight is pure regex).
[x] claim cap is per-call policy, not schema mutation
(no schema_version / canonicalization_version / chunking_version
bump anywhere in commits 5e8d662…5a60e85).
```
### 9.9 Seven-point program mapping
@ -849,18 +1042,31 @@ model profile = capability prior
governance hash = audit binding
```
### 9.11 Implementation commit sequence
### 9.11 Implementation commit sequence — actual
```
Commit 1 — Bench observability (LANDED 2ffed00)
Commit 2 — Quantifier classifier (Phase 1)
Commit 3 — Model-profile caps (Phase 2)
Commit 4 — Runner wiring + per-call cap (Phase 2)
Commit 5 — CLI flags --allow-broad / --reject-broad
Commit 6 — Optional reminder experiment (Phase 3, off-by-default)
Commit 7 — DAG/audit binding for quantifier_preflight node
2ffed00 Phase 0 bench-harness extension (FC rate, violation kinds)
5e8d662 Phase 0.x bench-row telemetry (pointer/bracket/profile)
926b05e Phase 1 pure quantifier classifier (dry-run wired)
84d5b5c Phase 2 model-profile caps + governance hash + dry-run
6f90f21 Phase 3 reminder injection mechanism (default off)
5a60e85 Phase 4 CLI flags + violation-kind tails + reject-broad
```
Phase-2 wiring (originally split between commits 3 and 4 in the
plan) landed in one commit (`84d5b5c`) — the cap-table data
structures and the runner/query wiring share enough machinery
that splitting them would have churned the diff without aiding
review.
DAG/audit binding for the `quantifier_preflight` node (originally
"commit 7" in the plan) is **not yet shipped**. Quantifier output
surfaces on the result dict and bench rows, but no `quantifier
_preflight` node is added to `run_dag`. Tracked as a Phase 5
(or §11 amend) follow-up — the §10.4 three-clause CTI contract
needs a run_dag node to be auditable post-hoc. Optional now;
required before the §9.5 Merkle-AGI-DAG framing fully holds.
### 9.12 Final position
> Address broad-quantifier collapse by turning quantifier intensity