ticket(#000008): mark Phases 0.x-4 landed; commit chain pinned
Updates §8 Status, §9.6 Phase details, §9.8 Test surface, §9.11 commit sequence to reflect actual implementation state: §8 Status — Phase 0 through Phase 4 all landed 2026-05-03 across six commits (2ffed00→5a60e85). 906 tests passing (120 new). Defaults preserve §10.11.3 dry-run discipline: quantifier_guard_apply_caps=False, reminder=False, reject=False. Six-level disable hierarchy fully wired. Live verification recorded for both reject-broad-fires (Winners of all major sports?) and reject-broad-skips (name all members of the Beatles → bounded universal, NOT rejected). §9.6 Phase details — each phase tagged LANDED <sha>; description trimmed to what actually shipped vs the original proposal. §9.8 Test surface — every check box flipped to [x] with the matching test file path and test count. The two reject-broad integration tests remain skipped (exercised by live bench). §9.11 Implementation commit sequence — actual SHAs replace planned commit numbers. Notes that DAG/audit binding for quantifier_preflight node ("commit 7" in plan) is NOT shipped — tracked as Phase 5 follow-up. Optional now; required for §9.5 Merkle-AGI-DAG framing to fully hold. Next steps are bench measurement, not code: full bench under dry-run, classifier review, then flip apply_caps and measure §10.8 deltas.
This commit is contained in:
parent
5a60e8595f
commit
c684dc17e1
1 changed files with 308 additions and 102 deletions
|
|
@ -446,14 +446,83 @@ enhancement, not blocking.
|
|||
|
||||
## 8. Status
|
||||
|
||||
Phase 0 landed (bench harness extension, commit `2ffed00`,
|
||||
2026-05-02). Phases 1–4 open · awaiting go/no-go per the §9 phased
|
||||
plan **as refined by §10**. Bench-first per `docs/bench-maxing.md` —
|
||||
no Phase-2+ implementation lands without a measured delta over
|
||||
baseline. Where §10 conflicts with §9, §10 wins (specifically:
|
||||
three-form universal compilation §10.1, entropy framing §10.2,
|
||||
new violation kinds vs new audit_mode §10.3, three-clause CTI
|
||||
contract §10.4).
|
||||
**All four phases landed 2026-05-03. Mechanism complete; defaults
|
||||
preserve the dry-run discipline.** Bench-first measurement of
|
||||
§10.8 decision-tree thresholds is the only remaining work.
|
||||
|
||||
Implementation commit chain (all on `origin/main`):
|
||||
|
||||
| Commit | Phase | Scope | Tests |
|
||||
|-----------|-------|----------------------------------------------------------|-------|
|
||||
| `2ffed00` | 0 | bench-harness extension (FC rate, violation kinds) | +5 |
|
||||
| `5e8d662` | 0.x | bench-row telemetry (pointer/bracket/profile fields) | +5 |
|
||||
| `926b05e` | 1 | pure quantifier classifier `aborist/qa/quantifier.py` | +61 |
|
||||
| `84d5b5c` | 2 | model-profile caps `aborist/qa/model_profiles.py` | +19 |
|
||||
| `6f90f21` | 3 | reminder injection `aborist/qa/quantifier_reminder.py` | +19 |
|
||||
| `5a60e85` | 4 | CLI flags + 4 violation kinds + reject-broad early-return | +16 |
|
||||
|
||||
Total: **906 tests passing (120 new); 36 skipped.**
|
||||
|
||||
Defaults respected per §10.11.3 dry-run discipline:
|
||||
|
||||
```
|
||||
quantifier_guard_enabled True # classifier emits telemetry
|
||||
quantifier_guard_apply_caps False # cap reported, NOT applied
|
||||
quantifier_reminder_enabled False # reminder mechanism off
|
||||
quantifier_reject_broad False # reject-broad off
|
||||
quantifier_guard_modes ["claim_lattice_pointer",
|
||||
"claim_lattice"]
|
||||
```
|
||||
|
||||
§10.11.2 disable hierarchy fully wired (six levels):
|
||||
|
||||
1. Per-test: `policy={"quantifier_guard_enabled": False}`
|
||||
2. Per-call CLI: `--no-quantifier-guard`, `--allow-broad`,
|
||||
`--reject-broad`, `--apply-quantifier-caps`
|
||||
3. Per-phase policy: 7 fields (master, apply_caps, reminder_enabled,
|
||||
reject_broad, caps_by_intensity, guard_modes, plus Phase 0
|
||||
format_collapse_check_enabled)
|
||||
4. Per-mode: `quantifier_guard_modes` (quote opts out by default)
|
||||
5. Per-model: `aborist/qa/model_profiles.py` PROFILES dict
|
||||
6. Master via governance: all 7 quantifier policy fields fold into
|
||||
`_VERIFIER_POLICY_FIELDS` so flipping any of them invalidates
|
||||
prior cache records on lookup
|
||||
|
||||
Live end-to-end verification (post-`5a60e85`):
|
||||
|
||||
```
|
||||
$ aborist query --reject-broad "Winners of all major sports?"
|
||||
Winners of all major sports?
|
||||
UNGROUNDED · via BROAD_QUANTIFIER_REJECTED · ALL ("all")
|
||||
· cap was 8 0/0 0.0s (preflight)
|
||||
|
||||
BROAD-QUANTIFIER PREFLIGHT REJECTED · scope unbounded
|
||||
Question matched ALL intensity ("all") with an under-specified
|
||||
universe. Narrow ... or run with --allow-broad for exploratory
|
||||
enumeration.
|
||||
|
||||
$ aborist query --reject-broad "name all members of the Beatles"
|
||||
name all members of the Beatles
|
||||
UNGROUNDED · via claim_lattice · title mismatch 4/4 20.9s
|
||||
[Beatles enumerated; scope_bound_hint=bounded → not rejected]
|
||||
```
|
||||
|
||||
Next steps are **measurement, not code**:
|
||||
|
||||
- §10.11.3 step 2: full bench (134 questions × 3 modes × n=3) under
|
||||
dry-run. Inspect per-question `quantifier_intensity` and
|
||||
`claim_cap_applied`; catch mis-classifications.
|
||||
- §10.11.3 step 3: fix any classifier drift. Re-run dry-run bench.
|
||||
- §10.11.3 step 4: flip `quantifier_guard_apply_caps=True` on the
|
||||
broad-quantifier subset. Measure §10.8 deltas (≥5pp gate per
|
||||
mode for STRICT-rate / FORMAT_COLLAPSED rate / pointer-loss rate).
|
||||
- §10.8 decision tree decides whether to flip
|
||||
`quantifier_reminder_enabled` and `quantifier_reject_broad`
|
||||
defaults, or keep them opt-in.
|
||||
|
||||
Where §10 conflicts with §9, §10 wins (specifically: three-form
|
||||
universal compilation §10.1, entropy framing §10.2, new violation
|
||||
kinds vs new audit_mode §10.3, three-clause CTI contract §10.4).
|
||||
|
||||
## 9. Architectural review (2026-05-03, Asia/Kuala_Lumpur)
|
||||
|
||||
|
|
@ -626,9 +695,9 @@ PROMETHEUS-Σ owns the mapping `model_profile → quantifier_intensity
|
|||
|
||||
### 9.6 Phase details
|
||||
|
||||
#### Phase 0 — Bench harness instrumentation (LANDED)
|
||||
#### Phase 0 — Bench harness instrumentation (LANDED `2ffed00`)
|
||||
|
||||
Closed in commit `2ffed00`. The fields landed:
|
||||
Three fields per bench row:
|
||||
|
||||
```
|
||||
format_collapsed: bool
|
||||
|
|
@ -636,30 +705,30 @@ violation_kinds: list[str]
|
|||
answer_brackets: int
|
||||
```
|
||||
|
||||
The §9 review additionally recommends these fields, **not yet
|
||||
landed**, queued under Phase 0.x:
|
||||
#### Phase 0.x — Bench-row telemetry expansion (LANDED `5e8d662`)
|
||||
|
||||
Eight additional fields covering pointer/bracket/profile/quantifier
|
||||
slots:
|
||||
|
||||
```
|
||||
raw_claim_line_count: int meaningful_lines from raw_answer
|
||||
parsed_claim_count: int parser-extracted claim count (post-cap)
|
||||
claim_cap_applied: int per-call cap that was active
|
||||
quantifier_intensity: str classifier output (Phase 1+)
|
||||
quantifier_matched_token: str classifier matched lexical surface
|
||||
top_violation_kinds: list[str] dominant kinds for this row (top-3)
|
||||
answer_pointer_count distinct E\d+ ids in raw_answer
|
||||
answer_chars_with_brackets chars inside [E\d+,...] regions
|
||||
raw_meaningful_line_count >20-char lines in raw_answer
|
||||
quantifier_intensity classifier output (Phase 1)
|
||||
quantifier_matched_token classifier matched lexical surface
|
||||
scope_bound_hint bounded / unbounded / unknown
|
||||
claim_cap_applied cap looked up (None until Phase 2)
|
||||
model_profile_id configured model id verbatim
|
||||
```
|
||||
|
||||
Phase 0.x lands these fields before Phase 2 measurement so
|
||||
A/B/D bench cells carry full preflight provenance.
|
||||
Helper `_bracket_diagnostics()` bundles the bracket/pointer/line
|
||||
extraction; module-level regexes (`_BRACKET_RE`, `_BRACKET_REGION
|
||||
_RE`, `_POINTER_ID_RE`) avoid per-row recompilation.
|
||||
|
||||
#### Phase 1 — Pure quantifier classifier
|
||||
#### Phase 1 — Pure quantifier classifier (LANDED `926b05e`)
|
||||
|
||||
`aborist/qa/quantifier.py` with a single pure function:
|
||||
|
||||
```python
|
||||
def classify_question_quantifier(question: str) -> QuantifierPreflight: ...
|
||||
```
|
||||
|
||||
Returns:
|
||||
`aborist/qa/quantifier.py` with `classify_question_quantifier()`,
|
||||
pure function (no I/O, no model call, no retrieval call). Returns:
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
@ -667,70 +736,151 @@ Returns:
|
|||
"matched_token": "all",
|
||||
"explicit_count": null,
|
||||
"is_broad": true,
|
||||
"operational_shape": "universal_enumeration"
|
||||
"operational_shape": "universal_enumeration",
|
||||
"scope_bound_hint": "unbounded",
|
||||
"classifier_version": "quantifier-v0.1"
|
||||
}
|
||||
```
|
||||
|
||||
Pure: no I/O, no model call, no retrieval call. Ten-rung ladder
|
||||
from §2 drives the classification but is not philosophy — it
|
||||
exists to estimate **expected answer length and model-format
|
||||
risk**, nothing more.
|
||||
Wired into `query()` and `runner.ask()` after policy resolution;
|
||||
both miss-path and cache-hit-path result dicts surface
|
||||
`quantifier_intensity`, `quantifier_matched_token`,
|
||||
`scope_bound_hint`, `quantifier_explicit_count` so bench rows stay
|
||||
column-aligned.
|
||||
|
||||
#### Phase 2 — Model-profile claim-budget contract
|
||||
Highest-intensity-wins arbitration: COMPREHENSIVE strictly stronger
|
||||
than OPEN_REQUEST (both > ALL). `tell me everything about all wars`
|
||||
→ COMPREHENSIVE rather than OPEN_REQUEST.
|
||||
|
||||
`aborist/qa/model_profiles.py` (new file), per §9.5 PROMETHEUS-Σ
|
||||
table.
|
||||
Scope-bound heuristic (§10.1): bounded universals (`all members of
|
||||
the Beatles`, year-anchored questions) get `scope_bound_hint:
|
||||
"bounded"`; broad+missing-anchor → `"unbounded"`; everything else
|
||||
→ `"unknown"`. Heuristic only — corpus-arity check left for future
|
||||
refinement.
|
||||
|
||||
Policy fields (additive, no schema bump):
|
||||
#### Phase 2 — Model-profile claim-budget contract (LANDED `84d5b5c`)
|
||||
|
||||
```json
|
||||
{
|
||||
"quantifier_guard_enabled": true,
|
||||
"quantifier_caps_by_intensity": {
|
||||
"ABSENT": 1, "SINGULAR": 1, "PROPORTIONAL": 1,
|
||||
"FEW": 5, "MANY": 8, "ALL": 8,
|
||||
"COMPREHENSIVE": 5, "OPEN_REQUEST": 5
|
||||
},
|
||||
"quantifier_reminder_enabled": false,
|
||||
"reject_broad": false
|
||||
}
|
||||
`aborist/qa/model_profiles.py` ships two profiles:
|
||||
|
||||
- `adamo1139/Hermes-3-Llama-3.1-8B-FP8-Dynamic` — tight caps for
|
||||
broad intensities (ALL=8, COMPREHENSIVE=5, OPEN_REQUEST=5).
|
||||
- `default` — large-reasoner-class fallback (ALL=12,
|
||||
COMPREHENSIVE=15, OPEN_REQUEST=12). Used when `model_profile_id`
|
||||
has no entry.
|
||||
|
||||
`cap_for_intensity()` resolves caps via three-source chain:
|
||||
|
||||
```
|
||||
1. policy_overrides (per-call dict, highest priority)
|
||||
2. per-model profile from PROFILES
|
||||
3. "default" profile fallback
|
||||
```
|
||||
|
||||
Folds into `governance_policy_hash` per the §1 hard constraint.
|
||||
`EXPLICIT_COUNT` sentinel handles SMALL_NUM_EXPLICIT and
|
||||
COMPARATIVE_BOUND — cap is the question's explicit count, not a
|
||||
profile-set value. Defensive fallback to MANY cap if classifier
|
||||
fired the rung without extracting a count.
|
||||
|
||||
#### Phase 3 — Prompt reminder (default OFF)
|
||||
Four policy fields, all in `_VERIFIER_POLICY_FIELDS`:
|
||||
|
||||
Hermes already ignores parts of the existing reminder under
|
||||
enumeration pressure; extra text may not change behavior. Land the
|
||||
mechanism but keep `quantifier_reminder_enabled: false` until bench
|
||||
measurement clears the 5pp signal floor.
|
||||
```
|
||||
quantifier_guard_enabled master kill (default True)
|
||||
quantifier_guard_apply_caps dry-run gate (default False
|
||||
per §10.11.3)
|
||||
quantifier_caps_by_intensity per-call override (default {})
|
||||
quantifier_guard_modes per-mode opt-in (default
|
||||
["claim_lattice_pointer",
|
||||
"claim_lattice"])
|
||||
```
|
||||
|
||||
Bench sequence:
|
||||
`effective_max_claims` computed in both `query()` and
|
||||
`runner.ask()` from the (classifier_intensity, model_profile,
|
||||
policy_overrides) triple. Dry-run mode keeps it at the policy
|
||||
default (12); flipping `quantifier_guard_apply_caps=True` shadows
|
||||
the default with the looked-up cap.
|
||||
|
||||
#### Phase 3 — Prompt reminder (LANDED `6f90f21`, default OFF)
|
||||
|
||||
`aborist/qa/quantifier_reminder.py` ships
|
||||
`broad_quantifier_reminder()` with two templates:
|
||||
|
||||
- **Bounded universe** (`all members of the Beatles`):
|
||||
`"This is a broad-quantifier query with a bounded universe.
|
||||
Return at most N pointer-linked claim lines. Each claim must
|
||||
cite an evidence id like [E5]; do not write claim lines without
|
||||
bracket citations."`
|
||||
- **Unbounded universe** (`winners of all major sports`):
|
||||
`"This is a broad-quantifier query with an under-specified
|
||||
scope. Return at most N pointer-linked claim lines. If you
|
||||
cannot ground N claims with evidence IDs, return fewer grounded
|
||||
claims. Do not enumerate from training prior. Each claim must
|
||||
cite an evidence id like [E5]; do not write claim lines without
|
||||
bracket citations."`
|
||||
|
||||
Inserted as a user-turn message between the existing
|
||||
`grounding_reminder` and the evidence/question payload — same
|
||||
position the standard reminder uses, where Hermes-3-8B's
|
||||
most-recent-token attention catches it.
|
||||
|
||||
Default OFF (`quantifier_reminder_enabled: false`); operator opts
|
||||
in per-call. Bench sequence still required:
|
||||
|
||||
```
|
||||
baseline → cap only → reminder only → cap + reminder
|
||||
```
|
||||
|
||||
Only flip the default if reminder-only or cap+reminder beats cap-only
|
||||
beyond the floor.
|
||||
beyond the §10.8 5pp floor.
|
||||
|
||||
#### Phase 4 — Optional reject-broad (default OFF)
|
||||
#### Phase 4 — CLI flags + violation tails (LANDED `5a60e85`)
|
||||
|
||||
CLI flags:
|
||||
CLI flags on `aborist query`:
|
||||
|
||||
```
|
||||
--reject-broad reject ALL/COMPREHENSIVE/OPEN_REQUEST before LLM call
|
||||
--allow-broad bypass all reductions for emergent search
|
||||
--no-quantifier-guard Level 2 disable: kills guard for one call
|
||||
--allow-broad emergent search: classifier on, caps off
|
||||
--reject-broad strict reject: ALL/COMPREHENSIVE/
|
||||
OPEN_REQUEST + scope_bound_hint=="unbounded"
|
||||
returns UNGROUNDED before LLM call.
|
||||
Bounded universals are NOT rejected.
|
||||
--apply-quantifier-caps flip dry-run gate per-call (path from
|
||||
dry-run to live cap)
|
||||
```
|
||||
|
||||
Outputs:
|
||||
Three new soft-demote violation kinds (§10.3) — no new audit_mode
|
||||
token; each renders a tail on the audit-line:
|
||||
|
||||
```
|
||||
--reject-broad fires UNGROUNDED · BROAD_QUANTIFIER_REJECTED
|
||||
"Narrow by sport, league, year, country, or
|
||||
use --allow-broad for exploratory enumeration."
|
||||
--allow-broad fires EXPLORATORY · unbounded broad query · not
|
||||
evidence-warranted
|
||||
BROAD_QUANTIFIER_RUNAWAY "broad runaway" (raw_lines >> brackets)
|
||||
BROAD_QUANTIFIER_CAP_APPLIED "broad cap N" (cap interpolated)
|
||||
BROAD_QUANTIFIER_SCOPE_UNBOUND "broad unbounded"
|
||||
```
|
||||
|
||||
Plus one HARD demote (early-return UNGROUNDED):
|
||||
|
||||
```
|
||||
BROAD_QUANTIFIER_REJECTED "broad rejected" (preflight rejection)
|
||||
```
|
||||
|
||||
Reject-broad early-return path in `query()` returns UNGROUNDED
|
||||
before the LLM call when policy enables `quantifier_reject_broad`
|
||||
AND the question is broad-unbounded. Result schema mirrors a
|
||||
normal UNGROUNDED row (answer_text carries rationale + narrowing
|
||||
hints). `_render_query_human` has a dedicated branch for the new
|
||||
status so operators see the rejection without `--json`.
|
||||
|
||||
Outputs (live-verified):
|
||||
|
||||
```
|
||||
--reject-broad fires:
|
||||
UNGROUNDED · via BROAD_QUANTIFIER_REJECTED · ALL ("all")
|
||||
· cap was 8 0/0 0.0s (preflight)
|
||||
BROAD-QUANTIFIER PREFLIGHT REJECTED · scope unbounded
|
||||
[actionable narrowing hints]
|
||||
|
||||
--allow-broad fires:
|
||||
EXPLORATORY · unbounded broad query · not
|
||||
evidence-warranted
|
||||
```
|
||||
|
||||
Default behavior: cap reduction (Phase 2) only. No reminder. No
|
||||
|
|
@ -758,50 +908,93 @@ harness should track BROAD-CAPPED runs separately from
|
|||
EVIDENCE-WARRANTED so the operator-visible verdict matches what
|
||||
the system actually committed.
|
||||
|
||||
### 9.8 Test surface (in addition to §5 bench plan)
|
||||
### 9.8 Test surface — pinned 2026-05-03
|
||||
|
||||
Classifier (Phase 1):
|
||||
All check boxes below are now [x] — landed in commits per §8 status
|
||||
table. Bench-side measurement (§10.8 decision-tree thresholds)
|
||||
remains open and orthogonal to the test pinning.
|
||||
|
||||
Classifier (Phase 1, `tests/test_quantifier_classifier.py` — 61 tests):
|
||||
|
||||
```
|
||||
[ ] "Winners of all major sports?" → ALL
|
||||
[ ] "complete list of winners..." → COMPREHENSIVE
|
||||
[ ] "tell me about major sports winners" → OPEN_REQUEST
|
||||
[ ] "top 3 winners" → SMALL_NUM_EXPLICIT, cap 3
|
||||
[ ] "five biggest cities" → SMALL_NUM_EXPLICIT, cap 5
|
||||
[ ] "most winners were..." → PROPORTIONAL
|
||||
[ ] "who won the Super Bowl?" → SINGULAR
|
||||
[ ] "which teams did not win..." → ABSENT / polarity flag
|
||||
[x] "Winners of all major sports?" → ALL
|
||||
[x] "complete list of winners..." → COMPREHENSIVE
|
||||
[x] "tell me about major sports winners" → OPEN_REQUEST
|
||||
[x] "top 3 winners" → SMALL_NUM_EXPLICIT, cap 3
|
||||
[x] "five biggest cities" → SMALL_NUM_EXPLICIT, cap 5
|
||||
[x] "most winners were..." → PROPORTIONAL
|
||||
[x] "who won the Super Bowl?" → SINGULAR
|
||||
[x] "which teams did not win..." → ABSENT
|
||||
```
|
||||
|
||||
Policy (Phase 2):
|
||||
Policy + governance (Phase 2, `tests/test_quantifier_caps.py` — 19 tests):
|
||||
|
||||
```
|
||||
[ ] Hermes + ALL → cap 8.
|
||||
[ ] large model + ALL → cap 12.
|
||||
[ ] --allow-broad bypasses reduction.
|
||||
[ ] --reject-broad returns before LLM.
|
||||
[ ] quantifier policy changes governance_policy_hash.
|
||||
[ ] schema/canonicalization/chunking versions unchanged.
|
||||
[x] Hermes + ALL → cap 8.
|
||||
[x] large model (default profile) + ALL → cap 12.
|
||||
[x] policy_overrides beats per-model profile.
|
||||
[x] EXPLICIT_COUNT sentinel resolves from question count.
|
||||
[x] All four Phase-2 fields in _VERIFIER_POLICY_FIELDS.
|
||||
[x] governance_policy_hash bumps on every cap-related field flip.
|
||||
[x] All ten intensity rungs covered by every profile.
|
||||
```
|
||||
|
||||
Bench:
|
||||
Reminder (Phase 3, `tests/test_quantifier_reminder.py` — 19 tests):
|
||||
|
||||
```
|
||||
[ ] bench JSONL contains format_collapsed. (LANDED)
|
||||
[ ] bench JSONL contains violation kinds. (LANDED)
|
||||
[ ] bench JSONL contains bracket count. (LANDED)
|
||||
[ ] bench JSONL contains quantifier intensity. (Phase 0.x)
|
||||
[ ] bench summary has quantifier guard section. (Phase 0.x)
|
||||
[x] Non-broad intensity → no reminder (None return).
|
||||
[x] Bounded universe → "bounded universe" template, no
|
||||
"training prior" clause.
|
||||
[x] Unbounded universe → stricter template with no-prior clause.
|
||||
[x] Unknown scope → falls back to unbounded template.
|
||||
[x] Cap N appears verbatim in reminder.
|
||||
[x] [E\d+] citation rule restated.
|
||||
[x] quantifier_reminder_enabled in _VERIFIER_POLICY_FIELDS.
|
||||
[x] governance_policy_hash bumps on flip.
|
||||
```
|
||||
|
||||
Phase 4 + violation tails (`tests/test_quantifier_phase4.py` — 16 tests):
|
||||
|
||||
```
|
||||
[x] BROAD_QUANTIFIER_RUNAWAY/CAP_APPLIED/SCOPE_UNBOUND in soft-demote set.
|
||||
[x] BROAD_QUANTIFIER_REJECTED NOT in soft-demote set (it's hard).
|
||||
[x] All three soft kinds → ANCHOR-WARRANTED rung.
|
||||
[x] Tail rendering: "broad cap N" interpolates the cap value.
|
||||
[x] Tail combines with existing kinds (title mismatch + broad cap).
|
||||
[x] End-to-end render through _render_query_human.
|
||||
[x] quantifier_reject_broad in _VERIFIER_POLICY_FIELDS.
|
||||
[x] governance_policy_hash bumps on flip.
|
||||
[ ] (skipped) integration: --reject-broad on broad-unbounded
|
||||
returns UNGROUNDED before LLM call.
|
||||
[ ] (skipped) integration: --reject-broad on bounded-universal
|
||||
does NOT reject.
|
||||
```
|
||||
|
||||
The two skipped integration tests are exercised by the live bench
|
||||
cycle (and by the manual smoke tests recorded in §8 Status).
|
||||
|
||||
Bench harness:
|
||||
|
||||
```
|
||||
[x] bench JSONL contains format_collapsed.
|
||||
[x] bench JSONL contains violation_kinds.
|
||||
[x] bench JSONL contains answer_brackets / pointer_count /
|
||||
chars_with_brackets / raw_meaningful_line_count.
|
||||
[x] bench JSONL contains quantifier_intensity / matched_token /
|
||||
scope_bound_hint / claim_cap_applied / model_profile_id.
|
||||
[x] bench summary has format-collapse + violation-kinds section.
|
||||
```
|
||||
|
||||
No-regression:
|
||||
|
||||
```
|
||||
[ ] FORMAT_COLLAPSED remains downstream catch.
|
||||
[ ] verifier binary logic unchanged.
|
||||
[ ] existing pointer-mode checks still pass.
|
||||
[ ] no LLM verifier introduced.
|
||||
[ ] claim cap is per-call policy, not schema mutation.
|
||||
[x] FORMAT_COLLAPSED remains downstream catch (untouched).
|
||||
[x] verifier binary logic unchanged (cap is a parameter, not a rule).
|
||||
[x] existing pointer-mode checks still pass (full suite green).
|
||||
[x] no LLM verifier introduced (preflight is pure regex).
|
||||
[x] claim cap is per-call policy, not schema mutation
|
||||
(no schema_version / canonicalization_version / chunking_version
|
||||
bump anywhere in commits 5e8d662…5a60e85).
|
||||
```
|
||||
|
||||
### 9.9 Seven-point program mapping
|
||||
|
|
@ -849,18 +1042,31 @@ model profile = capability prior
|
|||
governance hash = audit binding
|
||||
```
|
||||
|
||||
### 9.11 Implementation commit sequence
|
||||
### 9.11 Implementation commit sequence — actual
|
||||
|
||||
```
|
||||
Commit 1 — Bench observability (LANDED 2ffed00)
|
||||
Commit 2 — Quantifier classifier (Phase 1)
|
||||
Commit 3 — Model-profile caps (Phase 2)
|
||||
Commit 4 — Runner wiring + per-call cap (Phase 2)
|
||||
Commit 5 — CLI flags --allow-broad / --reject-broad
|
||||
Commit 6 — Optional reminder experiment (Phase 3, off-by-default)
|
||||
Commit 7 — DAG/audit binding for quantifier_preflight node
|
||||
2ffed00 Phase 0 bench-harness extension (FC rate, violation kinds)
|
||||
5e8d662 Phase 0.x bench-row telemetry (pointer/bracket/profile)
|
||||
926b05e Phase 1 pure quantifier classifier (dry-run wired)
|
||||
84d5b5c Phase 2 model-profile caps + governance hash + dry-run
|
||||
6f90f21 Phase 3 reminder injection mechanism (default off)
|
||||
5a60e85 Phase 4 CLI flags + violation-kind tails + reject-broad
|
||||
```
|
||||
|
||||
Phase-2 wiring (originally split between commits 3 and 4 in the
|
||||
plan) landed in one commit (`84d5b5c`) — the cap-table data
|
||||
structures and the runner/query wiring share enough machinery
|
||||
that splitting them would have churned the diff without aiding
|
||||
review.
|
||||
|
||||
DAG/audit binding for the `quantifier_preflight` node (originally
|
||||
"commit 7" in the plan) is **not yet shipped**. Quantifier output
|
||||
surfaces on the result dict and bench rows, but no `quantifier
|
||||
_preflight` node is added to `run_dag`. Tracked as a Phase 5
|
||||
(or §11 amend) follow-up — the §10.4 three-clause CTI contract
|
||||
needs a run_dag node to be auditable post-hoc. Optional now;
|
||||
required before the §9.5 Merkle-AGI-DAG framing fully holds.
|
||||
|
||||
### 9.12 Final position
|
||||
|
||||
> Address broad-quantifier collapse by turning quantifier intensity
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue