arborist/docs/tickets/ticket-000008-broad-quantifier-preflight-guard.md

Ticket #000008 — Broad-quantifier preflight guard

Status: open · awaiting go/no-go Opened: 2026-05-02 Scope: Detect under-specified quantifier shapes (all, every, everything, etc.) at the query layer and apply a per-model claim ceiling before the LLM call, instead of catching the resulting format collapse downstream. Audience: fox + future blackops shifts. Hard constraint: Pure additive policy. No bumps to schema_version, canonicalization_version, or chunking_version. Folds into governance_policy_hash so opt-out invalidates prior records on lookup. Verifier stays binary; FORMAT_COLLAPSED (already landed) keeps owning the downstream catch.

---

1. Problem statement

Hermes-3-Llama-3.1-8B-FP8 melts under under-specified broad-quantifier questions. The 2026-05-02 case fox surfaced:

make query Q="Winners of all major sports?" BURN=1

The model interpreted "all" as license to enumerate every adjacent fact in training prior, dumped 50+ free-form prose claims with zero [E\d+] pointer tags, and the verifier returned UNGROUNDED 0/2 (the parser caught two line-fragments). FORMAT_COLLAPSED (commit 2026-05-02b, this same session) closed the downstream signal gap — operators now see · format collapsed on the audit line — but the guardrail fires after a 13-second LLM call has already burned.

The cleaner fix is upstream. The same prompt that runaways on Hermes- 3-8B plausibly stays disciplined on Qwen 3 reasoner / GPT-4-class models, because larger models retain format-following discipline under quantifier-induced enumeration pressure. The "right" claim ceiling is therefore model-dependent — a hyperparameter we calibrate per endpoint, similar to max_context_chars_by_mode (docs/qa-modes-bench.md) and claim_lattice_max_claims_per_answer (currently 12, runner-default).

1.1 Why "all" specifically

Fox's framing: all reads to a small model like a prompt injection with positive emergent-search energy — useful when the operator wants an emergent enumeration (UNGROUNDED honesty is fine, the operator gets a wide scan), corrosive when the operator wants a grounded answer with the protocol respected.

The signal isn't a single keyword. It's a continuum of quantifier intensity (Section 2). Fox sketched it as:

none is safe one to 10 is likely safe many is safe but we should have a cut off based on the model's abilities that is vibed similar to the max token input and other hyper params we are learning about.

2. Quantifier intensity ladder

The categories below derive from formal-semantics quantifier theory (Mostowski generalized quantifiers; Barwise-Cooper; Partee D- vs A- quantifiers) intersected with the operational axis arborist actually needs: expected number of claims in the answer. Categories that don't change the expected answer length are dropped from this table and surfaced as adjacent dimensions in Section 2.1.

#	Intensity	Operational shape	Examples (lexical surfaces)	Default cap (small / large)	Notes
1	ABSENT	universal-negation, single claim	none, no X, nothing, nobody, nowhere, neither, never, not a single, zero, which X is not …	1 / 1	Hermes-3-8B also inverts under negation attention; cap-1 narrows the surface (see bench-maxing.md)
2	SINGULAR	one-fact wh / definite reference	what is X, who is X, the X, which X, whom, whose, name the X, identify the X, pick the X	1 / 1	Default for any wh-question without plural/quantifier marker
3	PROPORTIONAL	descriptive fraction	most, majority of, minority of, half, a third of, 25% of, the bulk of, the lion's share of	1 / 3	Answer is about a fraction, not a count of fractions. Stays low
4	SMALL_NUM_EXPLICIT	bounded by digit	top 3, five biggest, seven X, the X-th, first/second/third, last, pair of, couple of, dozen, a handful of	match digit / match digit	Read the digit/word and use it. dozen → 12, handful → 5
5	COMPARATIVE_BOUND	bounded by inequality	at least X, at most X, more than X, fewer than X, under X, over X, up to X, between X and Y, no more than X	match bound / match bound	Numeric bound from the question. Less common in QA but worth handling
6	FEW	small set, vague	some, a few, several, a couple, a handful, a small number of, a smattering of, not many, hardly any	5 / 5	Hermes-3-8B holds discipline here
7	MANY	medium set, vague	many, various, multiple, numerous, a number of, lots of, plenty of, a great many, multitudes, several dozen	8 / 12	Where the small/large model gap opens
8	ALL	universal quantifier	all, every, each, each and every, every single, the whole, the entirety of, the totality of, any (universal use), whatever, whoever	8 / 12	The 2026-05-02 fox case. Same cap as MANY but flagged as runaway-prone
9	COMPREHENSIVE	exhaustive request	comprehensive, complete list, complete inventory, exhaustive, definitive, everything you know, tell me everything, the whole story, the full picture, from A to Z	5 / 15	Stronger than ALL — explicitly requests exhaustion. Highest runaway risk on small models
10	OPEN_REQUEST	verb-driven enumeration	tell me about, describe, explain, summarize, give me an overview of, walk me through, what about, discuss, elaborate on, expound on	5 / 12	No explicit quantifier word — verb shape implies enumeration. The york-england failure shape (#000006)

Total: 10 rungs, up from 6.

2.1 Adjacent linguistic dimensions (orthogonal axes)

Not every linguistic feature changes the expected answer length. Some are real but belong on a separate axis from the quantifier ladder, and may warrant their own ticket later.

• Frequency / temporal universals: always, usually, typically, often, sometimes, rarely, never. These describe how often a property holds, not how many answers to give. Map to SINGULAR (one descriptive claim).
• Modality: must be, can be, might be, could be, should be. Map to SINGULAR — the modal flavor doesn't change count.
• Polarity: positive vs negative wh-questions. ABSENT covers pure negation; mixed polarity (who didn't sign the X) inherits from the wh-shape. The negation-attention failure mode lives in docs/bench-maxing.md, not here.
• Distributive vs collective: each (distributive — applies to individuals one-by-one) vs all (collective — applies to the set as a whole). Operationally similar for QA; both bucket into rung 8.
• Generic / kind-level reference: bare plural with no quantifier (cats are mammals). Maps to SINGULAR by default; the question is about the kind, not enumeration.
• List-shape verbs: list, name, enumerate, identify, cite. These imply enumeration but the count is bounded by the noun phrase that follows (list the planets → bounded by 8; list all primes → ALL rung). Treat as a trigger for ALL/MANY classification rather than its own rung.
• Hedging: roughly, approximately, about, around. These modify a numeric bound (about ten) — fold into the SMALL_NUM_EXPLICIT or COMPARATIVE_BOUND rung with the bound.

2.2 Why these specific categories matter for arborist

Each new rung names a distinct expected-answer-length distribution:

• PROPORTIONAL — answer is one descriptive claim (most cats are X), not a list. Without this rung it lands in MANY and gets a cap that's too generous.
• COMPARATIVE_BOUND — explicit numeric ceiling/floor in the question. Bounding the cap to the explicit number prevents the model from over-enumerating or under-enumerating.
• COMPREHENSIVE — strictly stronger than ALL. The runaway pressure on tell me everything you know about X is empirically worse than tell me about all X (york-england, ticket #000006). Worth its own rung with an aggressive small-model cap.
• OPEN_REQUEST — operationally produces enumeration without a quantifier word. The classifier needs to detect verb-driven shapes separately from word-driven shapes.

The ABSENT, SMALL_NUM_EXPLICIT, and SMALL_NUM rungs from the v1 table split: ABSENT remains as #1, SMALL_NUM_EXPLICIT promotes to #4, and SMALL_NUM (the implicit small-count case like a couple) folds into FEW (#6).

3. Design options

Option A — Quantifier-conditioned claim cap (recommended)

Add a query-layer preflight that classifies the question into one of the six rungs above, then sets claim_lattice_max_claims_per_answer per call (overriding the default 12) before retrieval and LLM call. The verifier already supports a per-call cap (max_claims_per_answer parameter on verify_claim_lattice / verify_claim_lattice_json), so this is policy-only — no verifier changes.

Pros:

• Reuses an existing knob. TOO_MANY_CLAIMS already demotes STRICT → HYBRID; we'd be lowering the cap for shapes that warrant it.
• Per-model calibration lives in policy dict, not code.
• Operator can override via CLI flag (escape hatch for emergent- search use cases — fox's "nice for emergent searches" point).

Cons:

• Doesn't shorten the LLM call directly. The model still sees the prompt, still tries to enumerate, still drifts. We just demote the resulting answer.
• Doesn't help the FORMAT_COLLAPSED case where the model emits zero brackets — TOO_MANY_CLAIMS only fires on PARSED claims.

Option B — Prompt-side reminder injection

When the classifier hits MANY / ALL rungs, append a stronger format-discipline reminder to the system prompt (e.g. "Cite at most N claims. If the corpus does not contain enough evidence to cite N claims, return UNGROUNDED rather than enumerating from training prior."). N = per-model cap from the table.

Pros:

• Targets the root cause: model behavior under broad-quantifier pressure.
• Cheap on tokens (one extra sentence in the system reminder).

Cons:

• Hermes-3-8B already ignores parts of the existing reminder under enumeration pressure (that's how FORMAT_COLLAPSED fires). Adding more reminder text may not change behavior.
• Folds into governance_policy_hash — requires bench measurement before/after to confirm any delta is real (see docs/bench-maxing.md 5pp signal floor).

Option C — Reject at query layer

When the classifier hits ALL on a small model and the operator hasn't opted in, return UNGROUNDED with a BROAD_QUANTIFIER_REJECTED violation before the LLM call. Operator gets fast feedback ("your question is too broad for this endpoint, try narrowing or use --allow-broad").

Pros:

• Saves the LLM call entirely (~10–15s cost on Hermes-3-8B).
• Honest failure shape — UNGROUNDED on a question we know we can't answer well.

Cons:

• False positives are operator-hostile (some "all" questions are genuinely answerable; e.g. "all members of the Beatles" → 4-claim answer, easily groundable).
• Couples query layer to model capability — needs the model profile to determine reject vs allow.

Option D — Hybrid (recommended composition)

Combine A + B. Classifier sets the cap (Option A) AND injects a mode-specific reminder (Option B). C stays available as an opt-in flag (--reject-broad) but isn't on by default.

This matches fox's framing: all is useful for emergent search, just not on small models when the operator wants groundedness. We keep both paths, default to grounded.

4. Recommendation

SUPERSEDED 2026-05-03 by §9 architectural review. The original Option-D recommendation below preserved as design log; phased plan in §9.1 is the current working order. Net change: defer reminder injection (Option B / Phase 3) and reject-broad (Option C / Phase 4) until Phase 2 cap-only measurement clears the 5pp signal floor.

Original (superseded):

Land Option D. Concrete plan:

1. New module arborist/qa/quantifier.py with classify_question_quantifier(question: str) -> dict returning {"intensity": "ALL"|"MANY"|"FEW"|"SMALL_NUM"|"SINGULAR"|"ABSENT", "matched_token": str, "explicit_count": int | None}. Pure function, no I/O.
2. New policy fields on runner.DEFAULT_POLICY / query.DEFAULT_QUERY_POLICY:
   • quantifier_guard_enabled (default True)
   • quantifier_caps_by_intensity — dict mapping intensity → int cap, with a default profile for hermes-3-llama-3.1-8b and a default fallback that matches today's behavior (cap 12 across the board, so opting in costs nothing).
3. Per-model profile registry. model_profile_hash (already in the 8-dim cache key) gets a quantifier-cap profile attached. The profile lives in arborist/qa/model_profiles.py (new file).
4. Preflight wiring in arborist/qa/runner.py:ask and arborist/qa/query.py:query: classify the question, look up the intensity-keyed cap from the model profile, override the claim_lattice_max_claims_per_answer for this call.
5. Optional reminder injection in the prompt builder (gated on quantifier_reminder_enabled, default False until bench confirms a positive delta — see Section 5).
6. CLI escape hatch --allow-broad → bypasses the cap reduction for explicit emergent-search use.
7. Folds into governance_policy_hash (already covered by the existing field-list mechanism — add the new keys to _VERIFIER_POLICY_FIELDS in keys.py).

5. Bench plan

Bench winners of all major sports? is queued in bench/qa_questions.txt under "broad descriptive — under-specified 'all'" (commit 2026-05-02b). Bench plan:

1. Baseline (n=3) on Hermes-3-8B with current policy (cap 12). Record FORMAT_COLLAPSED rate, audit_mode distribution, claim count.
2. Implement Option A (cap reduction only). Bench (n=3). Compare.
3. Implement Option B (reminder injection only, no cap change). Bench (n=3). Compare.
4. Implement Option D (A + B together). Bench (n=3). Compare.
5. Cross-model: same questions on Qwen 3 / GPT-4 (manual, not in automated bench yet). Confirm large models hold format discipline under broader caps without artificial truncation.

Signal floor: 5pp per docs/bench-maxing.md. Decisions need at least one of: STRICT-rate change, FORMAT_COLLAPSED rate change, claim-count distribution shift.

5.1 First baseline (2026-05-02T20-45-11Z, pre-extension)

Bench file bench/qa_questions_quantifier_baseline.txt. Result: bench/qa_results/2026-05-02T20-45-11Z.{jsonl,md}.

This baseline ran before the harness extension (§5.2), so the JSONL only carries summary numbers — no format_collapsed field, no violation_kinds array. Findings here are limited to verdict counts and the n_quotes range; richer diagnostics come from the second baseline (§5.1.1).

mode	verdicts (S/H/U)	n_quotes range	median ratio	median latency
quote	0 / 3 / 0	22, 22, 22	0.455	12.9s
claim_lattice_pointer	0 / 1 / 2	14, 21, 51	0.078	15.0s
claim_lattice (JSON)	0 / 2 / 1	1, 16, 16	0.062	17.2s

Observations from this run:

1. Pointer-mode runaway confirmed quantitatively. One sample emitted 51 claims — over 4× the current cap of 12.
2. JSON-mode self-limits via schema. Claim counts: 1, 16, 16. Structured shape forces brevity but doesn't make claims stick.
3. Quote mode has near-zero variance. All three samples produced exactly 22 quotes with 10 verified.
4. No STRICT in any mode across 9 samples. The under-specified all question is too broad for STRICT under current rules.
5. FORMAT_COLLAPSED detection blind — JSONL didn't carry the field. Cache-side inspection of surviving rows showed 1 bracket on the pointer survivor, just outside the FC trigger. Initial conclusion ("FC is rare for this shape") was wrong — see §5.1.1, where FC actually fires 1/3 once we capture the field directly.

5.1.1 Second baseline (2026-05-02T20-58-57Z, post-extension)

Same bench file, re-run after the harness extension landed. Result: bench/qa_results/2026-05-02T20-58-57Z.{jsonl,md}.

mode	verdicts (S/H/U)	format-collapse	mean brackets (raw)	median ratio	median latency
quote	0 / 3 / 0	0 / 3	0.0	0.435	12.6s
claim_lattice_pointer	0 / 0 / 3	1 / 3	3.7	0.000	8.1s
claim_lattice (JSON)	0 / 1 / 2	0 / 3	0.0	0.000	14.7s

Per-mode violation-kind tallies (counts = rows in which the kind fired at least once):

mode	NO_EVIDENCE_POINTER	TOO_MANY_CLAIMS	TITLE_MISMATCH	FORMAT_COLLAPSED	SCHEMA_INVALID	CITATION_MISMATCH	TOO_MANY_EVIDENCE_IDS	WARRANT_MISSING
quote	0	0	0	0	0	0	0	0
claim_lattice_pointer	3	2	1	1	1	0	0	0
claim_lattice (JSON)	0	0	0	0	0	2	1	1

Findings that change the §5.1 analysis:

1. NO_EVIDENCE_POINTER is the dominant pointer-mode gate (3/3), not TITLE_MISMATCH (1/3). Every pointer-mode run had at least one prose-line without an [E\d+] tag. The §5.1 inference that "all-claims-TITLE_MISMATCH" was the gate was wrong.
2. FORMAT_COLLAPSED actually fires 1/3 on this question — not the rare corner I called it in §5.1.
3. TOO_MANY_CLAIMS fires 2/3 in pointer mode — cap-12 demote is active more often than the §5.1 n_quotes column suggested.
4. JSON-mode failure shape is different. CITATION_MISMATCH 2/3 (claim text didn't textually overlap cited evidence) + TOO_MANY_EVIDENCE_IDS 1/3 + WARRANT_MISSING 1/3. None of these appear in pointer mode at this n. Suggests the per-mode policy knobs that matter are different per mode.
5. Latency variance dominates n=3. Pointer-mode median dropped 15.0s → 8.1s between baselines. Cache warmup + concurrency nondeterminism, not a real signal change.

What this implies for Section 3's options:

• Option A (cap reduction) would lower TOO_MANY_CLAIMS rate but probably not move the verdict — NO_EVIDENCE_POINTER fires upstream of the cap and is the dominant gate. Cap-only is unlikely to clear ≥5pp.
• Option B (prompt reminder) targets the actual gate (NO_EVIDENCE_POINTER = "model emits prose without tags"). If reminder injection improves bracket discipline, it should move this rate. Worth measuring.
• Option D (A + B) still recommended but motivation is now B carrying the verdict, A carrying the secondary cleanup.

5.2 Bench harness extension (landed 2026-05-02b)

Originally tracked here as a gap — closed in this same session.

Landed:

• arborist/qa/query.py:2580-2592 — format_collapsed and raw_answer surfaced on the query() result dict.
• bench/qa_sweep.py:_run_one — three new fields per row: format_collapsed, violation_kinds (sorted unique kind strings), answer_brackets (count of [E\d+] in raw_answer for lattice modes; 0 elsewhere).
• bench/qa_sweep.py:_summarize — per-mode FC count, violation_kind_counts: defaultdict(int), lattice-only answer_brackets_sum/answer_brackets_n.
• bench/qa_sweep.py:_render_markdown — new ## format-collapse + violation kinds section with per-mode FC rate, mean raw brackets, and one column per observed violation kind (union across the sweep).
• tests/test_bench_qa_sweep.py — 5 new tests pinning explicit- True-only FC counting, per-mode kind tallies, lattice-only bracket aggregation, FC-section rendering, and graceful no-violations degradation.

5.3 Sub-investigation: SCHEMA_INVALID in pointer mode (resolved)

The §5.1.1 table shows SCHEMA_INVALID: 1 under claim_lattice_pointer. Initial concern was that this kind originated only in the JSON-mode verifier and was somehow leaking into the pointer path.

Resolution (2026-05-02b): false alarm. SCHEMA_INVALID is also a legitimate pointer-mode kind, emitted by verify_claim_lattice in two well-defined cases:

• Tag with no claim text (verify.py:1242) — model emitted [E5] on a line with no actual claim text before the bracket.
• Bare-name claim (verify.py:1270) — claim has fewer than min_claim_content_tokens (default 3) content tokens, e.g. "T-rex. [E5]". Forces a sentence-shape claim with a predicate.

Both failures are structurally schema-invalid at the per-claim level — the surface form doesn't yield a meaningful claim/pointer pair — so the kind name is consistent. The claim_lattice JSON verifier reuses the same kind name for analogous failures (verify.py:1801,1806,1813,1838,1851,1877,1905), which is why the bench-side union table groups them under one column.

No action needed. Bench-side rendering is correct; the SCHEMA_INVALID:1 cell on the pointer-mode row is signal, not noise — one of the three pointer-mode runs emitted at least one bare-name or empty-text claim.

A useful refinement (out of scope for ticket #000008): split SCHEMA_INVALID by sub-reason at bench-aggregate scale so we can distinguish "model emitted bare-name claims" from "model emitted unparseable JSON envelope." Tracked as a future bench-renderer enhancement, not blocking.

6. Open questions

• Does the classifier need to handle multi-quantifier questions ("all winners and some losers")? Initial answer: take the highest intensity. Can refine on bench evidence.
• Should tell me about X (no explicit quantifier) classify as MANY? The "tell me all there is to know" precedent (york-england case, ticket #000006) suggests yes — operationally it produces the same enumeration pressure.
• Does the per-model profile belong in arborist/qa/model_profiles.py or extend the existing model_profile_hash derivation in keys.py? Initial answer: new file, hash-derived from the profile dict so changing a cap invalidates prior records.

7. Scope boundaries

• This ticket does NOT change the verifier. FORMAT_COLLAPSED stays the downstream catch.
• This ticket does NOT add a new audit_mode token. Cap demotion surfaces through the existing TOO_MANY_CLAIMS violation path on the audit-line tail.
• This ticket does NOT touch retrieval. Quantifier guard runs at policy layer only; retrieval pipeline (arborist/qa/query.py Sections 1–9) stays untouched.

8. Status

Closed · landed in 4f2b5a6 (2026-05-03).

All four phases landed; defaults flipped per §12.11 after the four-cell A/B + n=5 verification. Pure-Option-A landing: reminder default-on for lattice modes, cap default operator-opt-in with JSON-only allowlist. Pointer-mode cap is filtered out (wasted on the 0/45 STRICT floor). Bench-side mechanism + measurement complete.

Follow-ups split into separate tickets per the design log convention:

• Ticket #000009 — quantifier-preflight run-DAG node binding (Phase 5 / §9.5 Merkle-AGI-DAG framing). Required to close the audit-replay gap surfaced in §11.11 (currently quantifier output is on result dict but NOT in run_dag_root hash). Substantial separate work; cleaner as its own ticket.
• Cross-model Qwen / GPT-4 bench — manual operator task. Out-of-automated-scope; not blocking #000008's closure.

This ticket file remains in place as the design log per docs/TICKETS.md convention ("Do not delete tickets; they are the design log").

Implementation commit chain (all on origin/main):

Commit	Phase	Scope	Tests
2ffed00	0	bench-harness extension (FC rate, violation kinds)	+5
5e8d662	0.x	bench-row telemetry (pointer/bracket/profile fields)	+5
926b05e	1	pure quantifier classifier arborist/qa/quantifier.py	+61
84d5b5c	2	model-profile caps arborist/qa/model_profiles.py	+19
6f90f21	3	reminder injection arborist/qa/quantifier_reminder.py	+19
5a60e85	4	CLI flags + 4 violation kinds + reject-broad early-return	+16

Total: 906 tests passing (120 new); 36 skipped.

Defaults respected per §10.11.3 dry-run discipline:

quantifier_guard_enabled       True   # classifier emits telemetry
quantifier_guard_apply_caps    False  # cap reported, NOT applied
quantifier_reminder_enabled    False  # reminder mechanism off
quantifier_reject_broad        False  # reject-broad off
quantifier_guard_modes         ["claim_lattice_pointer",
                                "claim_lattice"]

§10.11.2 disable hierarchy fully wired (six levels):

1. Per-test: policy={"quantifier_guard_enabled": False}
2. Per-call CLI: --no-quantifier-guard, --allow-broad, --reject-broad, --apply-quantifier-caps
3. Per-phase policy: 7 fields (master, apply_caps, reminder_enabled, reject_broad, caps_by_intensity, guard_modes, plus Phase 0 format_collapse_check_enabled)
4. Per-mode: quantifier_guard_modes (quote opts out by default)
5. Per-model: arborist/qa/model_profiles.py PROFILES dict
6. Master via governance: all 7 quantifier policy fields fold into _VERIFIER_POLICY_FIELDS so flipping any of them invalidates prior cache records on lookup

Live end-to-end verification (post-5a60e85):

$ arborist query --reject-broad "Winners of all major sports?"
Winners of all major sports?
  UNGROUNDED · via BROAD_QUANTIFIER_REJECTED · ALL ("all")
  · cap was 8  0/0  0.0s  (preflight)

  BROAD-QUANTIFIER PREFLIGHT REJECTED · scope unbounded
  Question matched ALL intensity ("all") with an under-specified
  universe. Narrow ... or run with --allow-broad for exploratory
  enumeration.

$ arborist query --reject-broad "name all members of the Beatles"
name all members of the Beatles
  UNGROUNDED · via claim_lattice · title mismatch  4/4  20.9s
  [Beatles enumerated; scope_bound_hint=bounded → not rejected]

Next steps are measurement, not code:

• §10.11.3 step 2: full bench (134 questions × 3 modes × n=3) under dry-run. Inspect per-question quantifier_intensity and claim_cap_applied; catch mis-classifications.
• §10.11.3 step 3: fix any classifier drift. Re-run dry-run bench.
• §10.11.3 step 4: flip quantifier_guard_apply_caps=True on the broad-quantifier subset. Measure §10.8 deltas (≥5pp gate per mode for STRICT-rate / FORMAT_COLLAPSED rate / pointer-loss rate).
• §10.8 decision tree decides whether to flip quantifier_reminder_enabled and quantifier_reject_broad defaults, or keep them opt-in.

Where §10 conflicts with §9, §10 wins (specifically: three-form universal compilation §10.1, entropy framing §10.2, new violation kinds vs new audit_mode §10.3, three-clause CTI contract §10.4).

9. Architectural review (2026-05-03, Asia/Kuala_Lumpur)

De novo review applying the CTI / Merkle-AGI-DAG / Reverse-RAG / PROMETHEUS-Σ stack to this ticket. Verdict: directionally correct, but Option D should not land as default before Phase 0 observability is in place and Phases 1+2 measure where the verdict actually moves.

9.1 Phased rollout

Replaces §4's "land Option D" as the recommendation.

Phase 0 — Bench instrumentation first.
Phase 1 — Deterministic quantifier preflight classifier.
Phase 2 — Model-profile claim-budget contract.
Phase 3 — Optional prompt reminder, gated by measured improvement.
Phase 4 — Optional reject-broad mode, default off.

9.2 Go / hold matrix

GO   bench harness extension + quantifier classifier + policy metadata.
GO   per-model claim-budget cap as governance policy.
HOLD reminder injection as default until measured.
HOLD reject-at-query-layer as default; keep it opt-in.

9.3 What this is in the architecture

This is not primarily a retrieval, verifier, or prompt- engineering problem. It is a preflight governance problem:

query shape
+ model capability profile
+ answer mode
+ one-shot benchmark constraint
→ allowable claim budget and output contract

In PROMETHEUS-Σ terms: a difficulty / homeostasis controller. Detect when a query shape is likely to exceed the model's ability to stay grounded, then reduce the answer budget before the model call.

In CTI terms: a pre-answer clause contract with five clauses (question, quantifier, model, budget, admissibility). Example for fox's 2026-05-02 case:

Question clause:       "Winners of all major sports?"
Quantifier clause:     intensity = ALL / COMPREHENSIVE-adjacent
Model clause:          model = Hermes-3-8B; broad-quantifier
                       discipline = weak
Budget clause:         max_claims = calibrated cap
Admissibility clause:  claims beyond cap or without pointers
                       cannot be admitted

In Merkle-AGI-DAG terms: the preflight contract must be committed into the run DAG, not left as invisible policy.

9.4 Why baseline evidence complicates Option D

The §5.1.1 baseline shows the dominant pointer-mode gate is NO_EVIDENCE_POINTER (3/3), not TOO_MANY_CLAIMS or TITLE_MISMATCH. A claim cap alone (Option A) won't move the verdict — it only lowers the denominator. A reminder alone (Option B) might or might not move bracket discipline; we don't know yet. The correct first move is:

make the failure measurable per bench row

That is the §5.2 bench harness gap, landed in commit 2ffed00. Phase 0 is done. Phases 1+2 can now be measured against richer baselines.

9.5 Stack components

CTI: broad-quantifier preflight clause

A new CTI node before retrieval / model generation:

{
  "node_type": "quantifier_preflight",
  "question": "Winners of all major sports?",
  "intensity": "ALL",
  "matched_token": "all",
  "operational_shape": "universal_enumeration",
  "model_profile": "hermes-3-llama-3.1-8b",
  "claim_cap": 8,
  "allow_broad": false,
  "reject_broad": false,
  "policy_source": "quantifier_guard_v1"
}

This node is not a semantic-truth claim. It is a claim-budget contract.

Merkle-AGI-DAG: commit the preflight decision

Extend the run DAG so the quantifier decision is hash-bound (it causally changes output behavior, so audit must see it):

question
quantifier_preflight    ← new
retrieval_plan
retrieval_result
evidence_map
prompt
raw_model_output
parsed_claim_lattice
pointer_verify
format_check
anchor_warrant
final_label

Reverse-RAG: broad requests cannot imply complete enumeration

For broad quantifier queries, force one of three answer shapes:

bounded evidence-linked subset
ungrounded / under-specified
operator-opted emergent scan

For "Winners of all major sports?", a grounded one-shot system must not pretend it can enumerate "all major sports" without defining: which sports? which year? which league? which country? what counts as major?

Output label becomes one of:

BROAD-CAPPED · POINTER-LINKED · 8-claim cap · not exhaustive
UNGROUNDED · BROAD_QUANTIFIER_UNDER_SPECIFIED
EXPLORATORY · unbounded broad query · not evidence-warranted

PROMETHEUS-Σ: model-profile homeostasis

PROMETHEUS-Σ owns the mapping model_profile → quantifier_intensity → claim_cap:

{
  "hermes-3-llama-3.1-8b": {
    "ABSENT": 1, "SINGULAR": 1, "PROPORTIONAL": 1,
    "SMALL_NUM_EXPLICIT": "explicit_count",
    "FEW": 5, "MANY": 8, "ALL": 8,
    "COMPREHENSIVE": 5, "OPEN_REQUEST": 5
  },
  "large_reasoner_default": {
    "ABSENT": 1, "SINGULAR": 1, "PROPORTIONAL": 3,
    "FEW": 5, "MANY": 12, "ALL": 12,
    "COMPREHENSIVE": 15, "OPEN_REQUEST": 12
  }
}

9.6 Phase details

Phase 0 — Bench harness instrumentation (LANDED 2ffed00)

Three fields per bench row:

format_collapsed: bool
violation_kinds: list[str]
answer_brackets: int

Phase 0.x — Bench-row telemetry expansion (LANDED 5e8d662)

Eight additional fields covering pointer/bracket/profile/quantifier slots:

answer_pointer_count        distinct E\d+ ids in raw_answer
answer_chars_with_brackets  chars inside [E\d+,...] regions
raw_meaningful_line_count   >20-char lines in raw_answer
quantifier_intensity        classifier output (Phase 1)
quantifier_matched_token    classifier matched lexical surface
scope_bound_hint            bounded / unbounded / unknown
claim_cap_applied           cap looked up (None until Phase 2)
model_profile_id            configured model id verbatim

Helper _bracket_diagnostics() bundles the bracket/pointer/line extraction; module-level regexes (_BRACKET_RE, _BRACKET_REGION _RE, _POINTER_ID_RE) avoid per-row recompilation.

Phase 1 — Pure quantifier classifier (LANDED 926b05e)

arborist/qa/quantifier.py with classify_question_quantifier(), pure function (no I/O, no model call, no retrieval call). Returns:

{
  "intensity": "ALL",
  "matched_token": "all",
  "explicit_count": null,
  "is_broad": true,
  "operational_shape": "universal_enumeration",
  "scope_bound_hint": "unbounded",
  "classifier_version": "quantifier-v0.1"
}

Wired into query() and runner.ask() after policy resolution; both miss-path and cache-hit-path result dicts surface quantifier_intensity, quantifier_matched_token, scope_bound_hint, quantifier_explicit_count so bench rows stay column-aligned.

Highest-intensity-wins arbitration: COMPREHENSIVE strictly stronger than OPEN_REQUEST (both > ALL). tell me everything about all wars → COMPREHENSIVE rather than OPEN_REQUEST.

Scope-bound heuristic (§10.1): bounded universals (all members of the Beatles, year-anchored questions) get scope_bound_hint: "bounded"; broad+missing-anchor → "unbounded"; everything else → "unknown". Heuristic only — corpus-arity check left for future refinement.

Phase 2 — Model-profile claim-budget contract (LANDED 84d5b5c)

arborist/qa/model_profiles.py ships two profiles:

• adamo1139/Hermes-3-Llama-3.1-8B-FP8-Dynamic — tight caps for broad intensities (ALL=8, COMPREHENSIVE=5, OPEN_REQUEST=5).
• default — large-reasoner-class fallback (ALL=12, COMPREHENSIVE=15, OPEN_REQUEST=12). Used when model_profile_id has no entry.

cap_for_intensity() resolves caps via three-source chain:

1. policy_overrides (per-call dict, highest priority)
2. per-model profile from PROFILES
3. "default" profile fallback

EXPLICIT_COUNT sentinel handles SMALL_NUM_EXPLICIT and COMPARATIVE_BOUND — cap is the question's explicit count, not a profile-set value. Defensive fallback to MANY cap if classifier fired the rung without extracting a count.

Four policy fields, all in _VERIFIER_POLICY_FIELDS:

quantifier_guard_enabled        master kill         (default True)
quantifier_guard_apply_caps     dry-run gate        (default False
                                                     per §10.11.3)
quantifier_caps_by_intensity    per-call override   (default {})
quantifier_guard_modes          per-mode opt-in     (default
                                ["claim_lattice_pointer",
                                 "claim_lattice"])

effective_max_claims computed in both query() and runner.ask() from the (classifier_intensity, model_profile, policy_overrides) triple. Dry-run mode keeps it at the policy default (12); flipping quantifier_guard_apply_caps=True shadows the default with the looked-up cap.

Phase 3 — Prompt reminder (LANDED 6f90f21, default OFF)

arborist/qa/quantifier_reminder.py ships broad_quantifier_reminder() with two templates:

• Bounded universe (all members of the Beatles): "This is a broad-quantifier query with a bounded universe. Return at most N pointer-linked claim lines. Each claim must cite an evidence id like [E5]; do not write claim lines without bracket citations."
• Unbounded universe (winners of all major sports): "This is a broad-quantifier query with an under-specified scope. Return at most N pointer-linked claim lines. If you cannot ground N claims with evidence IDs, return fewer grounded claims. Do not enumerate from training prior. Each claim must cite an evidence id like [E5]; do not write claim lines without bracket citations."

Inserted as a user-turn message between the existing grounding_reminder and the evidence/question payload — same position the standard reminder uses, where Hermes-3-8B's most-recent-token attention catches it.

Default OFF (quantifier_reminder_enabled: false); operator opts in per-call. Bench sequence still required:

baseline → cap only → reminder only → cap + reminder

Only flip the default if reminder-only or cap+reminder beats cap-only beyond the §10.8 5pp floor.

Phase 4 — CLI flags + violation tails (LANDED 5a60e85)

CLI flags on arborist query:

--no-quantifier-guard      Level 2 disable: kills guard for one call
--allow-broad              emergent search: classifier on, caps off
--reject-broad             strict reject: ALL/COMPREHENSIVE/
                           OPEN_REQUEST + scope_bound_hint=="unbounded"
                           returns UNGROUNDED before LLM call.
                           Bounded universals are NOT rejected.
--apply-quantifier-caps    flip dry-run gate per-call (path from
                           dry-run to live cap)

Three new soft-demote violation kinds (§10.3) — no new audit_mode token; each renders a tail on the audit-line:

BROAD_QUANTIFIER_RUNAWAY       "broad runaway"     (raw_lines >> brackets)
BROAD_QUANTIFIER_CAP_APPLIED   "broad cap N"       (cap interpolated)
BROAD_QUANTIFIER_SCOPE_UNBOUND "broad unbounded"

Plus one HARD demote (early-return UNGROUNDED):

BROAD_QUANTIFIER_REJECTED      "broad rejected"    (preflight rejection)

Reject-broad early-return path in query() returns UNGROUNDED before the LLM call when policy enables quantifier_reject_broad AND the question is broad-unbounded. Result schema mirrors a normal UNGROUNDED row (answer_text carries rationale + narrowing hints). _render_query_human has a dedicated branch for the new status so operators see the rejection without --json.

Outputs (live-verified):

--reject-broad fires:
    UNGROUNDED · via BROAD_QUANTIFIER_REJECTED · ALL ("all")
    · cap was 8  0/0  0.0s  (preflight)
    BROAD-QUANTIFIER PREFLIGHT REJECTED · scope unbounded
    [actionable narrowing hints]

--allow-broad fires:
    EXPLORATORY · unbounded broad query · not
    evidence-warranted

Default behavior: cap reduction (Phase 2) only. No reminder. No rejection.

9.7 Labels — broad ≠ verified

Do not let all silently become "here are 51 prior-driven guesses."

Extend the label ladder (display layer; schema unchanged):

POINTER-LINKED          (existing)
ANCHOR-WARRANTED        (existing)
EVIDENCE-WARRANTED      (existing)
BROAD-CAPPED            new — bounded sample under broad-quantifier cap
BROAD-REJECTED          new — preflight rejection before LLM
FORMAT-COLLAPSED        existing tail
UNGROUNDED              (existing)

Do not call broad outputs EVIDENCE-WARRANTED unless every admitted claim passes pointer + anchor/warrant checks. Bench harness should track BROAD-CAPPED runs separately from EVIDENCE-WARRANTED so the operator-visible verdict matches what the system actually committed.

9.8 Test surface — pinned 2026-05-03

All check boxes below are now [x] — landed in commits per §8 status table. Bench-side measurement (§10.8 decision-tree thresholds) remains open and orthogonal to the test pinning.

Classifier (Phase 1, tests/test_quantifier_classifier.py — 61 tests):

[x] "Winners of all major sports?"          → ALL
[x] "complete list of winners..."           → COMPREHENSIVE
[x] "tell me about major sports winners"    → OPEN_REQUEST
[x] "top 3 winners"                         → SMALL_NUM_EXPLICIT, cap 3
[x] "five biggest cities"                   → SMALL_NUM_EXPLICIT, cap 5
[x] "most winners were..."                  → PROPORTIONAL
[x] "who won the Super Bowl?"               → SINGULAR
[x] "which teams did not win..."            → ABSENT

Policy + governance (Phase 2, tests/test_quantifier_caps.py — 19 tests):

[x] Hermes + ALL → cap 8.
[x] large model (default profile) + ALL → cap 12.
[x] policy_overrides beats per-model profile.
[x] EXPLICIT_COUNT sentinel resolves from question count.
[x] All four Phase-2 fields in _VERIFIER_POLICY_FIELDS.
[x] governance_policy_hash bumps on every cap-related field flip.
[x] All ten intensity rungs covered by every profile.

Reminder (Phase 3, tests/test_quantifier_reminder.py — 19 tests):

[x] Non-broad intensity → no reminder (None return).
[x] Bounded universe → "bounded universe" template, no
    "training prior" clause.
[x] Unbounded universe → stricter template with no-prior clause.
[x] Unknown scope → falls back to unbounded template.
[x] Cap N appears verbatim in reminder.
[x] [E\d+] citation rule restated.
[x] quantifier_reminder_enabled in _VERIFIER_POLICY_FIELDS.
[x] governance_policy_hash bumps on flip.

Phase 4 + violation tails (tests/test_quantifier_phase4.py — 16 tests):

[x] BROAD_QUANTIFIER_RUNAWAY/CAP_APPLIED/SCOPE_UNBOUND in soft-demote set.
[x] BROAD_QUANTIFIER_REJECTED NOT in soft-demote set (it's hard).
[x] All three soft kinds → ANCHOR-WARRANTED rung.
[x] Tail rendering: "broad cap N" interpolates the cap value.
[x] Tail combines with existing kinds (title mismatch + broad cap).
[x] End-to-end render through _render_query_human.
[x] quantifier_reject_broad in _VERIFIER_POLICY_FIELDS.
[x] governance_policy_hash bumps on flip.
[ ] (skipped) integration: --reject-broad on broad-unbounded
    returns UNGROUNDED before LLM call.
[ ] (skipped) integration: --reject-broad on bounded-universal
    does NOT reject.

The two skipped integration tests are exercised by the live bench cycle (and by the manual smoke tests recorded in §8 Status).

Bench harness:

[x] bench JSONL contains format_collapsed.
[x] bench JSONL contains violation_kinds.
[x] bench JSONL contains answer_brackets / pointer_count /
    chars_with_brackets / raw_meaningful_line_count.
[x] bench JSONL contains quantifier_intensity / matched_token /
    scope_bound_hint / claim_cap_applied / model_profile_id.
[x] bench summary has format-collapse + violation-kinds section.

No-regression:

[x] FORMAT_COLLAPSED remains downstream catch (untouched).
[x] verifier binary logic unchanged (cap is a parameter, not a rule).
[x] existing pointer-mode checks still pass (full suite green).
[x] no LLM verifier introduced (preflight is pure regex).
[x] claim cap is per-call policy, not schema mutation
    (no schema_version / canonicalization_version / chunking_version
     bump anywhere in commits 5e8d662…5a60e85).

9.9 Seven-point program mapping

D1  Stop making Hermes prove things:
    preflight constrains Hermes instead of asking it to self-control.
D2  Hermes emits pointer clauses:
    cap governs number of pointer clauses.
D3  Build CTI internally:
    quantifier preflight is a CTI governance node.
D4  Bind retrieval/evidence maps:
    quantifier preflight binds into run DAG beside retrieval plan.
D5  Verify pointers deterministically:
    cap and FORMAT_COLLAPSED stay deterministic.
D6  Anchor-class warrant before NLI:
    broad-quantifier guard prevents runaway before warrant overload.
D7  Rename labels honestly:
    broad outputs are BROAD-CAPPED / BROAD-REJECTED / UNGROUNDED,
    never silently EVIDENCE-WARRANTED.

9.10 Conceptual frame — query-shape circuit breaker

The quantifier guard is not a "claim cap." It is a query-shape circuit breaker. Broad quantifiers are predictable enumeration pressure:

broad quantifier
→ higher expected claim count
→ higher format-collapse risk
→ higher lazy-anchor risk
→ lower admissibility confidence on small models
→ preflight cap or rejection

The guard belongs before retrieval / model generation, not after verification. PROMETHEUS-Σ language:

quantifier intensity = reasoning difficulty signal
claim cap            = homeostatic control
model profile        = capability prior
governance hash      = audit binding

9.11 Implementation commit sequence — actual

2ffed00  Phase 0    bench-harness extension (FC rate, violation kinds)
5e8d662  Phase 0.x  bench-row telemetry (pointer/bracket/profile)
926b05e  Phase 1    pure quantifier classifier (dry-run wired)
84d5b5c  Phase 2    model-profile caps + governance hash + dry-run
6f90f21  Phase 3    reminder injection mechanism (default off)
5a60e85  Phase 4    CLI flags + violation-kind tails + reject-broad

Phase-2 wiring (originally split between commits 3 and 4 in the plan) landed in one commit (84d5b5c) — the cap-table data structures and the runner/query wiring share enough machinery that splitting them would have churned the diff without aiding review.

DAG/audit binding for the quantifier_preflight node (originally "commit 7" in the plan) is not yet shipped. Quantifier output surfaces on the result dict and bench rows, but no quantifier _preflight node is added to run_dag. Tracked as a Phase 5 (or §11 amend) follow-up — the §10.4 three-clause CTI contract needs a run_dag node to be auditable post-hoc. Optional now; required before the §9.5 Merkle-AGI-DAG framing fully holds.

9.12 Final position

Address broad-quantifier collapse by turning quantifier intensity into a committed preflight governance clause. Classify the query shape, select a per-model claim budget, bind that decision into the run DAG and governance hash, measure collapse / violation kinds in the bench harness, and only then test reminder injection or rejection modes. This preserves one-shot discipline, avoids verifier bloat, respects Hermes' limitations, and makes broad- query failures measurable instead of mysterious.

Recommendation now reads:

• Default grounded mode: Phase 2 cap only. No reminder. No reject.
• Emergent search mode: --allow-broad.
• Strict reject mode: --reject-broad (operator opt-in).
• Phase 3 reminder lands only if bench shows ≥5pp delta over Phase 2 alone.

10. Architectural review II — refinements (2026-05-03)

Second-pass de novo synthesis. Reaffirms §9's phased plan; the changes below refine specific sections without reopening the overall direction. Where §10 conflicts with §9, §10 wins.

10.1 Bounded vs unbounded vs emergent universals

The §2 ladder treats ALL as one rung. That is too coarse. CTI should compile broad universals into three operational forms, with different policies:

Bounded universal — set is naturally finite and corpus-known.

Example:    "all members of the Beatles"
Policy:     allow enumeration; cap = known/explicit bound (4)
Label:      EVIDENCE-WARRANTED if every member resolves

Unbounded / under-specified universal — set is undefined under current scope.

Example:    "winners of all major sports"
Open holes: which sports? which leagues? which year?
            which countries? what counts as major?
            current winners or historical?
Policy:     do not enumerate from model prior
            return scoped uncertainty OR bounded sample
Label:      UNGROUNDED · BROAD_QUANTIFIER_SCOPE_UNBOUND
            (or BROAD-CAPPED if a bounded sample is allowed)

Emergent-search universal — operator opted in to exploratory scan via --allow-broad.

Policy:     allow exploration; cap still applies unless
            explicitly overridden; no completeness claim
Label:      EXPLORATORY · unbounded broad query
            · not evidence-warranted

The Phase 1 classifier should not just emit intensity=ALL; it must also emit a scope_bound_hint field so the runner can route to the right form. Exact heuristics for bound detection (corpus arity check, explicit count, named-entity-set-cardinality) are sub-design under Phase 1 — initial classifier can default to unbounded for broad-quantifier+missing-domain shapes and require the operator to confirm bound via narrowing or --allow-broad.

10.2 Quantifier intensity as entropy-pressure signal

Cap is a consequence, not the primitive. PROMETHEUS-Σ should treat broad-quantifier intensity as an entropy-expansion signal:

broad quantifier
→ candidate answer-space cardinality expands
→ before evidence has bounded the domain
→ small models cannot retain format/grounding under that pressure

Preflight questions PROMETHEUS-Σ should answer before the LLM call:

Is the answer universe bounded?
Is there an explicit count?
Is there an explicit domain?
Is the model profile capable of format retention at this entropy?
Is the operator asking for grounded answer or exploratory scan?

Cap is one knob. Scope-bound check is another. Reminder is a third. They are independent levers on the same underlying entropy budget — Phase 2 lands the cap lever; Phase 1 + 10.1 lands the scope-bound lever; Phase 3 (held) is the reminder lever.

10.3 New violation kinds (no new audit_mode token)

Rather than minting a new audit_mode, add three pre-existing- ladder violation kinds. They show on the audit-line tail like warrant missing / format collapsed already do.

BROAD_QUANTIFIER_RUNAWAY      raw_line_count >> pointer_count;
                              soft-demote ANCHOR-WARRANTED → POINTER-LINKED
BROAD_QUANTIFIER_CAP_APPLIED  preflight cap fired below default 12;
                              caps at ANCHOR-WARRANTED;
                              tail: "broad cap N"
BROAD_QUANTIFIER_SCOPE_UNBOUND scope-bound hint fired in 10.1;
                              demotes to UNGROUNDED if no bounded
                              sample is allowed; tail: "broad unbounded"

Folds into governance_policy_hash via the §1 hard constraint and into _SOFT_DEMOTE_VIOLATION_KINDS in arborist/cli.py.

This keeps the schema and the four-rung ladder unchanged. Display labels carry the broad-quantifier signal via the existing tail mechanism, not a new token.

10.4 Three-clause CTI preflight contract

§9.5's CTI node is one flat clause. §10 splits it into three so the contract pre-binds three distinct downstream gates:

{
  "quantifier": {
    "intensity": "ALL",
    "matched_token": "all",
    "explicit_count": null,
    "scope_bound_hint": "unbounded",
    "broad_query_risk": "high"
  },
  "answer_contract": {
    "max_claims": 8,
    "manual_quotes_allowed": false,
    "evidence_pointer_required": true,
    "allow_unbounded_enumeration": false,
    "if_scope_unbounded": "return_bounded_summary_or_ungrounded"
  },
  "evidence_contract": {
    "max_evidence_ids_exposed": 8,
    "one_claim_per_line": true
  }
}

quantifier clause = classifier output (Phase 1). answer_contract clause = model-profile cap + scope policy (Phase 2). evidence_contract clause = retrieval-side cap on evidence_map size (no retrieval pipeline change; uses existing top_k and evidence-budget knobs).

Each clause is independently hashable so audit replay can name which clause produced the demotion.

10.5 Concrete output templates

What the system should say under each mode for Winners of all major sports?:

Default grounded (small model + Phase 2 cap):

BROAD-QUANTIFIER PREFLIGHT · scope unbounded

This question is under-specified for grounded enumeration:
- "all major sports" does not define the sport set,
- "winners" does not define season/year/league/event,
- returning all winners would require a bounded source universe.

Use a narrower query, e.g.:
"winners of the 2024 major US sports championships"
or run with --allow-broad for exploratory ungrounded scan.

--allow-broad (emergent search):

EXPLORATORY · broad quantifier allowed · not evidence-warranted

- Pointer-linked claims only where evidence exists.
- No completeness claim.
- Cap still enforced unless explicitly overridden via
  --max-claims-per-answer.

--reject-broad (strict):

UNGROUNDED · BROAD_QUANTIFIER_REJECTED

This query asks for an under-specified exhaustive set.
Narrow by sport, league, year, country, or use --allow-broad
for exploratory enumeration.

These templates make broad-query refusal an honest, actionable output rather than a 51-claim runaway labeled UNGROUNDED.

10.6 Bench fields — refinement on §9.6's Phase 0.x list

§9.6 lists raw_claim_line_count, parsed_claim_count, claim_cap_applied, quantifier_intensity, quantifier_matched_token, top_violation_kinds. §10 adds:

answer_pointer_count: int       count of [E\d+] tags in raw_answer
                                 (different from answer_brackets;
                                 distinct unique pointer ids)
answer_chars_with_brackets: int char count of bracket-tagged regions
model_profile_id: str           e.g. "hermes-3-llama-3.1-8b-fp8"
                                 (with the FP8 suffix; the
                                 ARBORIST_LLM_MODEL default uses the
                                 FP8-Dynamic variant)
scope_bound_hint: str           classifier output (10.1):
                                 bounded / unbounded / unknown

The model profile id correction matters: §9 used hermes-3-llama-3.1-8b but the live endpoint runs the -FP8-Dynamic variant (adamo1139/Hermes-3-Llama-3.1-8B-FP8-Dynamic per bench/qa_sweep.py:510). Profile keys must match what's actually deployed.

10.7 What NOT to do

Negative-constraint list to keep the implementation honest:

1. Do not assume lowering cap alone solves the failure.
   Baseline (§5.1.1) shows NO_EVIDENCE_POINTER 3/3 dominates;
   cap doesn't move that gate.
2. Do not add a new audit_mode token. Use violation kinds
   from §10.3.
3. Do not touch retrieval in this ticket. Retrieval changes
   require their own ticket and bench cycle.
4. Do not claim FORMAT_COLLAPSED is the dominant collapse mode
   for this question shape — the §5.1.1 baseline says it is
   1/3 with NO_EVIDENCE_POINTER 3/3.
5. Do not persist full `answer_text` in bench rows by default.
   Counts (line_count, pointer_count, bracket_count) are
   sufficient signal; full text is privacy-sensitive at scale.
6. Do not enable rejection by default before measuring false-
   positive rate (e.g. `all members of the Beatles` is bounded
   universal — should NOT reject).
7. Do not treat all `ALL`-intensity questions identically.
   Bounded vs unbounded universal split (§10.1) matters.

10.8 Decision tree — when to flip each phase on

After Phase 0.x telemetry exists and Phase 1+2 land in dry-run mode (cap visible in row, not yet applied):

If cap reduction shifts claim-count distribution OR
   STRICT/HYBRID/UNGROUNDED distribution by ≥5pp:
       → enable Option A (cap default-on).

If reminder injection reduces FORMAT_COLLAPSED OR
   pointer-loss rate by ≥5pp:
       → enable Option B (reminder default-on).

If A+B together outperform either alone by ≥5pp:
       → land Option D (cap + reminder default-on).

If broad small-model failures remain high after A+B:
       → expose --reject-broad as opt-in policy
         (Phase 4 stays opt-in by default).

Each gate is per-mode (quote / claim_lattice_pointer / claim_lattice JSON). A change that helps pointer mode but regresses quote mode is not a green light.

10.9 Five-step method walkthrough

Per CLAUDE.md § Five-step algorithm:

1. Less dumb requirements. Replace "for ALL questions reduce cap" with "for broad/under-specified quantifier questions, compile a model-profile-specific answer budget AND scope contract." (§10.1, §10.4)
2. Delete. Drop from immediate scope: full semantic quantifier theory beyond operational answer length, new audit_mode tokens, verifier rewrite, retrieval rewrite, schema/canonicalization/ chunking bumps, always-on reject, full answer_text persistence. The ten-rung ladder stays — it's compact and actionable.
3. Simplify / optimize. Pure classifier (§9.6.1). Highest intensity wins for multi-quantifier cases. Cap lookup is caps_by_intensity[intensity] — one dict access.
4. Accelerate. Vertical slice: telemetry → classifier+tests → policy lookup + governance hash → reminder off-by-default → CLI flags → bench A/B/D.
5. Automate. Only after measurement — directive_compliance tracking, regression fixtures, profile auto-load.

10.10 Final position (II)

Ticket #000008 should become a governance-bound CTI preflight contract, not merely a verifier cap. First fix bench telemetry (DONE, 2ffed00); then classify quantifier intensity (Phase 1); then bind cap + scope-bound + model-profile into governance hash and Merkle-DAG provenance (Phase 2); then measure A/B/D before enabling reminder or reject as default. This preserves one-shot discipline, avoids schema churn, keeps the verifier deterministic, and prevents Hermes from turning all into uncontrolled entropy expansion.

Net deltas vs §9:

• §10.1 — three-form universal compilation (bounded / unbounded / emergent), not a single ALL rung.
• §10.2 — entropy-pressure framing; cap is one knob among three.
• §10.3 — three new violation kinds, no new audit_mode token.
• §10.4 — three-clause CTI contract (quantifier / answer / evidence) instead of one flat clause.
• §10.5 — concrete output templates per policy mode.
• §10.6 — bench-field additions (answer_pointer_count, answer_chars_with_brackets, model_profile_id, scope_bound_hint); model_profile_id corrected to include the -fp8 suffix.
• §10.7 — explicit "what not to do" guardrail list.
• §10.8 — quantitative decision tree for phase enablement.
• §10.9 — five-step walkthrough binding to CLAUDE.md.

10.11 Disable hierarchy + dry-run discipline

This section is load-bearing. The whole rollout is bench-first; if any phase regresses behavior, we need to be able to turn it off per-call, per-mode, per-phase, or globally without rolling back code or invalidating cache more than necessary.

10.11.1 Honest expectation-setting

The §5.1.1 baseline says NO_EVIDENCE_POINTER fires 3/3 and TOO_MANY_CLAIMS 2/3 in pointer mode. Lowering the cap (Phase 2) likely pushes more rows into TOO_MANY_CLAIMS without changing UNGROUNDED — because NO_EVIDENCE_POINTER fires upstream of the cap. Realistic Phase-2-only outcome: measurement instrument that lands cleanly, but the verdict needle barely moves.

Phase 3 (reminder) targets the right gate (NO_EVIDENCE_POINTER = "model emits prose without tags") but Hermes already ignores parts of the existing reminder under enumeration pressure (§3 Option B con). It might move the gate by 5pp, or might not.

Phase 4 (reject-broad) mechanically works but carries false- positive risk: all members of the Beatles is a bounded universal (§10.1) that should not reject. Without the bounded-vs-unbounded classifier (Phase 1 + 10.1), Phase 4 would over-reject.

Implication: build the disable hierarchy before enabling any phase, so we can honestly measure each phase's effect in isolation and roll back any that regress.

10.11.2 Six-level disable hierarchy

From most granular to most global:

Level 1 — Per-test override
    Tests pass policy={"quantifier_guard_enabled": False}.
    Same mechanism claim_lattice_warrant_check_enabled already uses
    (arborist/qa/runner.py:191). One line. No environment changes.

Level 2 — Per-call CLI flag
    --no-quantifier-guard      kills Phase 1+2 for one call
    --allow-broad              keeps classifier on but skips
                               cap reduction for broad intensities
    --reject-broad             opts in to Phase 4
    Each flag is independent of the others; mix-and-match works.

Level 3 — Per-phase policy switch
    quantifier_guard_enabled: bool                 master kill
    quantifier_reminder_enabled: bool              Phase 3 lever
    reject_broad: bool                             Phase 4 lever
    quantifier_caps_by_intensity: dict             empty dict =
                                                   classifier still
                                                   runs but no caps
                                                   apply (telemetry
                                                   only)

Level 4 — Per-mode gating
    quantifier_guard_modes: list[str]
        default: ["claim_lattice_pointer", "claim_lattice"]
    Quote mode opts out by default — already stable HYBRID 0.455
    on baseline; its failure shape is different.

Level 5 — Per-model profile
    arborist/qa/model_profiles.py keys by model_profile_id.
    Models without an entry fall back to the "default" profile,
    which has the conservative caps from §10.6 (no Hermes-tight
    tightening).

Level 6 — Master kill via governance hash
    All policy fields fold into governance_policy_hash (§1 hard
    constraint). Setting quantifier_guard_enabled: False
    invalidates prior guard-on records on lookup, so guard-off
    runs never see guard-on cached results. Clean A/B at the
    cache layer.

These compose: Level 6 disabling everything globally is the nuclear option; Level 1 disabling for one test is the lightest. Most rollback scenarios use Level 3 (one phase's switch) or Level 4 (one mode opting out).

10.11.3 Dry-run discipline

Phase 1 lands in dry-run mode FIRST. Classifier emits quantifier_intensity, quantifier_matched_token, scope_bound_hint, and claim_cap_applied=null into bench rows, but the cap is NOT applied. Phase 2's cap-application gate is disabled by default (quantifier_guard_apply_caps: false).

Workflow:

1. Land Phase 1 classifier + Phase 2 cap-table data structures
   with quantifier_guard_apply_caps=false.
2. Run full bench (134 questions × 3 modes × n=3) under dry-run.
   Inspect per-question quantifier_intensity output. Catch
   mis-classifications: e.g. "what is the capital of france?"
   should classify SINGULAR, not OPEN_REQUEST; "tell me about
   connecticut" should classify OPEN_REQUEST.
3. Fix classifier. Re-run dry-run bench. Iterate until intensity
   distribution matches operator expectation across the question
   set.
4. ONLY THEN flip quantifier_guard_apply_caps=true on the targeted
   broad-quantifier question subset and measure A/B/D per §10.8.

Dry-run mode is itself a first-class policy field:

quantifier_guard_apply_caps: bool   default False through Phase 1
                                    rollout, flipped True only after
                                    classifier-output bench review
                                    passes

This catches mis-classification before it ever affects a verdict. Mis-classification with cap on = silent regression. Mis- classification with cap off = bench row showing the wrong intensity but correct audit_mode — visible in the next bench review.

10.11.4 Regression detection

Each phase's enablement must be paired with a no-regression fixture set:

Phase 1 (classifier dry-run):
    - 20 fixture questions covering the 10 intensity rungs (§2)
      with expected (intensity, scope_bound_hint) tuples.
    - Bench drift: classifier output distribution across
      bench/qa_questions.txt monitored across releases. >5pp
      drift in any rung's count = investigate.

Phase 2 (cap application):
    - For each non-broad fixture (capital-of-france, who-painted-
      mona-lisa), assert cap=12 still applies (no false-positive
      tightening).
    - For each broad fixture (winners-of-all-major-sports), assert
      cap matches profile table.

Phase 3 (reminder):
    - Strict pre/post bench: 5pp signal floor on FORMAT_COLLAPSED
      OR pointer-loss rate, per-mode.
    - Quote mode regression check: reminder must not appear in
      quote-mode prompts (the reminder is lattice-mode-only).

Phase 4 (reject-broad):
    - Bounded-universal fixture set: "all members of the Beatles",
      "all planets in the solar system", "all US states beginning
      with M". These MUST NOT reject when --reject-broad is on
      (would prove the bounded-vs-unbounded classifier from §10.1
      is working).
    - False-positive rate logged as a bench column.

10.11.5 Rollback playbook

If Phase 2 (or any later phase) regresses production behavior:

1. Set quantifier_guard_enabled: False in policy default
   (Level 3). One commit. Existing guard-on cache records
   become unreachable; new queries land at the pre-guard
   verdict.
2. If the regression is mode-specific, prefer Level 4: drop
   the affected mode from quantifier_guard_modes. Other modes
   keep the guard.
3. If the regression is question-shape-specific (e.g. classifier
   over-fires on factoid questions), fix the classifier in
   place; cache-records remain valid because classifier_version
   IS in governance_policy_hash via the policy field-list.
4. Code rollback (git revert) is a last resort. The policy-flip
   path should handle 95% of regression scenarios without
   reverting commits.

Net effect: every phase has at least three ways to disable it, and the dry-run gate (§10.11.3) prevents the most dangerous class of failure — silent classifier drift — from ever affecting a verdict.

11. Implementation inventory (2026-05-03)

Single-source-of-truth for what was actually built in the 2026-05-03 implementation pass. Use this as the diff between "what the design said" and "what the code does."

11.1 New modules

Path	Purpose	Phase
arborist/qa/quantifier.py	Pure 10-rung classifier + scope-bound heuristic	1
arborist/qa/model_profiles.py	PROFILES dict + cap_for_intensity() + EXPLICIT_COUNT sentinel	2
arborist/qa/quantifier_reminder.py	broad_quantifier_reminder() with bounded vs unbounded templates	3

11.2 Modified modules (new behavior)

Path	Change	Phase
arborist/qa/query.py	classifier wired post-policy-resolution; effective_max_claims lookup; reject-broad early-return; reminder injection	1, 2, 3, 4
arborist/qa/runner.py	same wiring as query() for arborist ask parity; new policy fields in DEFAULT_POLICY	1, 2, 3, 4
arborist/qa/keys.py	7 new fields in _VERIFIER_POLICY_FIELDS for governance-hash binding	2, 3, 4
arborist/cli.py	4 new CLI flags; 4 new violation kinds in _SOFT_DEMOTE_VIOLATION_KINDS; _render_warrant_tail extended; _render_query_human branch for broad_quantifier_rejected status	4
bench/qa_sweep.py	_bracket_diagnostics(); module regexes; 8 new fields in _run_one; aggregation in _summarize; new markdown section	0, 0.x

11.3 New test files

Path	Tests	Coverage
tests/test_quantifier_classifier.py	61	Every rung, scope-bound heuristic, highest-wins arbitration, regression-against-over-classification
tests/test_quantifier_caps.py	19	Per-model selection, EXPLICIT_COUNT sentinel, override precedence, governance-hash invalidation, profile shape
tests/test_quantifier_reminder.py	19	Gating, bounded/unbounded template selection, cap interpolation, [E\d+] rule restatement, governance binding
tests/test_quantifier_phase4.py	16	Soft-demote registration, ladder rung mapping, tail rendering with cap interpolation, end-to-end render, governance binding (+2 skipped integration placeholders)
tests/test_bench_qa_sweep.py (extended)	+5	_bracket_diagnostics() helper across empty, single-pointer, multi-pointer-in-one-bracket, format-collapsed shape

Total new tests: 120; full suite 906 passing, 36 skipped.

11.4 New policy fields (governance-hash bound)

All seven fold into _VERIFIER_POLICY_FIELDS (arborist/qa/keys.py) so flipping any of them invalidates prior cache records on lookup.

quantifier_guard_enabled         True   master kill
quantifier_guard_apply_caps      False  dry-run gate (§10.11.3)
quantifier_caps_by_intensity     {}     per-call override dict
quantifier_guard_modes           [...]  per-mode opt-in list
quantifier_reminder_enabled      False  Phase 3 reminder default off
quantifier_reject_broad          False  Phase 4 reject default off
claim_lattice_format_collapse_check_enabled  True  (Phase 0 prereq)

11.5 New CLI flags (arborist query)

--no-quantifier-guard       Level 2 disable (kills guard for one call)
--allow-broad               classifier on, caps off (emergent search)
--reject-broad              preflight rejection for broad-unbounded
--apply-quantifier-caps     flip Phase 2 dry-run gate per-call

11.6 New violation kinds

Three soft demotes (cap ladder at ANCHOR-WARRANTED) + one hard demote (early-return UNGROUNDED). All four render audit-line tails via _render_warrant_tail in arborist/cli.py:

BROAD_QUANTIFIER_RUNAWAY        soft   → "broad runaway"
BROAD_QUANTIFIER_CAP_APPLIED    soft   → "broad cap N"  (cap interpolated)
BROAD_QUANTIFIER_SCOPE_UNBOUND  soft   → "broad unbounded"
BROAD_QUANTIFIER_REJECTED       hard   → "broad rejected"  (preflight)

11.7 New result-dict fields

Surfaced on both query() and runner.ask() return values, miss- path AND cache-hit path (so bench rows stay column-aligned across hit/miss):

quantifier_intensity        ABSENT / SINGULAR / ... / OPEN_REQUEST
quantifier_matched_token    lexical surface that triggered the rung
quantifier_explicit_count   int when SMALL_NUM/COMPARATIVE; None else
scope_bound_hint            bounded / unbounded / unknown
claim_cap_applied           int (looked up; may not be applied —
                            depends on quantifier_guard_apply_caps)

11.8 New bench-row fields

Eight per-row additions to bench/qa_sweep.py:_run_one:

answer_pointer_count        distinct E\d+ ids in raw_answer
answer_chars_with_brackets  chars inside [E\d+,...] regions
raw_meaningful_line_count   >20-char lines in raw_answer
quantifier_intensity        Phase 1 classifier output
quantifier_matched_token    Phase 1 classifier output
scope_bound_hint            Phase 1 classifier output
claim_cap_applied           Phase 2 cap lookup (None when guard off)
model_profile_id            configured model id verbatim

11.9 Implementation-time decisions (not in design)

Choices made during implementation that didn't pre-exist in §9 / §10 but are now binding via tests + commits:

1. RUNG_PRIORITY puts COMPREHENSIVE above OPEN_REQUEST (arborist/qa/quantifier.py:_RUNG_PRIORITY). Caught during Phase 1 tests when tell me everything about all wars matched both templates. COMPREHENSIVE wins because §2.2 calls it "strictly stronger than ALL"; OPEN_REQUEST is a softer shape-detector.
2. Bounded-domain anchor regex list (arborist/qa/quantifier.py:_BOUNDED_DOMAIN_ANCHORS). Hand-curated list covering Beatles, US states/presidents, planets, founding fathers, continents, oceans, plus a year-anchor (?:19|20)\d{2} pattern and a season/year/championship/tournament prefix. Future refinement (§10.1 noted): corpus-arity check.
3. Reminder templates (arborist/qa/quantifier_reminder.py). Two-template split (bounded vs unbounded); the bounded template omits the "do not enumerate from training prior" clause because the corpus has the answer set. Unknown scope falls back to the stricter unbounded template (over-warn rather than under-warn).
4. Reject-broad answer_text format (arborist/qa/query.py). BROAD-QUANTIFIER PREFLIGHT REJECTED · scope unbounded header + actionable narrowing hints (year, league, country, category) + --allow-broad escape-hatch mention. Format chosen so an operator reading raw output sees both the WHY and the HOW-TO-FIX.
5. _render_query_human branch for broad_quantifier_rejected status (arborist/cli.py). Dedicated render path so the new status doesn't fall through to the generic "unknown error" handler. Renders intensity, matched token, cap, and the rejection text on a single audit-line + body.
6. EXPLICIT_COUNT defensive fallback to MANY cap (arborist/qa/model_profiles.py:cap_for_intensity). When the classifier fired SMALL_NUM_EXPLICIT or COMPARATIVE_BOUND but didn't extract a count (shouldn't happen but covers regressions), fall back to the same profile's MANY cap rather than crash.
7. Quote mode opts out of the guard by default (quantifier_guard_modes default). Quote mode showed stable HYBRID 0.455 on the §5.1.1 baseline — different failure shape, no benefit from broad-quantifier intervention. Quote rows still carry telemetry (classifier always runs) but the cap and reminder don't apply.

11.10 Live verification artifacts

• arborist query --reject-broad "Winners of all major sports?" → preflight rejection, 0.0s, no LLM call. Rendered output recorded in §8 Status.
• arborist query --reject-broad "name all members of the Beatles" → ran the LLM (scope_bound_hint=bounded correctly suppressed rejection), enumerated 4 verified Beatles members. Rendered output recorded in §8 Status.

11.11 What was NOT shipped today

• quantifier_preflight run-DAG node (§9.5 Merkle-AGI-DAG binding, §9.11 "commit 7"). Quantifier output surfaces on the result dict and bench rows, but no node is appended to run_dag. Optional now; required before the §9.5 framing fully holds. Tracked as Phase 5 follow-up.
• §10.4 three-clause CTI contract on the run DAG. The classifier output IS structured per §10.4 (quantifier / answer_contract / evidence_contract is reflected in result-dict field grouping), but it's not yet bound as a hashable run-DAG node. Same Phase 5 dependency.
• Live A/B/D bench measurements per §10.8 decision tree. Mechanism is ready; bench discipline (full sweep under dry-run, classifier review, then flip apply_caps) hasn't been executed. Bench-not-code task.
• Cross-model bench (Qwen / GPT-4). Manual cross-model verification of "large model holds format discipline at higher caps" not yet performed. Manual / out-of-automated-bench task.

12. Live bench measurements (2026-05-03)

First post-implementation bench cycle. Measures classifier output end-to-end with Phase 1+2 wired in dry-run mode (caps reported, not applied) per §10.11.3 step 2. Also catches one classifier defect not seen in the design phase.

12.1 Pre-bench: classifier distribution scan (free, no LLM)

Ran classify_question_quantifier() on all 73 questions in bench/qa_questions.txt. Surfaced one defect: how many X? mis-classified as MANY (4 of 7 broad classifications wrong; 33% false-positive rate among broad). All four were count-questions expecting a single numeric answer ("50 states", "206 bones") — SINGULAR is correct, MANY is wrong.

Fix landed d24291b: leading-anchor count-question short-circuit. ^\s*(?:and\s+|but\s+|so\s+)?how (?:many|much)\b → SINGULAR. Anchored at start so buried how many doesn't suppress the rest of the question's quantifier markers.

Post-fix distribution across the 73-question bench:

SINGULAR              65 (89.0%)   was 61 (84%)
OPEN_REQUEST           5 ( 6.8%)   unchanged
MANY                   0 ( 0.0%)   was 4 (5%)   ← all moved to SINGULAR
ALL                    1 ( 1.4%)   unchanged
COMPREHENSIVE          1 ( 1.4%)   unchanged
SMALL_NUM_EXPLICIT     1 ( 1.4%)   unchanged

Also added two bounded-universal fixtures (Finding 2 from review): name all members of the beatles and list all planets in the solar system. Both classify ALL · scope_bound_hint=bounded. Without these, the §10.1 bounded-vs-unbounded distinction had zero live bench coverage.

12.2 Live bench — broad subset (dry-run, apply_caps=False)

Bench file bench/qa_questions_quantifier_subset.txt — 9 broad questions (7 unbounded + 2 bounded). 3 modes × n=3 = 81 runs. Result: bench/qa_results/2026-05-03T12-29-53Z.{jsonl,md}.

Per-mode summary:

mode	S/H/U	strict-rate	mean ratio	latency
quote	15/12/0	0.56	0.900	16.7s
claim_lattice_pointer	0/18/9	0.00	0.473	12.2s
claim_lattice (JSON)	5/15/7	0.19	0.524	14.5s

Key finding — bounded vs unbounded matters in practice:

question	scope	quote	pointer	JSON
Winners of all major sports?	unbounded	H:3	U:3	H:2/U:1
Tell me everything about York	unbounded	S:3	H:3	U:3
Name all members of the Beatles	bounded	S:2/H:1	U:3	S:3
List all planets	bounded	H:3	U:3	S:1/H:2

JSON mode hits 3/3 STRICT on the Beatles — same model, same verifier, different scope_bound_hint, fundamentally different outcome. The §10.1 bounded vs unbounded split is empirically real, not just architectural.

Pointer-mode violation distribution (27 runs, broad subset):

CITATION_MISMATCH       14
TITLE_MISMATCH          10
NO_EVIDENCE_POINTER      9
TOO_MANY_CLAIMS          7   ← cap=12 firing on broad questions
SCHEMA_INVALID           4   ← bare-name + empty-text claims (verify.py:1242,1270)
WARRANT_MISSING          4
LAZY_ANCHOR_DEMOTE       3
POINTER_OVERFLOW_TRIMMED 3
FORMAT_COLLAPSED         2
DEFLECTION_DETECTED      1

12.3 Telemetry verified end-to-end

Per-question classifier output (sampled across the 81-row JSONL):

winners of all major sports?
    intensity=ALL                scope=unbounded   cap=8
tell me all there is to know about york england?
    intensity=COMPREHENSIVE      scope=unbounded   cap=5
tell me about connecticut
    intensity=OPEN_REQUEST       scope=unbounded   cap=5
describe the structure of DNA
    intensity=OPEN_REQUEST       scope=unbounded   cap=5
name all members of the beatles
    intensity=ALL                scope=bounded     cap=8
list all planets in the solar system
    intensity=ALL                scope=bounded     cap=8

Quote-mode rows record cap=None (mode opted out via quantifier_guard_modes default). Lattice-mode rows record the looked-up cap (5 / 8) but the verifier used 12 (apply_caps=False preserves dry-run discipline).

12.4 §10.8 decision-tree implications

Empirically:

• Cap-only (Phase 2 apply_caps=True) is unlikely to clear the 5pp gate for pointer mode. Pointer-mode is already 0 STRICT; lowering cap from 12→8 would increase TOO_MANY_CLAIMS firings (currently 7/27) but can't move the verdict floor below 0. Same conclusion §5.1.1 reached at n=3, now confirmed at n=27.
• NO_EVIDENCE_POINTER (9/27) is the load-bearing pointer-mode failure. This is exactly what Phase 3 reminder targets — the reminder restates the [E\d+] citation rule one user-turn before the question. Worth A/B testing with quantifier_reminder _enabled=True next.
• JSON mode benefits from bounded-vs-unbounded discrimination. 3/3 STRICT on Beatles vs 0/3 on york-england. The mode + scope combination is what matters; cap-only doesn't help here either.
• Quote mode is the workhorse for broad questions at 0.56 strict-rate. It paraphrase-verifies rather than pointer-verifies, so it doesn't have the bracket-discipline burden. Keeping quote out of quantifier_guard_modes (the default) is empirically validated.

12.5 Next bench cycles

Per §10.8 decision tree, A/B sequence with this same broad subset:

• Reminder only (apply_caps=False, reminder=True) — §12.6.
• Cap only (apply_caps=True, reminder=False) — §12.7.
• Cap + reminder (apply_caps=True, reminder=True) — §12.8.
• n=5 verification of cap+reminder — §12.10.
• Defaults flipped per §12.11 landing.

5pp signal floor per docs/bench-maxing.md for default-flip decisions. Each cycle adds 81 runs at ~12-17min on Hermes.

12.6 Reminder-only A/B (2026-05-03T12-38-53Z)

Same 9-question broad subset (bench/qa_questions_quantifier_subset.txt), 3 modes × n=3 = 81 runs. Policy override: --policy quantifier_reminder_enabled=true. Everything else defaulted (apply_caps=False, reject_broad=False).

Comparison against §12.2 baseline (reminder=False):

Mode	Strict-rate	Mean ratio	UNGROUNDED count
quote	0.56 → 0.52	0.900 → 0.845	0 → 0
claim_lattice_pointer	0.00 → 0.00	0.473 → 0.643 (+17pp)	9 → 6 (−3)
claim_lattice (JSON)	0.19 → 0.22	0.524 → 0.735 (+21pp)	7 → 1 (−22pp)

Pointer-mode violation kind shifts:

Kind	OFF	ON	Δ
FORMAT_COLLAPSED	2	0	−100%
NO_EVIDENCE_POINTER	9	6	−33%
TITLE_MISMATCH	10	15	+50%
TOO_MANY_CLAIMS	7	8	+14%
CITATION_MISMATCH	14	14	0
WARRANT_MISSING	4	3	−25%
LAZY_ANCHOR_DEMOTE	3	3	0
POINTER_OVERFLOW_TRIMMED	3	3	0
SCHEMA_INVALID	4	3	−25%

§10.8 gate verdict — MET.

The §10.8 rule is "If reminder injection reduces FORMAT_COLLAPSED OR pointer-loss rate by ≥5pp: enable Option B (reminder default-on)."

Both criteria cleared:

• FORMAT_COLLAPSED: 2/27 → 0/27 (Δ ≈ −7pp absolute, 100% relative).
• NO_EVIDENCE_POINTER (pointer-loss): 9/27 → 6/27 (Δ ≈ −11pp absolute, 33% relative).

Strongest signals are NOT in the headline strict-rate column. They're in:

• Mean-ratio jumped on both lattice modes: pointer +17pp, JSON +21pp. Grounded rows are MORE thoroughly grounded under reminder.
• JSON mode UNGROUNDED collapsed 7 → 1: 22pp reduction. The HYBRID pool grew (15 → 20) and one row escaped to STRICT. Operator-visible "didn't ground" reclassified as "partially grounded".
• FORMAT_COLLAPSED elimination: the reminder explicitly restated the [E\d+] citation rule, and Hermes followed it.

Caveats:

• n=3 × 9 questions = 27 per mode is small. Variance is real; Hermes nondeterminism dominates at this sample size.
• TITLE_MISMATCH went UP (10 → 15). The reminder may pressure Hermes to cite something rather than say "no evidence", picking up wrong-source citations as a side effect. Trade-off visible: NO_EVIDENCE_POINTER ↓ but TITLE_MISMATCH ↑.
• Quote-mode strict-rate dipped 4pp (0.56 → 0.52). The reminder shouldn't fire on quote (mode-gated via quantifier_guard_modes), so this is likely Hermes nondeterminism — but worth verifying with tighter n.

Recommendation:

The §10.8 gate is met. Reminder default-on for lattice modes is supported by the data. But hold the default flip until §12.7 cap-only and §12.8 cap+reminder cells run — per §10.8 we need to know whether cap+reminder beats reminder-only by ≥5pp before defaulting to D vs B.

12.7 Cap-only A/B (2026-05-03T12-47-23Z)

Same 9-question broad subset. Policy override: --policy quantifier_guard_apply_caps=true. Reminder=False.

Three-way comparison against §12.2 baseline and §12.6 reminder-only:

Mode	Baseline	Reminder-only	Cap-only
quote strict-rate	0.56	0.52	0.59 (noise)
pointer strict-rate	0.00	0.00	0.00
JSON strict-rate	0.19	0.22	0.33
pointer mean ratio	0.473	0.643	0.516
JSON mean ratio	0.524	0.735	0.643

JSON-mode verdict distribution:

Cell	STRICT	HYBRID	UNGROUNDED
Baseline	5	15	7
Reminder-only	6	20	1
Cap-only	9	12	6

Headline: cap and reminder help in DIFFERENT ways.

• Reminder rescues UNGROUNDED → HYBRID. JSON UNGROUNDED collapsed 7 → 1 (−22pp). Restating the citation rule keeps the model from giving up.
• Cap rescues HYBRID → STRICT. JSON STRICT jumped 5 → 9 (+14pp). Forcing fewer-but-better claims raises the per-claim grounding bar — the model can fully ground 5 claims it couldn't fully ground at 12.

§10.8 gate verdict — MET on JSON mode.

The §10.8 rule for cap-only is implicit ("Option A"): if cap reduction shifts STRICT/HYBRID/UNGROUNDED distribution by ≥5pp, enable Option A.

JSON mode +14pp on STRICT-rate alone clears the floor. Pointer mode unchanged at 0/27 (the cap forces TOO_MANY_CLAIMS to fire 20 times instead of 7, but pointer-tag failures upstream of the cap still gate the verdict).

Pointer-mode violation deltas (cap-only vs baseline):

Kind	Baseline	Cap-only	Δ
TOO_MANY_CLAIMS	7	20	+186%
TITLE_MISMATCH	10	13	+30%
NO_EVIDENCE_POINTER	9	7	−22%
CITATION_MISMATCH	14	15	+7%
FORMAT_COLLAPSED	2	2	0
WARRANT_MISSING	4	4	0

Cap firing 20/27 in pointer mode is exactly what we'd expect when the cap drops from 12 to 5/8 per intensity. The cap demotes STRICT → HYBRID via TOO_MANY_CLAIMS but doesn't fix pointer-tag discipline.

Implication for cap+reminder (§12.8):

The two interventions are complementary, not redundant. The reminder reduces the UNGROUNDED pool by improving format discipline; the cap promotes HYBRID to STRICT by tightening per-claim grounding requirements. If §12.8 confirms the compound effect, that's the §10.8 trigger to land Option D as default.

Predicted §12.8 outcome (n=3, 9 questions):

• JSON STRICT-rate: 9 + N more → 0.33 + ~5pp = ~0.40 (if effects compound)
• JSON UNGROUNDED: 1 (already low; reminder dominates this metric)
• Pointer STRICT-rate: still 0/27 (neither knob fixes pointer-tag discipline at this sample size)

12.8 Cap + reminder A/B (2026-05-03T12-54-11Z)

Same 9-question broad subset. Policy overrides: --policy quantifier_guard_apply_caps=true --policy quantifier_reminder_enabled=true.

Four-cell summary table:

Metric	Baseline	Reminder	Cap-only	Cap+Reminder
quote strict-rate	0.56	0.52	0.59	0.59
quote mean ratio	0.900	0.845	0.870	0.861
pointer strict-rate	0.00	0.00	0.00	0.00
pointer mean ratio	0.473	0.643	0.516	0.684
pointer UNGROUNDED	9	6	9	6
JSON strict-rate	0.19	0.22	0.33	0.30
JSON mean ratio	0.524	0.735	0.643	0.672
JSON UNGROUNDED	7	1	6	1
JSON STRICT count	5	6	9	8
pointer FORMAT_COLLAPSED	2	0	2	0
pointer NO_EVIDENCE_POINTER	9	6	7	7

§10.8 strict-gate verdict for cap+reminder:

The §10.8 rule is "If A+B together outperform either alone by ≥5pp: land Option D."

• vs cap-only (best STRICT-rate cell): cap+reminder JSON SR is −3pp (regression, within noise).
• vs reminder-only on JSON SR: cap+reminder is +8pp (clears floor).
• vs cap-only on pointer mean ratio: cap+reminder is +17pp (clears floor).
• vs reminder-only on pointer mean ratio: cap+reminder is +4pp (just below floor).

Mixed. Cap+reminder is strictly better than reminder-only on STRICT-rate and strictly better than cap-only on mean ratio + UNGROUNDED-reduction, but it is not strictly better than both alone on every metric.

12.9 Final A/B verdict + recommendation

The four cells reveal a clean mechanism asymmetry:

Reminder rescues UNGROUNDED → HYBRID
    (restates [E\d+] citation rule; reduces format-discipline
     failures upstream of the verifier)

Cap rescues HYBRID → STRICT
    (forces fewer-but-better claims; raises per-claim grounding
     bar so the model can fully ground 5-8 claims it couldn't
     fully ground at 12)

These are complementary, not redundant. But the §10.8 strict gate ("compound beats either alone by ≥5pp on every metric") is not cleanly met. Cap+reminder is the best CELL on pointer mean ratio (0.684) but not the best on JSON STRICT-rate (cap-only wins 0.33 vs 0.30).

Recommendation: Option A — single-knob defaults, per-mode tailored.

Lattice modes (claim_lattice_pointer, claim_lattice):

quantifier_guard_enabled       True   (already default)
quantifier_guard_apply_caps    True   ← FLIP for claim_lattice (JSON) ONLY
quantifier_reminder_enabled    True   ← FLIP for both lattice modes
quantifier_reject_broad        False  (operator opt-in only)

Pointer mode keeps apply_caps=False because:

• Pointer-mode STRICT-rate is 0/27 across all four cells; cap can't rescue what's structurally upstream of cap (pointer-tag discipline).
• Cap firing 20× under apply_caps=True adds TOO_MANY_CLAIMS noise without verdict gain.
• Reminder alone moves the right metric (NO_EVIDENCE_POINTER −33%, FORMAT_COLLAPSED −100%, mean ratio +17pp).

Quote mode keeps the guard mode-gated off (already default) because:

• Quote mode handles broad questions well already (0.56-0.59 strict-rate across all cells, paraphrase verifier).
• Quote-mode strict-rate jitter (±3pp across cells) is Hermes nondeterminism, not a guard effect.

Caveats blocking immediate flip:

• n=3 × 9 questions = 27 per cell. Variance band is ~3-4pp from Hermes nondeterminism. Some of the cell-to-cell deltas (cap+reminder 0.30 vs cap-only 0.33 on JSON SR) are within noise.
• Cap-only's +14pp gain on JSON STRICT-rate is clearly above the floor; reminder's −22pp UNGROUNDED reduction is clearly above the floor. Those two findings are robust.
• Compound-effect verification at higher n (n=5 or n=10) would tighten the cap+reminder vs cap-only comparison and confirm whether the 3pp gap is real or noise.

Recommended next step:

Run n=5 verification of the cap+reminder cell (same 9 questions, 135 runs at concurrency 4 ≈ ~6-8 min). If cap+reminder JSON STRICT-rate holds at ≥0.30 with n=5, the Option D gate is met (compound effect ≈ cap-only on STRICT, with the reminder benefits preserved). If it drops below 0.27, Option A is the right landing.

Either way, the disable hierarchy (§10.11.2) means flipping defaults is reversible per-call, per-mode, per-test. No commitment is permanent — flip the default, watch one bench cycle, revert if regression.

12.10 n=5 verification (2026-05-03T19-55-14Z)

Same 9-question broad subset, n=5 instead of n=3 (135 runs total). Same policy: apply_caps=True, reminder=True. Tightens variance on the cap+reminder cell to test whether the §12.8 0.30 vs §12.7 0.33 JSON STRICT-rate gap was real or noise.

Metric	n=3	n=5
JSON STRICT-rate	0.30	0.33
JSON UNGROUNDED rate	1/27 (4%)	2/45 (4%)
JSON STRICT count	8	15
pointer STRICT-rate	0/27	0/45
pointer mean ratio	0.684	0.634
pointer UNGROUNDED rate	6/27 (22%)	10/45 (22%)

Verdict: cap+reminder JSON STRICT-rate at n=5 = 0.33, matching cap-only (n=3 0.33). The §12.8 0.30 reading was downward Hermes nondeterminism. n=5 confirms the compound effect delivers cap-only's STRICT-rate AND reminder-only's UNGROUNDED-rescue simultaneously.

§10.8 strict gate verdict at n=5 — MET on multiple metrics:

• vs reminder-only on JSON STRICT-rate: +11pp (clears 5pp floor)
• vs cap-only on JSON UNGROUNDED rate: −18pp (clears floor)
• vs reminder-only on pointer mean ratio: −1pp (within noise)
• vs cap-only on pointer mean ratio: +12pp (clears floor)

Compound effect is real. But pointer-mode cap is structurally wasted: 0/45 STRICT across all four cells; cap-on-pointer fires TOO_MANY_CLAIMS 20× without verdict gain.

12.11 Final landing — Option A defaults flipped

The §10.8 strict gate for pure Option D ("cap + reminder default- on for all lattice modes") is met. But the data-driven cleaner landing is Option A — per-mode tailored:

Lattice modes (claim_lattice + claim_lattice_pointer):
  quantifier_reminder_enabled         True   ← FLIPPED 2026-05-03
  quantifier_guard_apply_caps         False  ← unchanged (operator opt-in)
  quantifier_apply_caps_modes         ["claim_lattice"]  ← NEW field
                                              JSON-only allowlist for
                                              when operator flips
                                              apply_caps=True

claim_lattice (JSON):
  effective behavior: reminder ON; cap applies when
                      apply_caps=True (operator-driven)

claim_lattice_pointer:
  effective behavior: reminder ON; cap NEVER applied (filtered out
                      by apply_caps_modes allowlist) since cap on
                      pointer is structurally wasted noise.

quote:
  effective behavior: unchanged. Mode-gated off via
                      quantifier_guard_modes default.

Code changes (commit pending):

• arborist/qa/runner.py: quantifier_reminder_enabled=True, new field quantifier_apply_caps_modes=["claim_lattice"].
• arborist/qa/query.py: same defaults.
• arborist/qa/keys.py: quantifier_apply_caps_modes added to _VERIFIER_POLICY_FIELDS so flipping it bumps governance hash.
• Cap-application gate now reads: if apply_caps AND mode in apply_caps_modes AND cap is not None:
• 5 new tests pin: reminder default ON for both runner.DEFAULT_POLICY and query.DEFAULT_QUERY_POLICY, apply_caps_modes default ["claim_lattice"], governance-hash invalidation on apply_caps_modes flip, apply_caps default still False (dry-run discipline preserved).

Operator commands after this landing:

# Default behavior (no flag): reminder ON for lattice modes; cap
# OFF (operator opt-in only).
$ arborist query "winners of all major sports?"

# Flip cap on for one call (JSON-only by default):
$ arborist query --apply-quantifier-caps "..."

# Override the allowlist to include pointer mode too (Option D):
$ arborist query --apply-quantifier-caps \
                --policy quantifier_apply_caps_modes='["claim_lattice","claim_lattice_pointer"]' "..."

# Disable the reminder for one call:
$ arborist query --policy quantifier_reminder_enabled=false "..."

# Master kill:
$ arborist query --no-quantifier-guard "..."

Six-level disable hierarchy still intact — no flag forced upon operators; the defaults simply move toward the §10.8-validated position. Reverting the default is a one-line policy edit per §10.11.5 rollback playbook.