# User-payload layout — where the question sits relative to evidence

A prompt-structure knob that decides where the question text appears
in the final user-turn message relative to the evidence block. Pure
lever on the LLM's attention budget — no change to the verifier, the
audit chain, or the corpus.

Policy field: `user_payload_layout ∈ {"tail", "bookend", "per_chunk"}`,
default `"tail"`. CLI: `--user-payload-layout` on `arborist query`
and `arborist ask`. Make: `LAYOUT=...` on `make query`. Folds into
`governance_policy_hash` so each layout cache-partitions cleanly.

## Verdict

```
GO:    static layouts shipped as opt-in policy knobs.
NO-GO: promote bookend/per_chunk as default. The n=3×75q curated
       bench (2026-05-27) found bookend statistically indistinguish-
       able from tail and per_chunk regressing −9.78pp on aggregate
       STRICT-rate against Hermes-3-8B. See §"Bench results".
DONE:  companion missed-answer falsification guard shipped as
       Ticket #000068 (Phase 1 sidecar + Phase 2 bench evidence +
       Phase 3 opt-in demote flag). Default OFF; Phase 4 default
       flip blocks on wider bench + human spot-check. See §"Verifier-
       blind missed-answer class" and docs/tickets/ticket-000068-*.md.
```

Layout is a real but narrow lever — useful for specific failure
shapes (small-model evidence-negation under long context), not a
universal default. Keep `tail` default; route `bookend`/`per_chunk`
via the broad-quantifier classifier on query types that match the
target failure mode. The verifier-side companion guard (#000068)
catches the residual failure class layout doesn't fully close.

## Why this exists — the Ballestrini case

Same query, same corpus, same retrieval, same evidence, same model
(`hermes-3-8B`, `https://hermes.ai.unturf.com/v1`), same 22.8 KB
prompt. The default `tail` layout produced:

> Veronica Ballestrini is a country music singer and songwriter who
> has released several songs. **However, the specific songs by her
> are not mentioned in the provided evidence blocks.**

Evidence block `E2` literally contained the song names: "Amazing",
"Out There Somewhere", "Fascinated", and references to music videos
for "What's Up With That" and "Don't Say". The model received an
evidence chunk packed with answers and synthesized a negation.

The verifier marked the run `EVIDENCE-WARRANTED` because the
negation literally matched no claim that needed grounding. The
guard fires on false positives — claims unsupported by evidence —
not on false negatives — answers the model declined to give. From
the verifier's view the run was clean; from a user's view it was
broken.

Root cause is the classic lost-in-the-middle / lost-at-the-front
attention failure mode on small (≤8B) models. The final user
message under `tail` was shaped:

```
EVIDENCE:

=== E1 (Veronica Ballestrini | primary_answer_source) ===
<span 1>

=== E2 (Veronica Ballestrini | primary_answer_source) ===
<span 2 — contains the song names>

=== E3 ... E8 ===
<spans 3-8>

---

QUESTION: songs by veronica ballestrini
```

The question is 28 characters at the tail of a 21,176-char evidence
wall. Hermes-3-8B's attention budget loses track of which question
it was asked while reading through eight evidence chunks. By the
time generation begins, the model's prior is "summarize the
evidence I just read" — and the evidence as a whole is mostly
biographical prose, so the summary leans biographical and skirts
the specific-song question.

## Three layouts

### `tail` (default — preserves prior cache)

```
EVIDENCE:
<body>
---
QUESTION: <q>
```

Original shape. Adequate for large models with long-context attention
(Qwen-27B, Claude, GPT-4, etc.). Brittle on small models with long
evidence blocks.

### `bookend` (top + bottom)

```
QUESTION: <q>

EVIDENCE:
<body>
---
QUESTION: <q>
```

Question is repeated before AND after the evidence. The leading
copy seeds the attention heads on the actual ask; the trailing copy
re-anchors right before generation. This is the standard
lost-in-the-middle mitigation (Liu et al. 2023, "Lost in the
Middle: How Language Models Use Long Contexts").

Effect on Hermes-3-8B for the Ballestrini case: model goes from
*"specific songs are not mentioned"* to extracting "Fascinated",
"Don't Say", album "What I'm All About", producer Cliff Downs.
Trade-off observed in this run: the model became over-eager and
conflated Veronica Ballestrini ↔ The Veronicas (Australian pop
rock duo whose articles were also in the top-8 retrieval), pulling
"This Love", "Revolution" into the answer as if they were
Ballestrini songs.

### `per_chunk` (bookend + per-block reminder)

```
QUESTION: <q>

EVIDENCE:

[for: <q>]

=== E1 (...) ===
<span 1>

[for: <q>]

=== E2 (...) ===
<span 2>

[for: <q>]

=== E3 ... E8 ===
<spans 3-8>

---

QUESTION: <q>
```

Bookend, plus a one-line `[for: <question>]` reminder injected
before each evidence block (cheap — `<28` chars × `N` blocks on a
22 KB prompt). The question reactivates per-chunk; each chunk's
attention window has the question text immediately preceding it.

Effect on Hermes-3-8B for the Ballestrini case: model correctly
extracts "Amazing" as Ballestrini's single, then **disambiguates**
The Veronicas as a separate Australian pop rock duo with their own
songs ("Popular", "This Love", "Revolution", "Leave Me Alone"),
and additionally surfaces Elvis Costello's "Veronica" as a third
distinct entity. 6/8 evidence sources used (vs 1/8 for tail, 3/8
for bookend). No false attribution.

The `per_chunk` reminder is the question text verbatim, not a
paraphrase — keeping it byte-identical so the model's attention
heads activate on the same tokens repeatedly. Reminders are not
fired for the leading block in lattice mode (the leading bookend
copy already covers it) — they appear before every internal block
boundary (`\n\n=== `).

## Where layout matters — model-size lever

Measured 2026-05-26 on the same Ballestrini query, same retrieval,
three layouts × two models (n=1 burn each — directional, not
defensible):

| Model        | tail                                  | bookend                                            | per_chunk                                      |
|--------------|---------------------------------------|----------------------------------------------------|------------------------------------------------|
| Hermes-3-8B  | broken — negates evidence             | works, but conflates Ballestrini ↔ The Veronicas   | works, correctly disambiguates three entities  |
| Qwen-27B     | works                                 | works (~identical output to tail)                  | works (~identical output to tail)              |

Qwen-27B is unaffected by layout on this query — its attention
budget is large enough that the question stays salient through 21
KB of evidence regardless of where it appears. Hermes-3-8B is
broken under `tail` and recovered by either `bookend` or
`per_chunk` on this specific case.

Layout matters most for **small models on long-context list/
extraction queries**. For larger models, layout may be behaviorally
near-neutral in observed output — but it is **not cache-neutral or
cost-neutral**: every layout change shifts `governance_policy_hash`,
changes `cache_key`, alters token count, and can affect latency &
answer wording even when correctness is unchanged.

## Bench results — 2026-05-27

Curated QA set, <!--AUTOCOUNT:fixture-rows:bench/qa_questions.txt-->76<!--/AUTOCOUNT-->
questions (75 at sweep start + 1 Ballestrini regression added
mid-session — won't affect already-run sweep results), n=3 samples
per cell, claim_lattice mode,
Hermes-3-8B via `https://hermes.ai.unturf.com/v1`, three layouts
sequentially with concurrency=4. 225 runs per layout, 675 total
LLM calls. Output under `bench/qa_results/layout-{tail,bookend,per_chunk}/`.

| Layout      | STRICT | HYBRID | UNGROUNDED | strict-rate | mean ratio | mean latency | Δ vs tail              |
|-------------|--------|--------|------------|-------------|------------|--------------|------------------------|
| tail        | 94     | 83     | 48         | 0.418       | 0.696      | 12.8s        | — (control)            |
| bookend     | 95     | 83     | 47         | 0.422       | 0.726      | 11.9s        | +0.44pp (noise)        |
| per_chunk   | 72     | 112    | 41         | 0.320       | 0.734      | 12.6s        | **−9.78pp** (real regression) |

Per the bench-maxing rule (5pp signal floor at n=3):

- **bookend ≈ tail.** 1 STRICT delta on 225 runs is noise.
  `mean ratio` (n_verified / n_quotes) improved 0.696 → 0.726, but
  the strict-rate verdict is the gate. Bookend is a safe no-op
  aggregate — no regression, no win.
- **per_chunk regresses significantly.** −22 STRICT, +29 HYBRID.
  Per-chunk reminder over-anchors the model into citing more
  evidence per claim:
  - `TOO_MANY_EVIDENCE_IDS` violations: tail 20 → bookend 28 →
    **per_chunk 54**
  - mean answer chars in the 32-64 KB prompt bucket: tail 720 →
    bookend 728 → **per_chunk 1,660** (2.3× longer answers)
  - `WARRANT_MISSING` stable across all three (~28-30), so the
    issue is **claim-volume inflation**, not deflection or
    warrant failure.
- **Per-prompt-size bucket** (claim_lattice, n_runs in brackets):

  | bucket   | tail [n] | bookend [n] | per_chunk [n] |
  |----------|----------|-------------|---------------|
  | 16-32 KB | 0.40 [141] | 0.41 [141]  | 0.34 [138]    |
  | 32-64 KB | 0.42 [81]  | 0.46 [81]   | 0.26 [84]     |

  Bookend nudges +4pp on the largest-prompt bucket (marginal,
  ~1.5σ). Per_chunk collapses by −16pp on the same bucket.
  Layout's aggregate effect is opposite to what the n=1
  Ballestrini anecdote suggested.

### Honest read

The Ballestrini case (small-model evidence-negation under
long-context tail) is a **real failure mode but rare in the
curated set**. Most questions don't trigger that specific
attention failure, so layout fixes don't move the aggregate
needle. `per_chunk` fixes the rare case at the cost of ~10pp
aggregate STRICT — a bad trade as a default. `bookend` is a wash.

**Decision**: keep `tail` as the default. Ship the policy field
as an operator opt-in. The Ballestrini case becomes a regression
fixture in `bench/qa_questions.txt`. Two follow-up options:

1. **Adaptive routing**: gate `per_chunk` behind the
   broad-quantifier classifier (`arborist/qa/quantifier.py`) so
   only list/extraction queries see the per-chunk reminder,
   narrow factoids stay on `tail`. The classifier already exists
   and folds into `governance_policy_hash`.
2. **Filtered re-bench**: run the same n=3 sweep on the
   "entity list" + "broad descriptive" + "bounded universals"
   subsections of `qa_questions.txt` only. Confirms whether
   `per_chunk` is a net positive *when the query shape actually
   matches* the failure class it targets, before wiring (1).

(1) without (2) risks installing a class-gated layout that helps
the gated class but hasn't been measured net-positive even
within the gated class. (2) is the prudent gate before (1).

## Caveats

- **`per_chunk` broadens recall on small models — and that
  broadening is the regression.** The per-chunk reminder activates
  attention on every evidence chunk, including chunks that are
  semantically unrelated. Good for "the answer is in chunk 7 of 8"
  cases; harmful when chunks 4–6 are off-topic. The Rule 8
  title-mismatch verifier check catches the worst of these (drops
  STRICT → HYBRID), but the answer text still grows longer and more
  digressive. The 2026-05-27 bench confirms this is a net negative
  on the curated set.
- **Quote-mode (`answer_mode=quote`) has no per-chunk boundary.**
  The body is the flat `Sources: ...` context string. `per_chunk`
  falls back to `bookend` in quote mode (same `format_user_payload`
  helper, `per_chunk_marker=None` for the runner's single-document
  path or `per_chunk_marker="=== Source: "` for query's multi-source
  context — see `arborist/qa/prompts.py`).
- **Cache scope.** Flipping the layout changes
  `governance_policy_hash` → changes `cache_key` → all prior
  cached answers are bypassed on lookup. The corpus (`documents`,
  `chunks`, FTS5 index, audit chain) is content-addressed and
  unaffected. Re-ingestion is a no-op (same `document_root` →
  idempotent upsert).
- **Verifier untouched.** `verifier_policy_hash` does not depend on
  `user_payload_layout`. The same hard checks (quote / span /
  entity / paraphrase verification, Rule 8 title-relevance, Rule 9
  subject-tokens-absent, claim-count ceiling) run regardless of
  layout. Layout is a model-input knob, not a verification knob.

## Verifier-blind missed-answer class

The Ballestrini failure under `tail` is not a hallucination. It
is a **false negative**:

```
Evidence contains the answer.
Model says the evidence does not contain the answer.
Verifier sees no unsupported positive claim → marks run clean.
User receives a false negative under EVIDENCE-WARRANTED.
```

This is a verifier-blind class. The existing layered verifier
(quote / span / entity / paraphrase + Rule 8 / Rule 9 / claim
ceiling) guards against *unsupported positive claims* — it has no
hook for *unsupported absences*. Layout fixes the attention placement
that produced this specific instance, but layout alone cannot
close the class — a sufficiently large prompt or an adversarial
phrasing can resurface the failure under any layout.

### Companion missed-answer guard — Ticket #000068 (shipped 2026-05-27)

Shipped as a deterministic sidecar in `arborist/qa/inspect.py:
diagnose_missed_answer`. Fires only when all three clauses hold:

```
(A) answer contains a denial/absence pattern (sealed v1 list:
    "not mentioned", "not provided", "the evidence does not say",
    "does not mention", "no specific", "no evidence", "cannot
    determine from the provided evidence", "is not stated",
    "is not specified")
(B) question is extraction/list shape (broad-quantifier classifier
    intensity ∈ {ALL, COMPREHENSIVE, OPEN_REQUEST, MANY, PLURAL}
    OR surface cue match: "songs by" / "works by" / "who wrote" /
    "what year" / "list of" / "name all" / …)
(C) evidence contains candidate spans (quoted strings / title-case
    spans / comma-list items / year / date) within a proximity
    window (default 600 chars) of CLEANED subject tokens (cue and
    relation words like "songs", "by", "who" stripped before
    proximity matching — without this the guard false-triggers on
    generic title-case spans near the question)
```

Output is `result["answerability"]` — None when the conjunction
doesn't fire, else a structured dict:

```
diagnostic_version            "missed-answer-v1"
answerability_warning         True
confidence_class              "weak" | "medium" | "strong"
triggered_clauses             {denial, extraction_shape,
                               candidate_proximity}
denial_pattern_matched        "not mentioned"
extraction_cue_matched        "songs by"
extraction_shape              "list" | "single_entity" | "open_list"
answer_type                   "title_like" | "person" | "date"
subject_tokens                ["veronica", "ballestrini"]
candidate_count               total candidates found
missed_answer_candidate_spans top-10 with offset_start / offset_end /
                              offset_basis="evidence_object_text" /
                              candidate_kind / nearest_subject_token /
                              subject_proximity_chars
```

Phase 2 measurement (`bench/qa_results/phase2-sidecar-on/2026-05-
27T14-16-22Z`): 228 runs, Hermes-3-8B, claim_lattice, tail layout.
**2/228 fires (0.88%), both strong confidence, both the Ballestrini
regression fixture. Precision 100%, zero false positives across the
226 non-Ballestrini runs.**

Phase 3 opt-in demote — `--demote-on-missed-answer` on `arborist
query`/`ask`. When set, the renderer demotes
`EVIDENCE-WARRANTED → EVIDENCE-MISSED-PARTIAL` on lattice modes for
strong/medium-confidence fires; lower rungs and non-lattice modes
get a `· missed-answer` tail tag. Default OFF; Phase 4 default flip
NO-GO until wider bench + human spot-check.

Discipline:

- **Sidecar, not verifier hook.** Same pattern as
  `arborist.qa.inspect.diagnose_*` (deflection / coherence /
  title-relevance) — read-only, never writes `providence_cache`
  or `audit_events`.
- **Hash partitioning:** sidecar-mode fields
  (`answerability_sidecar_enabled`, `answerability_threshold`,
  `denial_patterns_version`, `extraction_cues_version`) fold into
  `governance_policy_hash` only. The Phase 3 demote flag
  (`answerability_demote_enabled`) ALSO folds into
  `verifier_policy_hash` because flipping it changes the rendered
  audit_mode — a verifier-output property, so the verifier hash
  legitimately partitions.
- **Never promotes claims.** The trigger conjunction makes claim
  promotion structurally impossible — guard only fires on denial
  answers.

5F-Falsification fixture: the Ballestrini case is exactly the kind
of failure that selects for adding a new falsifier into the
substrate, then propagates the new fixture forward. The fixture is
in `bench/qa_questions.txt` under "entity list" with a 4-line
context comment pointing back here.

## Usage

```bash
# Per-call override
arborist query --user-payload-layout bookend "songs by veronica ballestrini"
arborist query --user-payload-layout per_chunk "songs by veronica ballestrini"

# Make
make query Q="songs by veronica ballestrini" LAYOUT=bookend BURN=1
make query Q="songs by veronica ballestrini" LAYOUT=per_chunk BURN=1

# Policy override in Python
policy = dict(DEFAULT_QUERY_POLICY)
policy["user_payload_layout"] = "per_chunk"
```

## Implementation map

- `arborist/qa/prompts.py` — `format_user_payload(question, body, *,
  layout, evidence_label, question_label, per_chunk_marker)`. Single
  source of truth; raises on unknown layout. `USER_PAYLOAD_LAYOUTS`
  constant lists valid values.
- `arborist/qa/query.py` — multi-source retrieval path. Three
  `_user_payload` closures (one per answer_mode) all call
  `format_user_payload`. Policy default added to
  `DEFAULT_QUERY_POLICY`.
- `arborist/qa/runner.py` — single-document path. Three
  `_user_payload` closures all call `format_user_payload`. Policy
  default added to `DEFAULT_POLICY`.
- `arborist/cli.py` — `--user-payload-layout` flag on both `query`
  and `ask` subcommands; flows into `call_policy`.
- `Makefile` — `LAYOUT=` plumbing on `make query`.

## Future hardening (not yet implemented)

These are followups flagged in the 2026-05-27 de-novo review. Not
required to ship the opt-in policy knob, but worth doing before
any default promotion or adaptive routing lands.

- **Bounded reminder text.** `per_chunk` currently echoes the
  question verbatim before each evidence block. Long or
  adversarial questions can bloat the prompt and widen the
  instruction-injection surface. Suggested signature extension:

  ```python
  def format_user_payload(
      question: str,
      body: str,
      *,
      layout: str = "tail",
      ...
      max_reminder_chars: int = 512,       # truncation cap
      reminder_mode: str = "text",         # "text" | "qid"
  ) -> str: ...
  ```

  `reminder_mode="qid"` would emit `[for qid=<question_hash>]` —
  preserves per-chunk anchor identity without re-inserting
  arbitrary user text N times. Trade-off: fewer lexical attention
  tokens. Worth A/B-benching against `reminder_mode="text"` on the
  filtered list-shape subset.

- **Structured evidence-block injection.** Current `per_chunk`
  implementation uses `body.replace("\n\n=== ", ...)` — works
  because `render_evidence_map` is deterministic, but a structured
  alternative (`format_user_payload(..., evidence_blocks=[...])`)
  would be more robust to upstream rendering changes. Adopt if/when
  the evidence map's block boundary convention changes.

- **Companion missed-answer falsification guard** — shipped as
  Ticket #000068 Phase 1+2+3. See §"Companion missed-answer guard"
  above and `docs/tickets/ticket-000068-*.md`.

## Roadmap

Phased gating — current state in **bold**.

```
Phase 0 — ticket finalization
   Open layout ticket. Open companion missed-answer guard ticket
   (#000068).
   Default = tail.

Phase 1 — implementation
   **DONE 2026-05-26 (commit 5674107).** format_user_payload helper,
   six callsites, CLI + Make plumbing, governance_policy_hash
   partitioning, USER_PAYLOAD_LAYOUTS constant, unknown-layout
   raises ValueError.

Phase 2 — regression fixtures
   **DONE 2026-05-27.** Ballestrini case added to bench/qa_questions.txt
   under entity-list section with 4-line context comment.
   Pending: The Veronicas disambiguation fixture, genuine-absence
   fixture, quote-mode fallback fixture, governance/verifier hash
   partitioning unit tests.

Phase 3 — bench
   **DONE 2026-05-27.** Three-layout sweep on curated 75-question
   set, claim_lattice mode, n=3, Hermes-3-8B. Result: tail/bookend
   are within noise; per_chunk regresses −9.78pp aggregate. See
   §"Bench results". Qwen-27B sanity check still open.

Phase 4 — selection
   **NO-GO on default promotion.** Tail remains default.
   Pending decision: adaptive routing via the broad-quantifier
   classifier (per_chunk only on list/extraction queries), gated
   by a filtered-subset re-bench to confirm net positive within
   the gated class.

Phase 5 — missed-answer guard (Ticket #000068)
   **DONE 2026-05-27.**
   - #000068 Phase 1 (sidecar): commit 2ab11d2. diagnose_missed_
     answer in arborist/qa/inspect.py with the three-clause
     conjunction, 36 tests, end-to-end live verified on Ballestrini.
   - #000068 Phase 2 (bench): commit ec55db5. bench/qa_sweep.py
     instrumented to surface answerability fires aggregate; 228-run
     bench at bench/qa_results/phase2-sidecar-on/ shows 2 fires,
     both Ballestrini, both strong confidence, 100% precision.
   - #000068 Phase 3 (opt-in demote): commit ec55db5. --demote-on-
     missed-answer flag on `arborist query`/`ask`, default OFF.
     answerability_demote_enabled folded into _VERIFIER_POLICY_FIELDS;
     renderer demotes EVIDENCE-WARRANTED → EVIDENCE-MISSED-PARTIAL
     for strong/medium fires.
   - #000068 Phase 4 (default demote-on): **NO-GO** until wider
     bench + human spot-check confirms low FP rate across diverse
     failure shapes. Per Dav1d 2026-05-27 §3.4: "a false sidecar
     warning is tolerable; a false audit-label demotion can damage
     trust in correct abstentions."
```

## Related

- `docs/qa-modes-bench.md` — three answer modes (quote /
  claim_lattice_pointer / claim_lattice). Layout is orthogonal to
  mode and folds into the same `governance_policy_hash`.
- `docs/bench-maxing.md` — bench discipline. The 2026-05-27 sweep
  applied the 5pp signal floor against n=3, and the verdict (no
  default promotion) followed directly from that rule.
- `docs/seven-point-program.md` — north-star directives. Layout is
  a D6/D7 (verifier discipline + small-model robustness) follow-up,
  not a new directive. The missed-answer guard (#000068, shipped
  2026-05-27) extends **D2 (pointer-grounding)** by covering
  *unsupported absence* alongside *unsupported presence* — same
  falsifier discipline, new failure shape.
- `docs/tickets/ticket-000068-verifier-blind-missed-answer-guard.md`
  — sibling ticket. Layout is the model-input lever; #000068 is the
  verifier-side adjunct. Read together for the full picture of the
  Ballestrini failure and the substrate's response to it.
- 2026-05-27 de-novo reviews by Dav1dPrometheus (private working
  docs at `~/Downloads/response_-_user-payload-layout.txt` and
  `~/Downloads/RESPONSE_final_ticket-000068-verifier-blind-missed-
  answer-guard.txt`) — split this work into two tickets: layout
  (this doc) + companion missed-answer guard (#000068). Verdict,
  errata, hardenings, and roadmap phasing above incorporate both
  reviews.
