JSON mode previously presented content-addressed evidence_ids
(``Eed1b6e396``-style) in the EVIDENCE block headers and expected
the same in the model's claim output. Hermes-3-8B was fabricating
plausible near-miss IDs (e.g. ``E1b6e396`` when the runtime had
``Eed1b6e396``) on cross-document relationship questions →
UNKNOWN_EVIDENCE_ID → UNGROUNDED, even when the answer text was
factually correct (e.g. "Homer Simpson's boss is Mr. Burns.").
Switching the prompt-facing surface to short pointer IDs (``E1``,
``E2``, …) — same as claim_lattice_pointer mode — closes the
hallucination loop:
- Pointer IDs are short, enumerable, and fabrication-obvious.
The model can't invent ``E27`` if only ``E1``-``E10`` were shown;
out-of-range IDs read as schema violations at first glance.
- The runtime still resolves each pointer_id to its content-
addressed evidence_id internally and stores THAT in
``evidence_id_pairs`` for the cache & run-DAG. Cache_keys stay
run-stable; only the prompt-facing string changes.
- JSON schema unchanged (``evidence_ids: [str, ...]``), so vLLM
guided_json continues to constrain output shape.
Live-test verification: ``who is homer simpson's boss?`` went from
JSON-mode UNGROUNDED 0/1 (hallucinated ``E1b6e396``) to STRICT 1/1
(model emits ``E1``, resolves cleanly). Homer fixture pin removed —
runs against the JSON default now.
Files touched:
- aborist/qa/evidence.py: render_evidence_block_for_json uses
e.pointer_id instead of e.evidence_id.
- aborist/qa/verify.py: verify_claim_lattice_json switched from
evidence_map_by_evidence_id to evidence_map_by_pointer_id;
captures both pointer_ids (model-emitted) and evidence_ids
(run-stable) in claim_statuses + evidence_id_pairs.
- aborist/qa/{runner,query}.py: claim_lattice_json_system_prompt
+ grounding_reminder describe pointer IDs; example shifts from
``E........`` placeholder to ``E1``.
- tests/test_verify_json.py: stub _ev() takes pointer_id; four
fixtures updated to set it.
- tests/test_qa_quality_live.py: Homer fixture unpinned (now
runs default JSON mode and grounds). Red-fish-blue-fish
fixture pinned to pointer mode — JSON mode hits a separate
token-budget runaway on "plot of X" prose-summary shapes
(~2/3 of samples blow max_tokens with whitespace spam after
the closing brace). Different failure mode, addressed in a
later commit.
460 unit tests + 11 live fixtures pass.
russell@unturf.com
bb8450d402