# Reference-frame failure class — the Orwell case

**Date:** 2026-05-01
**Scope:** Worked example documenting a failure mode in retrieval +
phrase-pattern boost as the response.
**Audience:** fox + future blackops shifts.
**Status:** investigation log; the phrase-pattern route landed in
the same commit window. Not a ticket — this is a journal entry.

---

## What happened

Run 1 (no augmentation):

```
make query Q="has oceania always been at war with east asia"
  HYBRID 2/16  via claim_lattice
  sources: Oceania · Asia · Outline of Oceania · Far East · ...
```

The system selected the **literal geography frame** — answered as
if asked about real-world Oceania and East Asia regions. The
Nineteen Eighty-Four article was nowhere in the top-K despite
existing in shard `003.db`.

Run 2 (manual augmentation — `K=` flag):

```
make query Q="has oceania always been at war with east asia? do you understand what this reference"
  STRICT 1/1  via claim_lattice
  sources: Nineteen Eighty-Four (top result)
```

Adding the literal word "reference" to the question pushed the
retrieval into the right map. The user solved it themselves with
operator hints — but a high-quality system shouldn't need those
hints for diagnostic phrases.

## The diagnosis

The cause was retrieval-shape, not model behavior:

```
Q tokens (FTS5-tokenized, no stem): {has, oceania, always, been, at, war, with, east, asia}
Q content tokens (stopword-stripped): {oceania, war, east, asia}
```

For these content tokens:

| Article                    | spin | glass | modeling | tensors | distinct match |
|----------------------------|------|-------|----------|---------|----------------|
| Nineteen Eighty-Four       | 0    | -     | -        | -       | (no overlap on title-tokens at all) |
| Oceania                    | many | -     | -        | -       | 1 token in title (oceania) |
| Foreign relations of Axis  | -    | -     | -        | -       | 0 |

The **diagnostic signal** wasn't in the content-token overlap. It
was in the verbatim 5-token phrase `oceania always been at war`
which appears in the 1984 article's body but nowhere else.

Pre-2026-05-01 retrieval had three routes (body BM25, title-LIKE,
core-keyword) — none of them captured "find articles whose body
contains a verbatim multi-token sequence from the question." So
the 1984 article was effectively invisible to the search.

## The fix — phrase-pattern retrieval route

`aborist/qa/query.py:_search_phrases` adds a fourth retrieval
route that runs an FTS5 quoted-phrase MATCH for each n-gram
extracted from the question:

```python
_question_phrases("has oceania always been at war with east asia", n=5)
# → ["has oceania always been at",
#    "oceania always been at war",  ← diagnostic Orwell signal
#    "always been at war with",
#    "been at war with east",
#    "at war with east asia"]
```

Two pass-throughs — n=6 (specificity 100) and n=5 (specificity 90).
4-grams were tried and rejected: "always been at war" matches generic
war-history articles too noisily for the rerank pipeline to separate.
5+ tokens trade recall for precision; most allusions survive at
length 5.

**`_filter_by_title_relevance` accept-path 4** lets phrase-route
hits bypass the title-token-overlap gate. The 1984 article's title
shares zero tokens with the question — without accept-path 4,
phrase-route candidates would be retrieved and immediately filtered
out before they could rerank into top-K.

## Result

After the fix:

```
make query Q="has oceania always been at war with east asia"
  EVIDENCE-LINKED · via claim_lattice  1/1  11.5s  (fresh)
  - The text does not directly state that Oceania has always been at
    war with East Asia. The passage describes a change in alliances,
    where Oceania switched from being allies with Eastasia to being
    allies with Eurasia, and the public was manipulated to accept
    this change without realizing it.
    [E13 | Nineteen Eighty-Four | 682f0a11: "...To hide such
    contradictions, history is re-written to explain that the (new)
    alliance always was so..."]
```

No operator augmentation needed. The Orwell frame surfaces from
phrase-route alone.

## Why not a "Reference Frame Router"?

An earlier sketch (in fox's review, 2026-05-01 Asia/Kuala_Lumpur)
proposed a hand-rolled `REFERENCE_PATTERNS` table mapping query
shapes ("has X always been at war with Y") to known references
(Orwell). That approach was rejected because:

- **Scaling**: allusions are long-tail. `winter is coming`,
  `the cake is a lie`, `may the force be with you`, `to be or
  not to be` — the catalog is open-ended.
- **Maintenance**: per-pattern code rots; new allusions need new
  rules.
- **The corpus already knows**: 1984 article exists in the corpus.
  The defect was retrieval not surfacing it. Fix retrieval, not
  add a new pre-retrieval stage.

Phrase-pattern boost generalizes: any verbatim 5+ token sequence
from the question that appears in an article body lifts that
article into consideration, regardless of whether it's an Orwell
reference, a Star Wars quote, a Hamlet line, or a meme.

## Bench coverage

Added to `bench/qa_questions.txt` under `# allusion / reference
frame`:

```
has oceania always been at war with east asia?
who said may the force be with you?
what does winter is coming mean?
what is the meaning of rosebud?
who said to be or not to be that is the question?
what does the cake is a lie reference?
```

A future bench sweep will reveal whether phrase-pattern boost
generalizes across these allusions or if some need different
treatment.

## Related concerns NOT addressed here

- **Source-role taxonomy** — proposed roles like
  `primary_reference_source` (cited reference work) vs
  `literal_world_background` (geography article that's tangentially
  related). Currently all retrieved sources land as
  `background_source` in the rendered output; finer-grained roles
  could improve answer framing. Out of scope for this fix.
- **Frame-assumption verifier field** — orthogonal to STRICT/HYBRID/
  UNGROUNDED labels. Records "answered as Orwell reference" vs
  "answered as geography." Useful for audit but requires verifier
  schema work; deferred.
- **Reference-aware prompting** — system prompt could acknowledge
  reference-frame ambiguity ("if the query reads as an allusion,
  prefer the reference frame when retrieved evidence supports it").
  Per fox's "less prompt engineering, more code-level discipline"
  preference, deferred until phrase-pattern boost's empirical
  performance is known.
