# Ticket #000056 — Operation Sandwich (cross-language grounding via query+display MT)

**Status:** implemented & landed 2026-05-17 — mechanism + bright
line verified live end-to-end (§8); 6 tests + full suite 2477
passed. **Default OFF** (`crosslang_translate_enabled`, gated under
Phase-0 `crosslang_guard_enabled`). NOT "cross-language solved":
the live flagship run is still `UNGROUNDED` (opus-mt
output↔title-normalisation recall gap, §8 limit 1) — recall tuning +
JSON-display polish (§8 limit 2) + SHA-pinning the manifest are open
follow-ups behind the experiment flag; a bench-gated default-flip is
a fox decision. Phase 1 of the #000001 §7 family.
**Opened:** 2026-05-17
**Asked by:** fox ("yes call it operation sandwich create a new
ticket and finish it") — closing the chain that started with the
2026-05-16 `¿Qué es el anarcocapitalismo?` field case.

**Why a new ticket (clears the don't-proliferate bar).** CLAUDE.md
default is *extend*, split only when the piece needs a **distinct
Dav1d-reviewable audience** or is an **architectural inflection**.
Operation Sandwich is both: it introduces a **model dependency**
(`[mt]`) and a **presentation-translation layer** — a self-contained
design decision a de-novo review must read independently of #000001's
retrieval-keywords provenance story. #000001 §7 explicitly anticipated
this split ("If the MT model choice becomes large, split the
model-distribution work into a later `[mt]` ticket, similar to
`[nli]` and vecpack"). #000001 §7 stays the Phase-0 home; this is
Phase 1. Cross-linked both ways.

**Audience:** fox + a Dav1d de-novo review of the
translation-in-the-pipeline boundary + maintainers of the verifier
proof path.

---

## 1. The sandwich

```
Spanish query ─▶ [MT es→en] ─▶ English retrieval + English answer + EXISTING verifier ─▶ [MT en→es] ─▶ Spanish display
                 translator IN        ←—— grounded core, byte-for-byte untouched ——→        translator OUT
                 (retrieval-side)      (English-vs-English — the proof)                    (presentation-only)
```

Translation lives on the **two edges**, never the middle. The
verifier, `audit_mode`, `cache_key`, `question_hash`,
`governance_policy_hash` all see only the English core. The Spanish
the user reads is a labelled rendering carrying **zero** grounding.

## 2. Hard constraints (the bright line, as code invariants)

1. **`answer_text` stays the grounded English string.** The verifier
   runs English-answer-vs-English-source, unchanged byte-for-byte.
2. **The Spanish rendering goes in a NEW field** (`display_answer`,
   `display_lang`, `display_translated=True`,
   `display_unverified_banner`). It is never the verifier's input,
   never what `audit_mode`/STRICT attaches to, never hashed into the
   proof. Same discipline as `cli._render_audit_label` (render
   projection ≠ proof).
3. **Query MT is retrieval-side, == `--retrieval-keywords`.** It
   changes which sources rank (→ `context_root` → `cache_key`
   indirectly, exactly the #000001 §5/§6 story), **not**
   `question_hash`. The MT engine identity binds into the **run-DAG
   retrieval stage** via the existing `retrieval_plan_hash` (the
   §7.3-deferred `RetrievalPlan` extension lands here).
4. **No translated answer ever re-enters the verifier** (the
   forbidden path / #000049 model-in-proof-path cage). Display MT is
   output-only and one-directional.
5. **Default OFF**, gated under `crosslang_guard_enabled`. The MT
   engine is the optional `[mt]` extra; absent → degrade to
   `available=False` → the sandwich silently no-ops (query proceeds
   as Phase-0). English path byte-identical by construction.
6. **Hash invariants (corrected 2026-05-17 — artifact over my own
   prose).** `governance_policy_hash` is `sha256` of the *whole*
   policy dict (`arborist/qa/keys.py:182`), so — like **every**
   policy flag (quantifier, metacognition, soft-preflight) — the
   crosslang flags DO move it. An earlier draft of this section
   wrongly said "governance untouched"; that was false and is
   corrected here. The behaviour is *correct*: a sandwich-on answer
   must not be served to a sandwich-off lookup — the cache partitions
   by config, it does not leak. The load-bearing invariants are the
   two that ARE untouched: **`question_hash`** (the user's Spanish
   question is preserved verbatim — MT never rewrites what was asked)
   and **`verifier_policy_hash`** (the verifier is byte-identical;
   the crosslang flags are absent from `_VERIFIER_POLICY_FIELDS`).
   The MT *engine identity* binds into the run-DAG retrieval plan
   (`RetrievalPlan.mt_*`), which is not a policy hash. Test:
   `test_sandwich_hash_invariants`.

## 3. Engine — local `[mt]` extra (fox-decided 2026-05-17)

`opus-mt-es-en` (query in) + `opus-mt-en-es` (display out),
Helsinki-NLP, Apache-2.0, hash-pinned by HF revision, weights
off-repo under `~/.arborist/models/mt/`, optional dependency. **Not
Hermes-3-8B** (blackops rule: 8B unfit for human-language
translation). **Not an external API**: reproducibility (a pinned
checkpoint replays byte-identically — provenance-critical for a
Merkle-replay system) + zero egress (Operation Voyeur / sovereignty)
+ es↔en is the best-resourced MT pair so the local-vs-frontier
quality gap is smallest exactly here. Grok stays the documented
future bench-comparison candidate (blackops: "evaluate before
committing spend"), not the v1 substrate. Mirrors the `[nli]` /
`ShadowNLI` pattern (#000049) and the vecpack off-box pattern
(#000051) exactly — zero new architectural surface.

## 4. Build

- `arborist/qa/mt/` — `manifest.json` (pinned es-en + en-es repos +
  revisions), `Translator` protocol, `OpusMTTranslator` (lazy
  `transformers` import, `available` flag, graceful degrade — the
  `ShadowNLI` template verbatim), `StubTranslator` (deterministic;
  the test substrate **and** the safe fallback when the extra is
  absent — identity passthrough, so the sandwich degrades to Phase-0
  behaviour, never raises).
- `[mt]` extra in `pyproject.toml`.
- `arborist/qa/query.py` — under
  `crosslang_guard_enabled AND crosslang_translate_enabled` AND the
  Phase-0 non-English signal fired AND translator `available`:
  es→en the query → `retrieval_query` **and** the LLM prompt
  (`llm_question`, so the model answers in English; `question_hash`
  still uses the original Spanish); after the verified English
  result, en→es the model **prose** (`raw_answer`, never the rendered
  evidence scaffold — pinned verbatim English spans must not be
  MT-ed) → `display_answer` (+ banner). Extend
  `RetrievalPlan` with `mt_engine` / `mt_manifest_hash` /
  `source_lang` (omitted-when-empty so existing `retrieval_plan_hash`
  values do not churn — the §5 zero-churn discipline).
- CLI `--crosslang-translate`; `make query XLANG_MT=1`;
  `tasks.py query XLANG_MT=1`.

## 5. Acceptance criteria

1. Sandwich path (StubTranslator): es query → en `retrieval_query`
   (asserted), English `answer_text` verified by the **unmodified**
   verifier, `display_answer` is the es rendering, banner present.
2. `question_hash` and `verifier_policy_hash` **identical** with the
   flag on vs off (the load-bearing invariant: user-question identity
   preserved, verifier byte-identical). `governance_policy_hash`
   **differs** (whole-policy hash — like every policy flag; correct
   cache partitioning, not a leak). Test:
   `test_sandwich_hash_invariants`.
3. `RetrievalPlan` MT fields bind into `run_dag` (different engine →
   different `retrieval_plan_hash`); empty MT fields → **unchanged**
   hash (no churn on non-MT runs).
4. The translated Spanish text is **never** the verifier's input —
   `answer_text` (English) is finalised and verified before display
   MT runs; the rendered evidence scaffold (pinned verbatim English
   spans) is never fed to MT (display translates `raw_answer` prose).
5. Default OFF: `crosslang_translate_enabled=False` → no `display_*`,
   pure Phase-0 behaviour. English query → byte-identical (signal
   None → translator never consulted).
6. `[mt]` absent → `OpusMTTranslator.available is False` → sandwich
   no-ops to Phase-0, never raises (graceful-degrade test).
7. The translated answer is **never** passed to `verify_*`
   (asserted structurally — the verifier input is `answer_text`,
   set before display MT runs).

## 6. Out of scope

Languages beyond es-v1; the Grok bench-comparison; an
`audit_mode`-affecting strict mode that folds MT into
`governance_policy_hash` (the #000001 §6 optional-strict analogue);
auto language *detection* beyond the Phase-0 deterministic signal
(accent-free Spanish still slips — a detector is future work).

## 7. Status

Implementation landed 2026-05-17 — see §8.

## 8. Landing — what works, and the honest limits

**Mechanism: complete and verified live end-to-end.** A real run
(`arborist query --crosslang-translate --json "¿Qué es el
anarcocapitalismo?"` against live Hermes + the real `opus-mt`
engine, which loads in this env via the `[nli]`-shared
`transformers`/`torch`):

- query translated es→en, LLM prompted in English, **`answer_text`
  is the English answer** ("Anarchocapitalism is a political
  philosophy and economic theory that advocates the elimination of
  the state…"); the verifier ran on that English core;
- `display_answer` is the opus-mt es rendering of the verified English
  prose ("El anarcocapitalismo es una filosofía política y una teoría
  económica que aboga por la eliminación del Estado…"),
  `display_translated=True`, `display_lang=es`, `engine=opus-mt-v1`,
  banner present;
- the bright line held: the Spanish text is additive/display-only;
  `question_hash` + `verifier_policy_hash` invariant (test
  `test_sandwich_hash_invariants`); `RetrievalPlan` MT binding is
  zero-churn on non-MT runs (`test_retrieval_plan_mt_fields_*`).
- 6 sandwich tests + full suite **2477 passed**, 0 regressions;
  `make`/`tasks.py`/CLI surface (`--crosslang-translate` /
  `XLANG_MT=1`) wired; default OFF.

**Two honest limitations the live run exposed (NOT solved here):**

1. **Recall is not yet good.** `opus-mt-es-en` rendered
   `anarcocapitalismo` → "Anarchocapitalism" (one token, no hyphen),
   which still did **not** rank the canonical *Anarcho-capitalism*
   article primary — the live run was `UNGROUNDED` with off-target
   sources (*Consensus decision-making*, *Karl Hess*, …). The
   architecture is correct; the MT-output↔title normalization /
   retrieval interaction needs tuning. This is exactly the experiment
   the **default-OFF flag + bench gate** exist for, and why the
   manifest revisions are placeholder (`main`, not SHA-pinned). "The
   sandwich works" means the *mechanism*; it does **not** mean
   cross-language grounding is solved.
2. **JSON/lattice display is rough.** In `claim_lattice` (JSON) mode
   `raw_answer` is JSON, so display-MT translates the envelope
   (`"declaraciones"`). Prose/quote mode renders clean. JSON-mode
   display polish (translate field *values* only, or render from the
   parsed claim list) is future work — banner-labelled and default-OFF,
   so acceptable as an experimental edge.

**Doc honesty correction (2026-05-17).** Earlier drafts of §2 #6 /
#000001 §7 said "governance untouched" — false: `governance_policy_hash`
is sha256 of the *whole* policy (`keys.py:182`), so the crosslang
flags move it like every policy flag (correct cache partitioning).
Corrected in §2 #6 (authoritative), #000001 §7, and both index rows;
artifact-over-instruction.

## 9. Fan-out benchmark (2026-05-17) — measured, with the dead ends named

**Repro (deterministic, no egress, no Hermes-for-translation).**
`bench/make_lang_questions.py <lang>` regenerates the per-language
question set from `bench/qa_questions.txt` via `opus-mt-en-<lang>`
(the sandwich's own engine); `bench/qa_sweep.py … --policy
crosslang_translate_enabled=true [--policy crosslang_entity_mask=…]
[--policy crosslang_source_lang=…]` runs the live sweep;
`bench/es_delta.py <en_baseline.jsonl> <xx_sandwich.jsonl>` is the
authoritative metric (per-question EN→xx transition).
`es_roundtrip_analysis.py` / `es_join_patterns.py` produced the
*refuted* metric below — kept only as the cautionary record.

**The real measurement (n=1, claim_lattice, same 75 questions).**
EN baseline **85 %** grounded (64/75) → es+sandwich (no mask)
**71 %** (53/75): the sandwich costs **−14 pp**. Transitions:
PRESERVED 31 · DOWNGRADE S→H 16 · **LOST 17** · N/A 11 (6 of which
*gained*).

**Dead end (named, not buried).** A deterministic es→en round-trip
content-token-recall metric (CLEAN 77 % / DRIFT 20 % / COLLAPSE 3 %)
looked like a sharp predictor of grounding loss. It is **not**:
against the live delta, CLEAN grounds 71 % and DRIFT 73 % (no
separation), and **12 of the 17 LOST questions round-tripped CLEAN**.
The metric is abandoned. This is another instance of the
already-codified CLAUDE.md bench-maxing standing lesson (a clean
synthetic metric ≠ real bench-qa behaviour — #000049 §7); **not**
re-added to CLAUDE.md (DRY — it is already there).

**The LOST 17, from the artifact (not a hypothesis):** ≈7 *entity
translated/garbled* — `New London`→"nuevo Londres", `Tarsus`→"Tarso",
**`Boltzmann`→"perntzmann"** (opus-mt hallucinated the name),
`World War 2`→"Segunda Guerra Mundial"→"Second World War" (≠ corpus
"World War II"); ≈3 *broad-enumeration* shapes ("name all members of
the beatles", "list all planets") — a quantifier-guard×translation
interaction, separate lever; ≈several *n=1 stochastic* (`founder of
microsoft?`, `what continent is egypt on?` round-tripped clean,
entities intact, yet LOST — needs n=3 to separate).

**Lever built + validated (the entity slice).**
`arborist/qa/mt/entity_mask.py` — deterministic mask/restore (quoted
spans + Capitalised runs; corpus-title anchoring deferred to v2),
`MaskedTranslator` wrapper, default-ON within the (default-OFF)
sandwich, `crosslang_entity_mask` toggles it for the A/B. Measured
through **real opus-mt** round-trip: `who is Paul of Tarsus?` base →
"Pablo de Tarso" (the exact LOST case) → **+mask → "Paul of
Tarsus"**; `Boltzmann` is now garble-proof (never reaches MT). 5
tests.

**Honest caveat — the bench under-states the lever.** The bench
questions are *lowercased* (`a bridge between new london…`), so the
Capitalisation detector cannot fire on the en side; on the xx→en
edge it still catches `Egipto`/`London` etc. So any lift on *this*
bench is a **lower bound** — real cased user input does strictly
better. The detector that catches lowercase entities is
**corpus-title anchoring** (protect any query span matching a known
article title), deferred to v2.

**Also landed:** the #000056 fan-out caught a real defect —
`get_translator()` built a fresh ~300 MB model per call → at
concurrency 4 the first sweep was **88 % engine-ERROR**
(`meta tensors` race). Fixed: process-level memoised singleton +
lazy *per-pair* load (6 manifest pairs never load eagerly) + load
lock; 6 concurrent threads → 0 errors. French (`opus-mt-fr-en/-en-fr`)
and Russian (`ru`) added as second/third bread (manifest +
`crosslang_source_lang`); fr question set generated (`GNU Linux`
survives). [Tasks #14–#17]

**Lift result (2026-05-17, measured — and it refutes the lever).**
n=1, claim_lattice, 75 q, vs EN baseline 64/75 (85 %):

| config        | grounded   | Δ vs EN | LOST |
|---------------|------------|---------|------|
| EN baseline   | 64 (85 %)  | —       | —    |
| es, no mask   | 53 (71 %)  | −14 pp  | 17   |
| es, +mask     | 49 (65 %)  | −20 pp  | 22   |
| **fr, no mask** | **46 (61 %)** | **−24 pp** | **22** |
| fr, +mask     | 35 (47 %)  | −38 pp  | 33   |

**Corrected read (fr no-mask measured 2026-05-18):** French was
*never* 47 % — that figure was the *failed mask*. fr no-mask is
**61 %**. The earlier "French much worse / −38 pp" was a mask
artifact; measuring it honestly fixed a wrong impression (this is
why "firm the numbers" was non-optional). Net: the sandwich grounds
**61–71 % of foreign queries where the pre-ticket baseline was
≈0 %**, in *two* languages; the entity-mask lever is net-negative in
*both* (es −4 q borderline, fr −11 q / −14 pp unambiguous) — dead as
built, correctly default-OFF.

**Entity-mask did not help — it is neutral-to-negative.** The
isolated `Paul of Tarsus` win did **not** replicate at bench scale:
es went 53→49 (−4 q ≈ the 5 pp n=1 noise floor — *ambiguous*, not a
confirmed regression but certainly no lift). fr is unambiguous and
bad (−38 pp, well past noise). My earlier "lever validated" claim
was premature; the bench refutes it. **Third instance this thread of
the CLAUDE.md bench-maxing lesson: an isolated/synthetic win ≠
bench-qa behaviour.** Likely causes (untested): the lowercased bench
gives the mask almost nothing to grab on the en side, so on the
xx→en edge it fires on *Spanish/French* capitalisation (which is not
English entity structure) and the sentinels perturb opus-mt's
translation of the rest of the sentence — net harm, not help.

**Honest standing verdict (comparator corrected 2026-05-17, fox).**
The right baseline is **not** native English — it is what
cross-language did *before this ticket*: the 2026-05-16
`¿Qué es el anarcocapitalismo?` run retrieved Spanish-song-title
noise, burned 10.4 s, returned UNGROUNDED — **≈0 % grounded**.
Against that true baseline:

- **The sandwich is a large net win: ~0 % → 71 % grounded (es)**, with
  the proof core never corrupted. The −14 pp vs *English* is the
  *cost of a capability that previously did not exist at all*, not a
  regression. Calling that a "fail" was a comparator error — doom-
  framing is as dishonest as hype.
- **The entity-mask *lever* is the genuine negative:** net-negative
  at scale (es 71→65, fr −38 pp), so it is now **default-OFF**
  (`crosslang_entity_mask=False`); the no-mask sandwich is the
  better config and the one worth keeping.
- French is materially weaker than Spanish (fr+mask 47 %; fr no-mask
  unmeasured — likely higher since mask hurt es). Still ≫ the ≈0 %
  baseline.

So: **keep the sandwich (clear win over no support); drop the mask
default (failed lever); the −14 pp-vs-English is a discount on a new
capability, not a defect.** Remaining: n=3 to firm es/fr numbers,
fr no-mask measurement, corpus-title anchoring (the one untried
detector that works on lowercase). Default-OFF as an *experiment
flag*, but the experiment is now **promising, not failing**. fox
decides next spend; the honest recommendation flipped — this is
worth continuing, just not with the mask.
