#000052 §3.1: diagnose_coherence — lexical answer-shape sidecar

Per-sentence shape check (no model) emitting kind ∈
{phrase_component_reuse, circular, vacuous, ok, empty}:
  - circular: subject content-tokens ⊆ predicate's and the predicate
    leads with a subject token ("Water is water").
  - phrase_component_reuse: subject quotes a phrase, predicate reuses
    one of that phrase's own tokens as a bare "the/a/an <token>"
    referent — the 2026-05-12 field case ("the phrase 'Zionist entity'
    is used as the entity"), a token collision the verifier +
    deflection + title-relevance all pass and NLI returns neutral on.
    Copulas inside a quoted span are skipped so 'war is peace' doesn't
    break the subject/predicate split.
  - vacuous: predicate is only placeholder hypernyms + filler ("X is
    a thing").

Conservative — no full token-salad parsing; legit definitions pass ok.
Surfaced in inspect_cache_key + the `arborist inspect` human view
(· incoherent: <kind>). Advisory only — never writes providence_cache
/ audit_events / run_dag_root; demote-only verifier hook deliberately
not wired. 9 tests; full suite green (2500 passed).

docs/TICKETS.md

@@ -101,7 +101,7 @@ Newest first. Update on every open/close.
 
 | ID       | Title                                          | Status                | Opened     | Directive |
 |----------|------------------------------------------------|-----------------------|------------|-----------|
-| #000052  | Relevance + coherence meta-cognition (answer-*shape* signals) | open · awaiting go/no-go · doc-only scaffold. Two more read-only, demote-only, never-in-proof-path sidecars joining the `diagnose_deflection` / `diagnose_metaphor_deflection` / `diagnose_title_relevance` / soft-preflight family: **(1) `diagnose_coherence`** — word-salad / circular ("X is X") / vacuous / underspecified answers; lexical/parse-shape, **no model** — the near-term win (recommended now). **(2) `diagnose_relevance`** — semantic (not just lexical) "aboutness": does the answer address the question; is each claim about its cited source? Today's checks (subject-anchor token overlap, stemmed title-stem overlap) are *lexical* and a token collision defeats them — a small *aboutness/reranker* model (NOT NLI — entailment ≠ topicality) under #000049 §7's discipline cage verbatim (demotion-only, hash-pinned, `relevance_model_version`→`governance_policy_hash` iff it touches `audit_mode`, shadow-first, `[…]` extra, the §7 #20 haystack lesson — never over the whole context); gated on evidence, travels with #000049's model question. Motivating field case (2026-05-12, fox): the `claim_lattice` query that returned *"the phrase 'Zionist entity' is sometimes used as the entity, referring to the State of Israel"* at `EVIDENCE-WARRANTED-PARTIAL 2/3` — incoherent + token-collision recombination that NLI can't catch (returns *neutral*, not *contradiction*) and both lexical relevance checks waved through. Flags an upstream retrieval ticket (polysemy / title-token-soup) as the root-cause fix, not scoped here. #000049 sibling | 2026-05-12 | —         |
+| #000052  | Relevance + coherence meta-cognition (answer-*shape* signals) | in progress — **§3.1 `diagnose_coherence` landed** (lexical, no model: `circular` / `phrase_component_reuse` / `vacuous`; in `arborist/qa/inspect.py`, surfaced via `inspect_cache_key` + `arborist inspect` `· incoherent: <kind>`; 9 tests; demote-policy hook deliberately not wired — advisory only). Joins the `diagnose_deflection` / `diagnose_metaphor_deflection` / `diagnose_title_relevance` / soft-preflight family of read-only, demote-only, never-in-proof-path sidecars; `phrase_component_reuse` catches the motivating field case (a subject quoting a phrase, a predicate reusing one of that phrase's own tokens as a bare `the <token>` referent). **Still open: (2) `diagnose_relevance`** — semantic (not just lexical) "aboutness": does the answer address the question; is each claim about its cited source? Today's checks (subject-anchor token overlap, stemmed title-stem overlap) are *lexical* and a token collision defeats them — a small *aboutness/reranker* model (NOT NLI — entailment ≠ topicality) under #000049 §7's discipline cage verbatim (demotion-only, hash-pinned, `relevance_model_version`→`governance_policy_hash` iff it touches `audit_mode`, shadow-first, `[…]` extra, the §7 #20 haystack lesson — never over the whole context); gated on evidence, travels with #000049's model question. Motivating field case (2026-05-12, fox): the `claim_lattice` query that returned *"the phrase 'Zionist entity' is sometimes used as the entity, referring to the State of Israel"* at `EVIDENCE-WARRANTED-PARTIAL 2/3` — incoherent + token-collision recombination that NLI can't catch (returns *neutral*, not *contradiction*) and both lexical relevance checks waved through. Flags an upstream retrieval ticket (polysemy / title-token-soup) as the root-cause fix, not scoped here. #000049 sibling | 2026-05-12 | —         |
 | #000051  | Federated vecpack distribution (gossip the embedding backfill) | open · awaiting go/no-go · doc-only scaffold. Makes `chunk_vecs` a distributable artifact: backfill once on any CPU box (cloud / Prometheus-Σ sweep — #000037 §3.1), publish a **vecpack** `(shard_root, vec_backend_version, [(leaf_hash, embedding_blob)…])` over the mesh wire layer, every peer pulls + bulk-loads (sub-ms/chunk on the receiver — the laptop never runs the transformer). Keyed on `leaf_hash` (portable) not `chunk_id` (shard-local). Vecpacks are **soft data** — embeddings are `UNGROUNDED`, never proof path — so a cheap structural sanity gate (chunk exists locally w/ matching leaf_hash, right blob length for (dim,quant), finite norm, backend_version matches) suffices, no Merkle-proof-grade verification needed. Supplies #000050's prereq #1 ("a vecpack exists & is imported on the bench box", not "fox embedded the corpus locally"). GPU producer (the fast path): bge-small-en-v1.5 batched on a CUDA box (4090) ≈ 10³–10⁴ chunks/s → full 6.24M-chunk corpus in *minutes*, not days — drop a CUDA `Embedder` into `default_embedder()`; CUDA stack lives only on the producer box, never in arborist's `python+sqlite3` core. The mechanism behind whitepaper §1's "the embedding pass runs off the device". #000039 / #000050 sibling | 2026-05-12 | —         |
 | #000050  | Vec RRF hybrid fusion (#000039 Phase 2) | open · awaiting go/no-go · doc-only scaffold; design in #000039 §4.2 (RRF) + §8 (the gate). Wire `VecBackend` as a 5th retrieval route in `query.py`, RRF-merged (route provenance carried) with the 4 FTS5 routes; UNGROUNDED hits, additive not replacement. Phase-2 sub-items now explicit: **accept-path-5** in `_filter_by_title_relevance` (low-title-overlap vec hits survive only via a stronger span-level warrant, never similarity-score alone — else the title gate drops exactly the semantic candidates vec exists for & the bench shows no lift); **six** vec config fields fold into `governance_policy_hash` (recipe-named quant `int8sym`) **+ a cache-write guard** blocking `providence_cache` persistence for vec/hybrid runs until that's wired; **run-DAG records the vec stage** (backend version, six fields, top_k, query-embedding hash, candidate chunk_ids+distances). **Gated** on (a) a corpus backfill **distributed via #000051** AND (b) a **four-condition** recall bench (A FTS5-only / B vec-only / C RRF hybrid / D candidate-union-no-RRF) clearing the 5pp floor incl. C-beats-D, on semantic-allusion + curated + **adversarial-semantic-neighbor** fixtures (else park, vec stays opt-in `--backend vec`; if C≈D ship the union, drop RRF). #000039 follow-up | 2026-05-12 | —         |
 | #000049  | Attribution-aware grounding check (the recombination boundary) | open · boundary accepted · production no-go · shadow-path approved (de novo review 2026-05-13 — ticket §7) · doc-only; the home for #000048's deferred §2.3 — closing the 2 recombination over-grounds in `falsification-hard` (hard-003 Mercury / hard-005 Einstein) needs an attribution / dependency-parse or mini-NLI check, which is *not lexical* (#000048 §5). Discipline question answered: a small fixed purpose-built NLI/entailment *model* may influence `audit_mode` only as an opt-in, hash-pinned, governance-hashed, **demotion-only contradiction veto** after shadow-mode evidence (never promotes — `MODEL_ASSISTED_DEMOTION`, never `MODEL_ASSISTED_PROMOTION`). Production verifier unchanged; `falsification-hard` stays 10/12 as an honest boundary marker. Roadmap: Phase 0 (this amendment) → Phase 1 (shadow design: NLI manifest, fetch/verify, `nli_pair@v1` canonicalization, recombination-risk trigger) → Phase 2 (bench-only shadow impl, `[nli]` extra, `make fetch-nli`) → Phase 3 (demotion-only runtime, gated) → Phase 4 (mesh blob sync); §7 #12 six-condition bench gate required before Phases 2–4; if NLI ever affects `audit_mode`, `nli_policy_hash` folds into `governance_policy_hash`. **Phase-2 candidate bench done 2026-05-12** (`~/git/arborist-nli-bench/`, commits `829f9a4` + `a1cb28d`; ticket §7 #18): checkpoint-agnostic harness runs the §7 #5 clause-level algorithm over 28 synth recombination cases (incl. the 2 fixtures + harder shapes) + 26 legit cases (true summaries + near-miss decoys). 4 working candidates; `nli-MiniLM2-L6-H768` (82M, 45ms p50 CPU), `deberta-v3-base-mnli-fever-anli` (184M, 223ms), `bart-large-mnli` (407M, 259ms) all 28/28 catch · 0/26 FP with the standard θe=0.9 entailment guard; `cross-encoder/nli-deberta-v3-base` 27/28; deberta-large repo-id TODO. **Key finding: the §7 #5 two-threshold rule is load-bearing** — 3 of 4 candidates argmax-contradict 1/26 legit cases on the *wrong* source clause (competing-superlative confusion, e.g. "largest hot desert" vs "largest desert overall"); the entailment guard filters every one because another clause restates the claim → 0% guarded FP vs ~4% single-threshold. Picture: recombination is *easy* for any modern NLI checkpoint — differentiator is cost/robustness, MiniLM is the cost-pick, bart-large the threshold-robust pick. **Phase-2 shadow scaffold landed in arborist 2026-05-12** (ticket §7 #19): `arborist/qa/nli/` (manifest pins MiniLM @ a fixed HF revision + θc 0.5/θe 0.9 + 2 alternates; `ShadowNLI`/`shadow_check` lazy-imports `transformers`+`torch` behind a new `[nli]` extra, degrades to `available=False` when absent — SHADOW ONLY, never an `audit_mode` input, manifest not yet in `governance_policy_hash` per §7 #2) + `bench/scripts/nli_shadow_sweep.py` + `make bootstrap-nli` / `make bench-nli-shadow` + 16 tests. Synthetic sweep (116 records): 28/28 recombination demoted, 0/26 FP on legit summaries, 0/9 fires on already-`STRICT_SPAN`. **First bench-qa-traffic sweep** (§7 #20 — `ARBORIST_NLI_SHADOW=1 make bench-qa-smoke`, 15 cells; `query.py` surfaces verifier-input text gated off-by-default, `qa_sweep.py` carries it, `nli_shadow_sweep.py` reads it): the *naive* "NLI on every context clause" scaffold has a **~30% would-demote rate on STRICT answers** — a haystack/multiple-comparisons artifact (real Wikipedia contexts → 100–336 clauses; `max`-over-all almost always hits a tangential "contradiction"). Candidate-clause restriction added (§7 #21 — NLI runs on the top-6 source clauses by content-token overlap, not the whole context; `max_candidate_clauses=6`): STRICT would-demote 30% → 20% on the smoke, overall 47% → 33% — **helps, not fixed**; recombination-risk split doesn't separate either. Residual STRICT false-contras land at ~0.83–0.92, so θc would need ≈ 0.90 (up from the clean-set 0.5): at θc=0.90 the data in hand gives 27/28 synthetic recombination recall, 0/26 synthetic legit FP, 0/10 smoke STRICT FP — but n=10 is far too small to set a threshold on. Remaining: a fuller `ARBORIST_NLI_SHADOW=1 make bench-qa` run → ~hundreds of STRICT cells → sweep θc/θe → confirm → set it; until then θc stays 0.5 and runtime NLI demotion stays off. Standing lesson: the clean synthetic eval (§7 #18) does not predict bench-qa-traffic precision. Production verifier unchanged; `falsification-hard` stays 10/12. #000048 follow-up | 2026-05-12 | —         |

docs/tickets/ticket-000052-relevance-coherence-meta-cognition.md

@@ -1,6 +1,13 @@
 # Ticket #000052 — Relevance + coherence meta-cognition (answer-*shape* signals)
 
-**Status:** open · awaiting go/no-go · doc-only scaffold
+**Status:** in progress — §3.1 `diagnose_coherence` landed (lexical, no
+model: `circular` / `phrase_component_reuse` / `vacuous`; sidecar in
+`arborist/qa/inspect.py`, surfaced in `inspect_cache_key` + the
+`arborist inspect` human view as `· incoherent: <kind>`; 9 tests in
+`tests/test_inspect.py`; demote-policy hook deliberately NOT wired —
+advisory only, off the proof path). §3.2 (relevance *model*) still
+gated on evidence + travels with #000049's model question. §4.2
+(upstream retrieval polysemy fix) still a flag, not scoped here.
 **Opened:** 2026-05-12
 **Scope:** Decide whether — and if so how — to add two more read-only
 meta-cognition sidecars to the family that already includes
@@ -269,15 +276,33 @@ is a flag, not a deliverable here.
 
 ## 6. Status
 
-**Open · awaiting go/no-go.** Doc-only scaffold. First decision: build
-the lexical `diagnose_coherence` (§3.1) now? — low-risk, model-free,
-fits `inspect.py`; the recommendation is yes. Second decision (gated
-on evidence, travels with #000049): is a relevance *model* (§3.2)
-warranted, and if so it ships under #000049 §7's cage, shadow-first.
-Third (a flag, not this ticket): does fox want the upstream retrieval
-polysemy fix tracked? Until then: nothing lands; the existing
-deflection / title-relevance / metaphor sidecars stand as the only
-shape signals.
+**In progress.**
+
+- **§3.1 `diagnose_coherence` — landed.** Lexical, model-free, in
+  `arborist/qa/inspect.py`: per-sentence shape check emitting
+  `kind ∈ {phrase_component_reuse, circular, vacuous, ok, empty}`
+  (severity `phrase_component_reuse > circular > vacuous`).
+  `phrase_component_reuse` is the one that catches the motivating
+  field case — a subject that quotes a phrase, a predicate that reuses
+  one of that phrase's own content tokens as a bare `the/a/an <token>`
+  referent (copulas inside a quoted span are skipped so `'war is
+  peace'` doesn't break the subject/predicate split). Surfaced in
+  `inspect_cache_key`'s return dict (`"coherence": {...}`) and the
+  `arborist inspect` human view (`· incoherent: <kind>`). 9 tests in
+  `tests/test_inspect.py`. **Demote-policy hook deliberately NOT
+  wired** — like every sibling sidecar this is advisory render-tail +
+  bench telemetry only; if fox later wants the verifier to demote
+  STRICT → HYBRID on a hard coherence_kind, that's a separate policy
+  field that folds into `governance_policy_hash`.
+- **§3.2 relevance *model* — still the open question.** Gated on
+  evidence (does the lexical coherence check + the existing
+  deflection/title sidecars + an upstream retrieval fix together still
+  leave a measurable real-traffic incoherence/off-topic rate?) and
+  travels with #000049's "may a fixed small model touch `audit_mode`?"
+  decision. When warranted it ships under #000049 §7's cage,
+  shadow-first.
+- **§4.2 upstream retrieval polysemy fix — a flag, not this ticket.**
+  Open its own retrieval ticket if fox wants it tracked.
 
 ---