#000052 §3.1: diagnose_coherence — lexical answer-shape sidecar

Per-sentence shape check (no model) emitting kind ∈
{phrase_component_reuse, circular, vacuous, ok, empty}:
  - circular: subject content-tokens ⊆ predicate's and the predicate
    leads with a subject token ("Water is water").
  - phrase_component_reuse: subject quotes a phrase, predicate reuses
    one of that phrase's own tokens as a bare "the/a/an <token>"
    referent — the 2026-05-12 field case ("the phrase 'Zionist entity'
    is used as the entity"), a token collision the verifier +
    deflection + title-relevance all pass and NLI returns neutral on.
    Copulas inside a quoted span are skipped so 'war is peace' doesn't
    break the subject/predicate split.
  - vacuous: predicate is only placeholder hypernyms + filler ("X is
    a thing").

Conservative — no full token-salad parsing; legit definitions pass ok.
Surfaced in inspect_cache_key + the `arborist inspect` human view
(· incoherent: <kind>). Advisory only — never writes providence_cache
/ audit_events / run_dag_root; demote-only verifier hook deliberately
not wired. 9 tests; full suite green (2500 passed).

CLAUDE.md

@@ -278,6 +278,22 @@ revert without reading why. When in doubt, walk the
   token in question must appear in answer). Suppressed for date /
   count / cause shapes ("when", "why", "how many"). See
   `arborist/qa/inspect.py`.
+- **Coherence sidecar** (#000052 §3.1): `diagnose_coherence(answer)`
+  in `arborist/qa/inspect.py` — per-sentence lexical shape check,
+  **no model**. Emits `kind ∈ {phrase_component_reuse, circular,
+  vacuous, ok, empty}`: `circular` = subject content-tokens ⊆
+  predicate's and the predicate leads with a subject token ("X is
+  X"); `phrase_component_reuse` = the subject quotes a phrase & the
+  predicate reuses one of that phrase's own tokens as a bare
+  `the/a/an <token>` referent (the 2026-05-12 "the phrase 'Zionist
+  entity' is used as the entity" field case — a token collision the
+  verifier + deflection + title-relevance all pass and NLI returns
+  *neutral* on); `vacuous` = predicate is only placeholder hypernyms
+  + filler ("X is a thing"). Surfaced in `inspect_cache_key` + the
+  `arborist inspect` human view (`· incoherent: <kind>`). Advisory
+  sidecar — **never** writes `providence_cache` / `audit_events` /
+  `run_dag_root`; a demote-only verifier hook is possible but
+  deliberately not wired (would fold into `governance_policy_hash`).
 - **Title-relevance hard check (Rule 8)**: `_claim_title_overlap`
   in `arborist/qa/verify.py`. For each claim that resolved, at least
   one cited evidence's source title must share ≥1 stemmed content

arborist/cli.py

@@ -1361,6 +1361,17 @@ def _render_inspect_human(result: dict) -> str:
             )
     lines.append("")
 
+    coh = result.get("coherence") or {}
+    if coh.get("kind") not in (None, "ok", "empty"):
+        ev = coh.get("evidence") or {}
+        if coh["kind"] == "phrase_component_reuse":
+            detail = f" (reuses {ev.get('reused_tokens')} from a quoted phrase)"
+        else:  # circular | vacuous
+            detail = f" (subject={ev.get('subject_tokens')})"
+        lines.append(f"  · incoherent: {coh['kind']}{detail}")
+        lines.append(f"      sentence: {_short(ev.get('sentence', ''), 140)}")
+        lines.append("")
+
     if not diagnoses:
         lines.append("(no unverified spans)")
         return "\n".join(lines)

arborist/qa/inspect.py

@@ -831,6 +831,271 @@ def diagnose_deflection(question_text: str, answer_text: str) -> dict[str, Any]:
     }
 
 
+# --- diagnose_coherence (#000052 §3.1 — lexical, no model) ----------------
+
+# Placeholder-hypernym heads: nouns that by themselves carry no
+# differentia. A predicate consisting only of these (plus stopwords and
+# filler) is vacuous ("X is a thing", "X refers to a concept").
+_COHERENCE_VACUOUS_HEADS = frozenset(
+    """
+    thing things concept concepts term terms word words idea ideas notion
+    notions object objects entity entities phrase phrases expression
+    expressions name names something anything stuff topic subject matter
+    """.split()
+)
+
+# Tokens that never lift a predicate above "vacuous" on their own —
+# frequency adverbs, hedges, and the bland connective verbs that
+# introduce a definition without supplying any content. A legitimate
+# definition always carries content tokens beyond these.
+_COHERENCE_FILLER = frozenset(
+    """
+    sometimes often usually generally typically commonly occasionally
+    used use uses using refer refers referring referred meaning meanings
+    mean means context contexts various certain some many several
+    different particular related associated known called regarded
+    considered described defined sense senses way ways kind kinds type
+    types form forms sort sorts manner manners
+    """.split()
+)
+
+# Definitional connectives. First match (leftmost) is the split point.
+# Multi-word ones precede single "is/are/..." so "refers to" splits as
+# a unit rather than on a bare "is" elsewhere in the sentence.
+_COHERENCE_COPULAS = (
+    "referred to as", "considered to be", "refers to", "refer to",
+    "referring to", "defined as", "used as", "known as", "regarded as",
+    "denotes", "signifies", "means", "meaning",
+    "is", "are", "was", "were", "be", "being", "been",
+)
+
+# Determiner + bare noun in the predicate — the "the X" / "as the X"
+# referent shape that, when X is a token from a quoted phrase in the
+# subject, signals phrase-component reuse.
+_COHERENCE_BARE_REFERENT_RE = None  # compiled lazily
+
+
+def _coherence_sentences(text: str) -> list[str]:
+    """Split an answer into candidate assertions: on line breaks,
+    sentence terminators, and semicolons; strip a leading bullet /
+    numbering marker per line. Over-splitting is fine — each fragment
+    is analysed independently."""
+    import re
+    if not text:
+        return []
+    out: list[str] = []
+    for line in re.split(r"[\r\n]+", text):
+        line = re.sub(r"^\s*(?:[-*•–]|\d+[.)])\s+", "", line.strip())
+        for piece in re.split(r"(?<=[.!?;])\s+", line):
+            piece = piece.strip()
+            if piece:
+                out.append(piece)
+    return out
+
+
+def _coherence_quoted_token_sets(sentence: str) -> list[set[str]]:
+    """Content-token sets, one per quoted span in the sentence (straight
+    or smart quotes). Skips apparent contractions/possessives — an
+    opening quote glued to a preceding letter — and single-content-token
+    'quotes' (scare-quote emphasis, not a named phrase)."""
+    import re
+    out: list[set[str]] = []
+    for m in re.finditer(
+        r"(?<![A-Za-z])['\"‘“]([^'\"’”\r\n]{2,80})['\"’”]",
+        sentence,
+    ):
+        toks = set(_content_tokens_in_order(m.group(1)))
+        if len(toks) >= 2:
+            out.append(toks)
+    return out
+
+
+def _coherence_quoted_ranges(sentence: str) -> list[tuple[int, int]]:
+    """Char ranges [start, end) of quoted spans — so the copula split
+    doesn't fire on an 'is'/'are' that lives inside a quoted phrase
+    (e.g. 'war is peace')."""
+    import re
+    return [
+        m.span()
+        for m in re.finditer(
+            r"(?<![A-Za-z])['\"‘“][^'\"’”\r\n]{2,80}['\"’”]", sentence
+        )
+    ]
+
+
+def _coherence_split_on_copula(sentence: str) -> tuple[str, str] | None:
+    """Split a sentence at its first definitional connective (one not
+    inside a quoted span) → (subject_text, predicate_text). None if no
+    copula present."""
+    import re
+    low = sentence.lower()
+    quoted = _coherence_quoted_ranges(sentence)
+
+    def _in_quote(i: int) -> bool:
+        return any(a <= i < b for a, b in quoted)
+
+    best: tuple[int, int] | None = None
+    for cop in _COHERENCE_COPULAS:
+        for m in re.finditer(r"\b" + re.escape(cop) + r"\b", low):
+            if _in_quote(m.start()):
+                continue
+            if best is None or m.start() < best[0]:
+                best = (m.start(), m.end())
+            break  # leftmost occurrence of this copula is enough
+    if best is None:
+        return None
+    return sentence[: best[0]].strip(), sentence[best[1] :].strip()
+
+
+def _coherence_bare_referents(predicate_text: str) -> set[str]:
+    """Stemmed content tokens that appear in the predicate as a bare
+    'the/a/an <noun>' referent (covers 'as the X', 'is the X', etc. —
+    the determiner is what matters, not the preceding word)."""
+    import re
+    global _COHERENCE_BARE_REFERENT_RE
+    if _COHERENCE_BARE_REFERENT_RE is None:
+        _COHERENCE_BARE_REFERENT_RE = re.compile(
+            r"\b(?:the|a|an)\s+([A-Za-z][A-Za-z'\-]+)", re.IGNORECASE
+        )
+    out: set[str] = set()
+    for m in _COHERENCE_BARE_REFERENT_RE.finditer(predicate_text):
+        t = _stem_for_deflection(m.group(1).lower())
+        if len(t) >= 3 and t not in _DEFLECTION_STOPWORDS:
+            out.add(t)
+    return out
+
+
+def _coherence_classify_sentence(sentence: str) -> dict[str, Any] | None:
+    """Return a finding dict for the most severe shape problem in this
+    sentence, or None if it looks well-formed.
+
+    Severity: phrase_component_reuse > circular > vacuous."""
+    split = _coherence_split_on_copula(sentence)
+    if split is None:
+        return None
+    subj_text, pred_text = split
+    subj_toks = set(_content_tokens_in_order(subj_text))
+    pred_in_order = _content_tokens_in_order(pred_text)
+    pred_toks = set(pred_in_order)
+
+    # 1. phrase_component_reuse — subject quotes a phrase; predicate
+    #    re-uses one of that phrase's own tokens as a bare referent.
+    quoted_sets = _coherence_quoted_token_sets(subj_text)
+    if quoted_sets:
+        bare = _coherence_bare_referents(pred_text)
+        for qs in quoted_sets:
+            reused = sorted(qs & bare)
+            if reused:
+                return {
+                    "kind": "phrase_component_reuse",
+                    "sentence": sentence,
+                    "reused_tokens": reused,
+                    "quoted_phrase_tokens": sorted(qs),
+                }
+
+    # 2. circular — subject content tokens are a non-empty subset of the
+    #    predicate's, and the predicate leads with a subject token (or
+    #    the predicate is itself entirely vacuous).
+    pred_non_vacuous = [
+        t for t in pred_in_order
+        if t not in _COHERENCE_FILLER and t not in _COHERENCE_VACUOUS_HEADS
+    ]
+    pred_all_vacuous = len(pred_toks) > 0 and not pred_non_vacuous
+    if subj_toks and subj_toks <= pred_toks:
+        leads_with_subject = bool(pred_in_order) and pred_in_order[0] in subj_toks
+        if leads_with_subject or pred_all_vacuous:
+            return {
+                "kind": "circular",
+                "sentence": sentence,
+                "subject_tokens": sorted(subj_toks),
+                "predicate_tokens": sorted(pred_toks),
+            }
+
+    # 3. vacuous — predicate has no content beyond placeholder hypernyms
+    #    + filler (and the subject did carry a real topic).
+    if subj_toks and (not pred_toks or pred_all_vacuous):
+        return {
+            "kind": "vacuous",
+            "sentence": sentence,
+            "subject_tokens": sorted(subj_toks),
+            "predicate_tokens": sorted(pred_toks),
+        }
+    return None
+
+
+_COHERENCE_SEVERITY = {"phrase_component_reuse": 3, "circular": 2, "vacuous": 1}
+
+
+def diagnose_coherence(answer_text: str) -> dict[str, Any]:
+    """Soft signal: is the answer a well-formed assertion, or
+    word-salad / circular / vacuous?
+
+    Three lexical, model-free shape checks over each sentence
+    (#000052 §3.1):
+
+      - ``circular`` — ``<subject> <copula> <predicate>`` where the
+        subject's content tokens are a non-empty subset of the
+        predicate's and the predicate leads with a subject token (or the
+        predicate is itself entirely vacuous). Catches "X is X", "the
+        entity is the entity referring to …".
+      - ``phrase_component_reuse`` — the subject quotes a phrase, and
+        the predicate uses one of that phrase's own content tokens as a
+        bare ``the/a/an <token>`` referent. Catches the 2026-05-12 field
+        case: *"the phrase 'Zionist entity' is sometimes used as the
+        entity, referring to the State of Israel"* — defining a phrase
+        in terms of a word inside it (a token collision the lexical
+        verifier + ``diagnose_deflection`` + ``diagnose_title_relevance``
+        all wave through, and one NLI returns *neutral* on).
+      - ``vacuous`` — the predicate (post-copula, stopwords + filler
+        adverbs/hedges removed) is empty or only placeholder hypernyms
+        ("X is a thing", "X refers to a concept used in some contexts").
+
+    Pure lexical / parse-shape — no model, no proof-path write. Same
+    discipline as ``diagnose_deflection`` / ``diagnose_title_relevance``:
+    advisory render-tail (``· incoherent: <kind>``) + bench telemetry
+    only; never enters the binary verifier output, ``providence_cache``,
+    ``audit_events``, or ``run_dag_root``. A policy flag *could* let the
+    verifier demote STRICT → HYBRID on a hard coherence_kind (mirroring
+    Rule 8's ``TITLE_MISMATCH`` demote) — off by default, folds into
+    ``governance_policy_hash`` iff ever enabled; not wired here.
+
+    Conservative by construction: it does NOT attempt full token-salad
+    parsing — only the three shapes above, each gated tightly enough
+    that a legitimate definition carrying any real differentia passes.
+
+    Returns:
+        {
+          "kind": "phrase_component_reuse" | "circular" | "vacuous"
+                  | "ok" | "empty",
+          "evidence": {... worst finding ...} | None,
+          "findings": [ {sentence, kind, ...}, ... ],   # all non-ok
+          "n_sentences": int,
+        }
+    """
+    sentences = _coherence_sentences(answer_text or "")
+    if not sentences:
+        return {"kind": "empty", "evidence": None, "findings": [], "n_sentences": 0}
+    findings: list[dict[str, Any]] = []
+    for s in sentences:
+        f = _coherence_classify_sentence(s)
+        if f is not None:
+            findings.append(f)
+    if not findings:
+        return {
+            "kind": "ok",
+            "evidence": None,
+            "findings": [],
+            "n_sentences": len(sentences),
+        }
+    worst = max(findings, key=lambda f: _COHERENCE_SEVERITY.get(f["kind"], 0))
+    return {
+        "kind": worst["kind"],
+        "evidence": worst,
+        "findings": findings,
+        "n_sentences": len(sentences),
+    }
+
+
 def inspect_cache_key(
     cache_key: str,
     *,
@@ -920,6 +1185,11 @@ def inspect_cache_key(
     # answering). Sidecar-only, never feeds back into providence.
     deflection = diagnose_deflection(row["question_text"], row["answer_text"])
 
+    # Soft coherence signal — see diagnose_coherence (#000052 §3.1).
+    # Lexical shape check: circular / phrase-component-reuse / vacuous
+    # answers. Sidecar-only, never feeds back into providence.
+    coherence = diagnose_coherence(row["answer_text"])
+
     # Authorship warrant ladder (#000026 Phase 3) — sidecar classifier
     # that names the strongest-tier authorship signal in cited
     # evidence. Returns NO_AUTHORSHIP_SIGNAL when the question doesn't
@@ -957,5 +1227,6 @@ def inspect_cache_key(
         },
         "unverified": diagnoses,
         "deflection": deflection,
+        "coherence": coherence,
         "authorship": authorship,
     }

docs/TICKETS.md

@@ -101,7 +101,7 @@ Newest first. Update on every open/close.
 
 | ID       | Title                                          | Status                | Opened     | Directive |
 |----------|------------------------------------------------|-----------------------|------------|-----------|
-| #000052  | Relevance + coherence meta-cognition (answer-*shape* signals) | open · awaiting go/no-go · doc-only scaffold. Two more read-only, demote-only, never-in-proof-path sidecars joining the `diagnose_deflection` / `diagnose_metaphor_deflection` / `diagnose_title_relevance` / soft-preflight family: **(1) `diagnose_coherence`** — word-salad / circular ("X is X") / vacuous / underspecified answers; lexical/parse-shape, **no model** — the near-term win (recommended now). **(2) `diagnose_relevance`** — semantic (not just lexical) "aboutness": does the answer address the question; is each claim about its cited source? Today's checks (subject-anchor token overlap, stemmed title-stem overlap) are *lexical* and a token collision defeats them — a small *aboutness/reranker* model (NOT NLI — entailment ≠ topicality) under #000049 §7's discipline cage verbatim (demotion-only, hash-pinned, `relevance_model_version`→`governance_policy_hash` iff it touches `audit_mode`, shadow-first, `[…]` extra, the §7 #20 haystack lesson — never over the whole context); gated on evidence, travels with #000049's model question. Motivating field case (2026-05-12, fox): the `claim_lattice` query that returned *"the phrase 'Zionist entity' is sometimes used as the entity, referring to the State of Israel"* at `EVIDENCE-WARRANTED-PARTIAL 2/3` — incoherent + token-collision recombination that NLI can't catch (returns *neutral*, not *contradiction*) and both lexical relevance checks waved through. Flags an upstream retrieval ticket (polysemy / title-token-soup) as the root-cause fix, not scoped here. #000049 sibling | 2026-05-12 | —         |
+| #000052  | Relevance + coherence meta-cognition (answer-*shape* signals) | in progress — **§3.1 `diagnose_coherence` landed** (lexical, no model: `circular` / `phrase_component_reuse` / `vacuous`; in `arborist/qa/inspect.py`, surfaced via `inspect_cache_key` + `arborist inspect` `· incoherent: <kind>`; 9 tests; demote-policy hook deliberately not wired — advisory only). Joins the `diagnose_deflection` / `diagnose_metaphor_deflection` / `diagnose_title_relevance` / soft-preflight family of read-only, demote-only, never-in-proof-path sidecars; `phrase_component_reuse` catches the motivating field case (a subject quoting a phrase, a predicate reusing one of that phrase's own tokens as a bare `the <token>` referent). **Still open: (2) `diagnose_relevance`** — semantic (not just lexical) "aboutness": does the answer address the question; is each claim about its cited source? Today's checks (subject-anchor token overlap, stemmed title-stem overlap) are *lexical* and a token collision defeats them — a small *aboutness/reranker* model (NOT NLI — entailment ≠ topicality) under #000049 §7's discipline cage verbatim (demotion-only, hash-pinned, `relevance_model_version`→`governance_policy_hash` iff it touches `audit_mode`, shadow-first, `[…]` extra, the §7 #20 haystack lesson — never over the whole context); gated on evidence, travels with #000049's model question. Motivating field case (2026-05-12, fox): the `claim_lattice` query that returned *"the phrase 'Zionist entity' is sometimes used as the entity, referring to the State of Israel"* at `EVIDENCE-WARRANTED-PARTIAL 2/3` — incoherent + token-collision recombination that NLI can't catch (returns *neutral*, not *contradiction*) and both lexical relevance checks waved through. Flags an upstream retrieval ticket (polysemy / title-token-soup) as the root-cause fix, not scoped here. #000049 sibling | 2026-05-12 | —         |
 | #000051  | Federated vecpack distribution (gossip the embedding backfill) | open · awaiting go/no-go · doc-only scaffold. Makes `chunk_vecs` a distributable artifact: backfill once on any CPU box (cloud / Prometheus-Σ sweep — #000037 §3.1), publish a **vecpack** `(shard_root, vec_backend_version, [(leaf_hash, embedding_blob)…])` over the mesh wire layer, every peer pulls + bulk-loads (sub-ms/chunk on the receiver — the laptop never runs the transformer). Keyed on `leaf_hash` (portable) not `chunk_id` (shard-local). Vecpacks are **soft data** — embeddings are `UNGROUNDED`, never proof path — so a cheap structural sanity gate (chunk exists locally w/ matching leaf_hash, right blob length for (dim,quant), finite norm, backend_version matches) suffices, no Merkle-proof-grade verification needed. Supplies #000050's prereq #1 ("a vecpack exists & is imported on the bench box", not "fox embedded the corpus locally"). GPU producer (the fast path): bge-small-en-v1.5 batched on a CUDA box (4090) ≈ 10³–10⁴ chunks/s → full 6.24M-chunk corpus in *minutes*, not days — drop a CUDA `Embedder` into `default_embedder()`; CUDA stack lives only on the producer box, never in arborist's `python+sqlite3` core. The mechanism behind whitepaper §1's "the embedding pass runs off the device". #000039 / #000050 sibling | 2026-05-12 | —         |
 | #000050  | Vec RRF hybrid fusion (#000039 Phase 2) | open · awaiting go/no-go · doc-only scaffold; design in #000039 §4.2 (RRF) + §8 (the gate). Wire `VecBackend` as a 5th retrieval route in `query.py`, RRF-merged (route provenance carried) with the 4 FTS5 routes; UNGROUNDED hits, additive not replacement. Phase-2 sub-items now explicit: **accept-path-5** in `_filter_by_title_relevance` (low-title-overlap vec hits survive only via a stronger span-level warrant, never similarity-score alone — else the title gate drops exactly the semantic candidates vec exists for & the bench shows no lift); **six** vec config fields fold into `governance_policy_hash` (recipe-named quant `int8sym`) **+ a cache-write guard** blocking `providence_cache` persistence for vec/hybrid runs until that's wired; **run-DAG records the vec stage** (backend version, six fields, top_k, query-embedding hash, candidate chunk_ids+distances). **Gated** on (a) a corpus backfill **distributed via #000051** AND (b) a **four-condition** recall bench (A FTS5-only / B vec-only / C RRF hybrid / D candidate-union-no-RRF) clearing the 5pp floor incl. C-beats-D, on semantic-allusion + curated + **adversarial-semantic-neighbor** fixtures (else park, vec stays opt-in `--backend vec`; if C≈D ship the union, drop RRF). #000039 follow-up | 2026-05-12 | —         |
 | #000049  | Attribution-aware grounding check (the recombination boundary) | open · boundary accepted · production no-go · shadow-path approved (de novo review 2026-05-13 — ticket §7) · doc-only; the home for #000048's deferred §2.3 — closing the 2 recombination over-grounds in `falsification-hard` (hard-003 Mercury / hard-005 Einstein) needs an attribution / dependency-parse or mini-NLI check, which is *not lexical* (#000048 §5). Discipline question answered: a small fixed purpose-built NLI/entailment *model* may influence `audit_mode` only as an opt-in, hash-pinned, governance-hashed, **demotion-only contradiction veto** after shadow-mode evidence (never promotes — `MODEL_ASSISTED_DEMOTION`, never `MODEL_ASSISTED_PROMOTION`). Production verifier unchanged; `falsification-hard` stays 10/12 as an honest boundary marker. Roadmap: Phase 0 (this amendment) → Phase 1 (shadow design: NLI manifest, fetch/verify, `nli_pair@v1` canonicalization, recombination-risk trigger) → Phase 2 (bench-only shadow impl, `[nli]` extra, `make fetch-nli`) → Phase 3 (demotion-only runtime, gated) → Phase 4 (mesh blob sync); §7 #12 six-condition bench gate required before Phases 2–4; if NLI ever affects `audit_mode`, `nli_policy_hash` folds into `governance_policy_hash`. **Phase-2 candidate bench done 2026-05-12** (`~/git/arborist-nli-bench/`, commits `829f9a4` + `a1cb28d`; ticket §7 #18): checkpoint-agnostic harness runs the §7 #5 clause-level algorithm over 28 synth recombination cases (incl. the 2 fixtures + harder shapes) + 26 legit cases (true summaries + near-miss decoys). 4 working candidates; `nli-MiniLM2-L6-H768` (82M, 45ms p50 CPU), `deberta-v3-base-mnli-fever-anli` (184M, 223ms), `bart-large-mnli` (407M, 259ms) all 28/28 catch · 0/26 FP with the standard θe=0.9 entailment guard; `cross-encoder/nli-deberta-v3-base` 27/28; deberta-large repo-id TODO. **Key finding: the §7 #5 two-threshold rule is load-bearing** — 3 of 4 candidates argmax-contradict 1/26 legit cases on the *wrong* source clause (competing-superlative confusion, e.g. "largest hot desert" vs "largest desert overall"); the entailment guard filters every one because another clause restates the claim → 0% guarded FP vs ~4% single-threshold. Picture: recombination is *easy* for any modern NLI checkpoint — differentiator is cost/robustness, MiniLM is the cost-pick, bart-large the threshold-robust pick. **Phase-2 shadow scaffold landed in arborist 2026-05-12** (ticket §7 #19): `arborist/qa/nli/` (manifest pins MiniLM @ a fixed HF revision + θc 0.5/θe 0.9 + 2 alternates; `ShadowNLI`/`shadow_check` lazy-imports `transformers`+`torch` behind a new `[nli]` extra, degrades to `available=False` when absent — SHADOW ONLY, never an `audit_mode` input, manifest not yet in `governance_policy_hash` per §7 #2) + `bench/scripts/nli_shadow_sweep.py` + `make bootstrap-nli` / `make bench-nli-shadow` + 16 tests. Synthetic sweep (116 records): 28/28 recombination demoted, 0/26 FP on legit summaries, 0/9 fires on already-`STRICT_SPAN`. **First bench-qa-traffic sweep** (§7 #20 — `ARBORIST_NLI_SHADOW=1 make bench-qa-smoke`, 15 cells; `query.py` surfaces verifier-input text gated off-by-default, `qa_sweep.py` carries it, `nli_shadow_sweep.py` reads it): the *naive* "NLI on every context clause" scaffold has a **~30% would-demote rate on STRICT answers** — a haystack/multiple-comparisons artifact (real Wikipedia contexts → 100–336 clauses; `max`-over-all almost always hits a tangential "contradiction"). Candidate-clause restriction added (§7 #21 — NLI runs on the top-6 source clauses by content-token overlap, not the whole context; `max_candidate_clauses=6`): STRICT would-demote 30% → 20% on the smoke, overall 47% → 33% — **helps, not fixed**; recombination-risk split doesn't separate either. Residual STRICT false-contras land at ~0.83–0.92, so θc would need ≈ 0.90 (up from the clean-set 0.5): at θc=0.90 the data in hand gives 27/28 synthetic recombination recall, 0/26 synthetic legit FP, 0/10 smoke STRICT FP — but n=10 is far too small to set a threshold on. Remaining: a fuller `ARBORIST_NLI_SHADOW=1 make bench-qa` run → ~hundreds of STRICT cells → sweep θc/θe → confirm → set it; until then θc stays 0.5 and runtime NLI demotion stays off. Standing lesson: the clean synthetic eval (§7 #18) does not predict bench-qa-traffic precision. Production verifier unchanged; `falsification-hard` stays 10/12. #000048 follow-up | 2026-05-12 | —         |

docs/tickets/ticket-000052-relevance-coherence-meta-cognition.md

@@ -1,6 +1,13 @@
 # Ticket #000052 — Relevance + coherence meta-cognition (answer-*shape* signals)
 
-**Status:** open · awaiting go/no-go · doc-only scaffold
+**Status:** in progress — §3.1 `diagnose_coherence` landed (lexical, no
+model: `circular` / `phrase_component_reuse` / `vacuous`; sidecar in
+`arborist/qa/inspect.py`, surfaced in `inspect_cache_key` + the
+`arborist inspect` human view as `· incoherent: <kind>`; 9 tests in
+`tests/test_inspect.py`; demote-policy hook deliberately NOT wired —
+advisory only, off the proof path). §3.2 (relevance *model*) still
+gated on evidence + travels with #000049's model question. §4.2
+(upstream retrieval polysemy fix) still a flag, not scoped here.
 **Opened:** 2026-05-12
 **Scope:** Decide whether — and if so how — to add two more read-only
 meta-cognition sidecars to the family that already includes
@@ -269,15 +276,33 @@ is a flag, not a deliverable here.
 
 ## 6. Status
 
-**Open · awaiting go/no-go.** Doc-only scaffold. First decision: build
-the lexical `diagnose_coherence` (§3.1) now? — low-risk, model-free,
-fits `inspect.py`; the recommendation is yes. Second decision (gated
-on evidence, travels with #000049): is a relevance *model* (§3.2)
-warranted, and if so it ships under #000049 §7's cage, shadow-first.
-Third (a flag, not this ticket): does fox want the upstream retrieval
-polysemy fix tracked? Until then: nothing lands; the existing
-deflection / title-relevance / metaphor sidecars stand as the only
-shape signals.
+**In progress.**
+
+- **§3.1 `diagnose_coherence` — landed.** Lexical, model-free, in
+  `arborist/qa/inspect.py`: per-sentence shape check emitting
+  `kind ∈ {phrase_component_reuse, circular, vacuous, ok, empty}`
+  (severity `phrase_component_reuse > circular > vacuous`).
+  `phrase_component_reuse` is the one that catches the motivating
+  field case — a subject that quotes a phrase, a predicate that reuses
+  one of that phrase's own content tokens as a bare `the/a/an <token>`
+  referent (copulas inside a quoted span are skipped so `'war is
+  peace'` doesn't break the subject/predicate split). Surfaced in
+  `inspect_cache_key`'s return dict (`"coherence": {...}`) and the
+  `arborist inspect` human view (`· incoherent: <kind>`). 9 tests in
+  `tests/test_inspect.py`. **Demote-policy hook deliberately NOT
+  wired** — like every sibling sidecar this is advisory render-tail +
+  bench telemetry only; if fox later wants the verifier to demote
+  STRICT → HYBRID on a hard coherence_kind, that's a separate policy
+  field that folds into `governance_policy_hash`.
+- **§3.2 relevance *model* — still the open question.** Gated on
+  evidence (does the lexical coherence check + the existing
+  deflection/title sidecars + an upstream retrieval fix together still
+  leave a measurable real-traffic incoherence/off-topic rate?) and
+  travels with #000049's "may a fixed small model touch `audit_mode`?"
+  decision. When warranted it ships under #000049 §7's cage,
+  shadow-first.
+- **§4.2 upstream retrieval polysemy fix — a flag, not this ticket.**
+  Open its own retrieval ticket if fox wants it tracked.
 
 ---
 

tests/test_inspect.py

@@ -21,6 +21,7 @@ import pytest
 from arborist.qa.inspect import (
     _classify_span,
     _normalize,
+    diagnose_coherence,
     diagnose_deflection,
     diagnose_metaphor_deflection,
     diagnose_title_relevance,
@@ -852,3 +853,88 @@ def test_inspect_authorship_repo_owner_tier(tmp_path):
     )
     result = inspect_cache_key("03" * 32, qa_db=qa_db, shards_dir=tmp_path)
     assert result["authorship"]["tier"] == "AUTHOR_REPOSITORY_OWNER"
+
+
+# --- diagnose_coherence (#000052 §3.1) ----------------------------------
+
+
+def test_coherence_phrase_component_reuse_zionist_field_case():
+    # The 2026-05-12 field case: a phrase defined in terms of a word
+    # inside it. Lexical verifier scored it EVIDENCE-WARRANTED-PARTIAL;
+    # deflection/title-relevance both waved it through; NLI returns
+    # neutral. This sidecar is the one that catches it.
+    d = diagnose_coherence(
+        "The phrase 'Zionist entity' is sometimes used as the entity, "
+        "referring to the State of Israel."
+    )
+    assert d["kind"] == "phrase_component_reuse"
+    assert d["evidence"]["reused_tokens"] == ["entity"]
+    assert "entity" in d["evidence"]["quoted_phrase_tokens"]
+
+
+def test_coherence_circular_x_is_x():
+    d = diagnose_coherence("Water is water.")
+    assert d["kind"] == "circular"
+    assert d["evidence"]["subject_tokens"] == ["water"]
+
+
+def test_coherence_circular_subject_subset_of_predicate():
+    d = diagnose_coherence(
+        "The entity is the entity referring to the State of Israel."
+    )
+    assert d["kind"] == "circular"
+
+
+def test_coherence_vacuous_predicate_only_hypernyms():
+    d = diagnose_coherence("Happiness refers to a concept used in some contexts.")
+    assert d["kind"] == "vacuous"
+    d2 = diagnose_coherence("Gravity is a thing.")
+    assert d2["kind"] == "vacuous"
+
+
+def test_coherence_ok_on_well_formed_definitions():
+    for ans in (
+        "Paris is the capital of France.",
+        "A poodle is a type of dog.",
+        "The Beatles were a band formed in Liverpool in 1960.",
+        "The term 'open source' refers to software whose source code is "
+        "publicly available.",
+        "The phrase 'break a leg' is used as an idiom meaning good luck.",
+    ):
+        assert diagnose_coherence(ans)["kind"] == "ok", ans
+
+
+def test_coherence_copula_inside_quotes_does_not_break_split():
+    # 'war is peace' contains "is" — the split must not fire on it.
+    d = diagnose_coherence(
+        "The phrase 'war is peace' is a slogan from the novel "
+        "Nineteen Eighty-Four."
+    )
+    assert d["kind"] == "ok"
+
+
+def test_coherence_empty_and_pronoun_subject_are_not_flagged():
+    assert diagnose_coherence("")["kind"] == "empty"
+    assert diagnose_coherence("   \n  ")["kind"] == "empty"
+    # pronoun subject → no real topic token → conservatively not flagged
+    assert diagnose_coherence("It is something used in various contexts.")[
+        "kind"
+    ] == "ok"
+
+
+def test_coherence_reports_all_findings_and_worst_kind():
+    d = diagnose_coherence(
+        "- The phrase 'Zionist entity' is used as the entity.\n"
+        "- Happiness is a concept."
+    )
+    assert d["kind"] == "phrase_component_reuse"  # most severe wins
+    kinds = {f["kind"] for f in d["findings"]}
+    assert kinds == {"phrase_component_reuse", "vacuous"}
+    assert d["n_sentences"] == 2
+
+
+def test_coherence_is_pure_no_side_effects():
+    # No DB, no files — diagnose_coherence is a pure function over text.
+    before = diagnose_coherence("Water is water.")
+    after = diagnose_coherence("Water is water.")
+    assert before == after