#000052 §3.2.2 refinement: claim-lattice metadata cleaning — STRICT FP drops universally (-0.5 to -5.6 pts), L-2 down to 1.5%

Hand-inspection of the bottom-15 STRICT-fires from the raw §3.2.2 step 2
sweep showed claim-lattice overlay markup ([E\d+ | title | hash: '…'])
depressing scores on correct concise answers (the 6× Henry-VIII case),
while true-positive deflections (broad-question / narrow-answer like
'winners of all major sports?' → just-one-sport) remained correctly
low-scored. So the noise FP class is the bracket metadata; cleaning it
should reduce FP without losing true-positive signal.

Built clean_for_relevance() in arborist/qa/relevance/shadow.py — strips
[E\d+ | ... ] blocks + trailing '...']' tails. Baked into
ShadowRelevance.check_question_answer / check_claim_source by default
(opt out with clean_input=False). relevance_shadow_sweep.py applies it
to inputs before _score_batch (opt out with --no-clean).

Re-ran the full 6-model sweep on the 808-cell pooled STRICT with
cleaning:
  bge-reranker-large   21.8% → 18.6%  (-3.2)
  MiniLM-L-4-v2        15.0% →  9.4%  (-5.6 pts, -37% rel)
  MiniLM-L-6-v2        11.6% →  9.0%  (-2.6)
  MiniLM-L-12-v2       11.0% →  8.0%  (-3.0)
  bge-reranker-base     9.5% →  9.0%  (-0.5)
  MiniLM-L-2-v2         4.5% →  1.5%  (-3.0 pts, -67% rel)

Universal improvement, every model better. Big surprise: MiniLM-L-2-v2
— the model that FAILED the candidate-bench separability (margin
-1.97, declared 'capacity floor') — has the LOWEST real-traffic FP
rate at its own cb θ (1.5%). Because L-2's compressed score range
gives it a low cb θ which few real STRICT pairs score below.
SEVENTH instance of 'candidate-bench doesn't predict real-traffic'.

Runtime-veto verdict UNCHANGED — still not viable; smallest fp=0 θ on
real STRICT is below the cb NEG max for every model, so at any
runtime-safe θ the catch on cb NEG is 0/12. But cleaning is now FREE
improvement for any soft-signal / advisory / contrastive use of the
relevance score. Hand-inspected bottom-10 post-cleaning confirms true-
positive deflection signal preserved.

bench/scripts/relevance_shadow_sweep.py

@@ -106,12 +106,14 @@ def main(argv=None) -> int:
                     help="manifest model id(s) comma-sep, or 'all' (default), or 'primary'")
     ap.add_argument("--out", type=Path,
                     default=REPO / "bench" / "results" / "relevance-shadow-sweep-pooled-strict.json")
+    ap.add_argument("--no-clean", action="store_true",
+                    help="skip the claim-lattice metadata cleaning before scoring (see #000052 §3.2.2 refinement: cleaning drops STRICT FP rate ~22%% relative)")
     args = ap.parse_args(argv)
 
     try:
         import torch  # noqa
         sys.path.insert(0, str(REPO))
-        from arborist.qa.relevance.shadow import ShadowRelevance, load_manifest
+        from arborist.qa.relevance.shadow import ShadowRelevance, load_manifest, clean_for_relevance
     except ImportError as e:
         print(f"[relevance-sweep] missing dependency: {e}  (pip install 'arborist[nli]')",
               file=sys.stderr)
@@ -151,8 +153,15 @@ def main(argv=None) -> int:
         if not rel.available:
             print(f"[relevance-sweep]   skip — {rel._reason}", file=sys.stderr); continue
         t0 = time.time()
-        # batched scoring: build all (q, a) pairs once
-        pairs = [(p["question"], p["answer"]) for p in strict]
+        # batched scoring: build all (q, a) pairs once, optionally
+        # claim-lattice-cleaned (the §3.2.2 refinement — drops STRICT
+        # FP ~22% relative on the pooled-808 sample without losing
+        # true-positive deflection signal).
+        if args.no_clean:
+            pairs = [(p["question"], p["answer"]) for p in strict]
+        else:
+            pairs = [(clean_for_relevance(p["question"]),
+                      clean_for_relevance(p["answer"])) for p in strict]
         scores = rel._score_batch(pairs)
         infer_s = time.time() - t0
         q = _quantiles(scores)