Ran diagnose_deflection and diagnose_coherence on the 20+20 NEG
fixtures from step 3 parts A+B, plus all 808 STRICT.
Result: 0/40 NEG fire on diagnose_deflection (because the fixtures
are token-overlap-correct by construction — the question's subject
appears in the answer; that's exactly the failure mode #000052 was
built to catch beyond lexical). 0/40 on diagnose_coherence (the
fixtures are well-formed sentences).
So:
UNION (relevance OR deflection) = relevance alone (no lift)
INTERSECT (relevance AND deflection) = 0/40
Multi-signal combination doesn't help on these failure shapes.
But the architectural finding is positive in a different way: the
sidecars and the relevance reranker cover NON-OVERLAPPING failure
shapes cleanly:
- diagnose_coherence: structural breakage (word-salad, vacuous,
phrase-component-reuse). Owns the 'incoherent answer' slice.
- diagnose_deflection: token-overlap mismatch (subject anchor not
in answer). Owns the 'wholesale topic drift' slice.
- bge-large relevance: semantic aboutness mismatch despite shared
tokens. Owns the 'topic-collision / mis-cite / on-topic-but-
not-answering' slice — what §3.2 was built for.
Each signal owns its own slice; combining is redundant on these
cases. That's the architectural validation of the §3.1 + §3.2 +
existing-lexical-sidecars split as orthogonal, not overlapping.
Manifest runtime_viability.as_multi_signal_factor updated from
'viable' to 'TESTED — does not lift; sidecars are complementary
not combinatorial on these shapes'. Cleaned up stale
nli-shadow-grid-n1-minilm.json.
Also cleaned up a stale nli-shadow-grid JSON.
The §3.2 arc is now complete:
step 1 candidate-bench (overclaim, contrived data)
step 2 real STRICT FP (over-pessimistic 'NOT VIABLE')
step 3A real-context deflection (positive reversal, 55% at fp=0)
step 3B real-context mis-cite (100% at fp=0, motivating failure covered)
step 3C multi-signal combination (no lift; clean architectural split)
The recommended operating point holds: bge-reranker-large + cleaned
+ θ ≤ -2.42 → 100% mis-cite, 55% deflect, 0% STRICT FP.
russell@unturf.com
7910440e50