{
  "_comment": "Pinned aboutness/relevance reranker for the #000052 §3.2 SHADOW path. SHADOW MODE ONLY — and §3.2.2 step 2 (2026-05-13, bench/results/relevance-shadow-sweep-pooled808.json) showed the candidate-bench θ values DO NOT survive real bench-qa traffic: at the candidate-bench fp=0 θ, every one of 6 swept rerankers false-fires on 4.5–21.8% of pooled bench-qa STRICT answers (bge-large worst at 21.8%, MiniLM-L-2 best at 4.5%). The smallest θ that yields fp=0 on real STRICT is BELOW the candidate-bench NEG max for every model — meaning at the runtime-safe θ, catch on the 12 candidate-bench NEG = 0/12. Structural reason: real STRICT answers have a much wider score distribution (bge-large: min -2.20, p10 +2.99, p50 +5.80) than the tight contrived candidate-bench POS band, so the distributions overlap heavily and no θ separates them. The relevance reranker CANNOT be promoted to a runtime demotion-only veto on this design (joins #000049 §7 #27 in 'partial-on-candidate-bench, fails the real-traffic gate'). The PRIMARY below is the candidate-bench winner kept for reference; the operating point is null because no runtime threshold is viable. The score remains useful as: (a) an advisory soft signal (render-tail `· relevance: X.Y`, no thresholding); (b) one factor in a multi-signal advisory (relevance + title-mismatch + coherence + deflection); (c) contrastive use (Q→A vs claim→source delta, or relative ranking within an answer's claims). Reuses [nli] extra (transformers+torch+optimum).",
  "relevance_model_version": "relevance-shadow-v1-bge-reranker-large",
  "hf_repo": "BAAI/bge-reranker-large",
  "license": "MIT",
  "source_url": "https://huggingface.co/BAAI/bge-reranker-large",
  "approx_mb": 560,
  "score_shape": "single logit per (query, document) pair, NOT a softmax — higher = more relevant; trained on MS MARCO passage-ranking, so the score is well-calibrated for 'is this doc relevant to this query?' and uncalibrated for any other notion of similarity",
  "max_length": 256,
  "truncation_policy": "query_then_document@v1",
  "pair_serialization": "relevance_pair@v1",
  "demote_below_score": null,
  "demote_below_score_comment": "STAYS NULL pending fox+dav1d sign-off, but step 3 part B (2026-05-13 — real-context deflection [n=20] + mis-cite [n=20] fixtures) confirmed VIABLE: bge-reranker-large + cleaned + θ ≤ -2.42 catches 100% of real-context MIS-CITE (the Zionist-shape motivating failure) AND 55% of real-context DEFLECTION, with strictly 0/808 real-STRICT FP. Mis-cite max bge-score is -3.81 vs STRICT min -2.42 — strictly separable for mis-cite. Deflection harder (some deflections score similarly to weak STRICT). MiniLM-L-6 + cleaned + θ=+3: 100% mis-cite + 65% deflect at 0.4% STRICT FP — cost-pick. Setting demote_below_score requires the runtime promotion decision per #000049 §7 #2 — at that point relevance_policy_hash (model+max_length+cleaning+θ+aggregation) folds into governance_policy_hash.",
  "runtime_viability": {
    "as_runtime_demotion_veto": "VIABLE at low-to-moderate FP — REVISED 2026-05-13 §3.2.2 step 3. The step-2 'NOT VIABLE' verdict was an artifact of using candidate-bench NEG as the recall denominator (tight contrived NEG band overlapped real STRICT). With REAL-CONTEXT deflection fixtures as the recall side (bench/fixtures/5f/relevance-deflection-realcontext-v1.jsonl, 20 hand-crafted Q→A deflections against real bench-qa questions), bge-reranker-large catches 11/20 = 55% at strictly 0% real-STRICT FP (θ ≤ -2.42), and pushes to ~95% at ~2% FP. MiniLM-L-6 (cleaned): 65% catch at 0.4% FP (θ=+3), 90% at 1.9% FP (θ=+5). These ARE usable runtime soft-veto operating points.",
    "step3_pareto_minilm_l_6_cleaned_2026_05_13_partB": [
      {"theta": -5.29, "miscite_catch": "14/20 = 0.70", "deflect_catch": "2/20 = 0.10", "combined": "16/40 = 0.40", "strict_fp": "0/808 = 0.000", "note": "fp=0 floor"},
      {"theta": 0.0,   "miscite_catch": "19/20 = 0.95", "deflect_catch": "7/20 = 0.35", "combined": "26/40 = 0.65", "strict_fp": "2/808 = 0.003"},
      {"theta": 2.0,   "miscite_catch": "19/20 = 0.95", "deflect_catch": "11/20 = 0.55", "combined": "30/40 = 0.75", "strict_fp": "2/808 = 0.003"},
      {"theta": 3.0,   "miscite_catch": "20/20 = 1.00", "deflect_catch": "13/20 = 0.65", "combined": "33/40 = 0.82", "strict_fp": "3/808 = 0.004", "note": "cost-pick — 100% mis-cite + 65% deflect at sub-1% FP"},
      {"theta": 5.0,   "miscite_catch": "20/20 = 1.00", "deflect_catch": "18/20 = 0.90", "combined": "38/40 = 0.95", "strict_fp": "15/808 = 0.019"},
      {"theta": 7.0,   "miscite_catch": "20/20 = 1.00", "deflect_catch": "20/20 = 1.00", "combined": "40/40 = 1.00", "strict_fp": "84/808 = 0.104", "note": "too FP for runtime"}
    ],
    "step3_pareto_bge_large_cleaned_2026_05_13_partB": [
      {"theta": -2.42, "miscite_catch": "20/20 = 1.00", "deflect_catch": "11/20 = 0.55", "combined": "31/40 = 0.78", "strict_fp": "0/808 = 0.000", "note": "PRIMARY RECOMMENDATION — 100% mis-cite + 55% deflect at strictly 0% FP. Zionist-shape motivating failure fully covered."},
      {"theta": 0.0,   "miscite_catch": "20/20 = 1.00", "deflect_catch": "17/20 = 0.85", "combined": "37/40 = 0.93", "strict_fp": "15/808 = 0.019"},
      {"theta": 2.0,   "miscite_catch": "20/20 = 1.00", "deflect_catch": "18/20 = 0.90", "combined": "38/40 = 0.95", "strict_fp": "36/808 = 0.045"},
      {"theta": 3.0,   "miscite_catch": "20/20 = 1.00", "deflect_catch": "19/20 = 0.95", "combined": "39/40 = 0.97", "strict_fp": "71/808 = 0.088"}
    ],
    "step3_finding_miscite_vs_deflection": "Mis-cite is structurally much easier than deflection — bge-large mis-cite scores range -9.37 to -3.81 (max), strictly below STRICT min (-2.42). Deflection range -5.57 to +5.01 overlaps STRICT bottom. So the runtime veto is FULLY effective on mis-cite (the Zionist failure mode) and PARTIALLY effective on deflection. That's appropriate to the structural difference: mis-cite is 'wrong topic entirely', deflection is 'right topic, not answering'.",
    "as_advisory_soft_signal": "viable — surface the raw score on the render tail or as bench telemetry; no threshold needed",
    "as_multi_signal_factor": "TESTED 2026-05-13 (§3.2.2 step 3 part C) — DOES NOT LIFT precision/recall on the failure shapes §3.2 covers. diagnose_deflection fires on 0/40 of the deflection+miscite NEG fixtures because the fixtures are token-overlap-correct (which is the whole point — they're the cases the lexical sidecar misses); diagnose_coherence fires on 0/40 because the fixtures are well-formed. So UNION = relevance alone, INTERSECT = 0. The architectural finding is positive in a different way: the lexical sidecars and the relevance reranker cover NON-OVERLAPPING failure shapes (structural vs subject-anchor vs aboutness). Each signal owns its own slice cleanly; combining them is redundant on these cases.",
    "as_contrastive_signal": "TESTED 2026-05-13 — DOES NOT SEPARATE on real bench-qa data: 97% of STRICT-fires at cb-θ have BOTH Q→A and A→ctx_lead low (co-varying); only 3% match the 'high A→ctx, low Q→A' deflection signature. Contrastive rescue path closed — but it's no longer needed since real-context deflections (step 3) DID separate cleanly with absolute thresholds.",
    "preprocessing_cleaning_is_free_improvement": "claim-lattice metadata [E\\d+ | … | hash: '…'] depresses scores by ~3-6 pts on cleaned vs raw; clean_for_relevance is baked into check_* by default. Universal win across all 6 swept models (-0.5 to -5.6 pts FP)."
  },
  "alternates": [
    {"relevance_model_version": "relevance-shadow-v1-bge-reranker-large", "hf_repo": "BAAI/bge-reranker-large", "license": "MIT", "approx_mb": 560, "note": "LARGE end of the spectrum — added §3.2.1 round-2 (does bigger BGE beat bge-base?)"},
    {"relevance_model_version": "relevance-shadow-v1-bge-reranker-base", "hf_repo": "BAAI/bge-reranker-base", "license": "MIT", "approx_mb": 280, "candidate_bench_margin": 3.076, "note": "BAAI general-purpose reranker; second-best candidate-bench margin after electra-base (round 1)"},
    {"relevance_model_version": "relevance-shadow-v1-ms-marco-MiniLM-L-6-v2", "hf_repo": "cross-encoder/ms-marco-MiniLM-L-6-v2", "license": "Apache-2.0", "approx_mb": 80, "candidate_bench_margin": 3.053, "note": "the original primary before §3.2.1 candidate-bench"},
    {"relevance_model_version": "relevance-shadow-v1-ms-marco-MiniLM-L-12-v2", "hf_repo": "cross-encoder/ms-marco-MiniLM-L-12-v2", "license": "Apache-2.0", "approx_mb": 130, "candidate_bench_margin": 2.663, "note": "deeper L-12 MiniLM; round-1 worst — deeper ≠ better here"},
    {"relevance_model_version": "relevance-shadow-v1-ms-marco-MiniLM-L-4-v2", "hf_repo": "cross-encoder/ms-marco-MiniLM-L-4-v2", "license": "Apache-2.0", "approx_mb": 50, "note": "SMALL end — added §3.2.1 round-2 (does L-4 still separate the candidate-bench?)"},
    {"relevance_model_version": "relevance-shadow-v1-ms-marco-MiniLM-L-2-v2", "hf_repo": "cross-encoder/ms-marco-MiniLM-L-2-v2", "license": "Apache-2.0", "approx_mb": 30, "note": "TINY end — added §3.2.1 round-2 (the smallest reranker; floor of the size spectrum)"}
  ],
  "candidate_bench_results_2026_05_13_round2": {
    "fixtures": "bench/fixtures/5f/relevance-aboutness-v1.jsonl (14 POS + 12 NEG)",
    "grid_output": "bench/results/relevance-aboutness-candidatebench-round2.json",
    "size_spectrum_mb": "30 → 50 → 80 → 110 → 130 → 280 → 560 (~20×)",
    "rank_by_sep_margin": [
      {"model": "bge-reranker-large",     "approx_mb": 560, "margin": 6.285, "min_pos":  4.643, "max_neg": -1.641, "fp0_theta": 4.643, "catch_neg": "12/12"},
      {"model": "ms-marco-electra-base",  "approx_mb": 110, "margin": 5.349, "min_pos":  0.050, "max_neg": -5.300, "fp0_theta": 0.050, "catch_neg": "12/12"},
      {"model": "ms-marco-MiniLM-L-4-v2", "approx_mb":  50, "margin": 3.187, "min_pos":  6.736, "max_neg":  3.549, "fp0_theta": 6.736, "catch_neg": "12/12"},
      {"model": "bge-reranker-base",      "approx_mb": 280, "margin": 3.076, "min_pos":  2.395, "max_neg": -0.681, "fp0_theta": 2.395, "catch_neg": "12/12"},
      {"model": "ms-marco-MiniLM-L-6-v2", "approx_mb":  80, "margin": 3.053, "min_pos":  6.754, "max_neg":  3.701, "fp0_theta": 6.754, "catch_neg": "12/12"},
      {"model": "ms-marco-MiniLM-L-12-v2","approx_mb": 130, "margin": 2.663, "min_pos":  7.138, "max_neg":  4.475, "fp0_theta": 7.138, "catch_neg": "12/12"},
      {"model": "ms-marco-MiniLM-L-2-v2", "approx_mb":  30, "margin": -1.970, "min_pos": 3.371, "max_neg":  5.341, "fp0_theta": 3.371, "catch_neg": "10/12", "note": "NOT clean-separable — floor of the size spectrum; some NEG outscore some POS"}
    ],
    "findings": {
      "biggest_is_best_within_a_family": "bge-large (560MB) +6.285 > bge-base (280MB) +3.076 — within BGE family, doubling size doubles margin",
      "but_not_across_families": "electra-base (110MB) +5.349 beats bge-base (280MB) +3.076 and beats MiniLM-L-12 (130MB) +2.663 — architecture/training-corpus matters more than parameter count in this range",
      "deeper_is_not_better_within_MS_MARCO_MiniLM": "L-4 (50MB, +3.187) > L-6 (80MB, +3.053) > L-12 (130MB, +2.663) — adding depth from L-6 to L-12 HURTS; same 'deeper ≠ better' pattern as #000049 §7 #24's bart-large-vs-deberta-large",
      "capacity_floor_between_L_2_and_L_4": "L-2 (30MB) NOT clean-separable (margin -1.970, 10/12 catch); L-4 (50MB) clean-separable (+3.187, 12/12) — the floor is between them",
      "primary_pick": "bge-reranker-large for the most cushion (+6.285), trading 5× size vs electra-base for ~17% more margin — picks the model most likely to survive a denominator growth per §7 #25's lesson; electra-base is the cost alternate if real-traffic-FP shadow sweep at step 2 shows bge-large isn't worth the size"
    },
    "note": "All 6 except L-2 clean-separate on this 26-pair candidate-bench. The thresholds above are CANDIDATE-BENCH ONLY — they WILL move when the denominator grows (§7 #25→#26 walked back twice on the NLI side; the same pattern is expected here). demote_below_score stays null until §3.2.2 step 2 (real-traffic shadow sweep on pooled bench-qa STRICT) sets it."
  },
  "rationale_for_cross_encoder_not_bi_encoder": "A bi-encoder (sentence-transformers, vec-style) gives an unsupervised cosine similarity that's good for retrieval *ranking* but not calibrated for the 'is this on-topic?' question we need here. A cross-encoder reranker is trained on (query, doc) pairs with relevance labels — its single logit IS the 'is this doc relevant to this query?' score we want. The architectural cross-link to #000050/#000051's vec hybrid: that work uses *bi-encoders* (fastembed, bge-small) for retrieval recall; this work uses a *cross-encoder* for downstream aboutness scoring. They are complementary — vec finds candidates, the cross-encoder scores them."
}
