# Ticket #000011 — SOFT_PREFLIGHT_HINT model-assisted sidecar

**Status:** closed · landed 2026-05-04 (zero-shot, full
implementation per §13)
**Opened:** 2026-05-04
**Scope:** Add a model-assisted preflight sidecar that augments the
deterministic detectors from #000010 with an LLM-driven shape
classifier. Strict guardrail per #000010 §18 / source doc: the
sidecar produces ONLY soft hints labeled `SOFT_PREFLIGHT_HINT`; it
cannot create `PREFLIGHT_OK` or `PREFLIGHT_BLOCKED` without
deterministic support. Hard rule (D1) preserved.
**Audience:** fox + future blackops shifts.
**Hard constraint:** Same as #000010 §1: no `schema_version`,
`canonicalization_version`, or `chunking_version` bumps; pure
additive policy folded into `governance_policy_hash`. The sidecar
NEVER alters the verifier's hard path; sidecar output lives on
result-dict + bench rows + the run-DAG preflight payload but is
clearly tagged `SOFT_*` so audit replay can distinguish hard from
soft signal.

---

## 1. Premise

The deterministic detectors from #000010 (temporal, contradiction,
false-premise-lite, out-of-corpus) are conservative by design —
lexical pattern matching catches obvious shapes & misses subtle
ones. A model-assisted preflight could catch:

- **Subtle false premises** that don't match the lite presupposition
  patterns (e.g. "Why does the Federal Reserve refuse to audit
  itself?" — false premise hidden in "refuse").
- **Multi-hop contradictions** invisible to lexical pair matching.
- **Style-of-reasoning needs** (compare/contrast, summarize, list)
  that don't surface in the quantifier classifier.
- **Implicit time-anchors** ("now that the AI bubble has popped")
  the lexical temporal detector misses.

The architectural rule from #000010 §18 forbids putting an LLM in
the hard preflight path. But a *sidecar* — soft signal, never gates
admissibility, can be wrong without breaking guarantees — is
admissible under the same separation-of-concerns the substrate
already maintains for hard vs soft hashes.

## 2. What the sidecar produces

A separate `SoftPreflightHint` dataclass distinct from `QuestionState`:

```python
@dataclass(frozen=True)
class SoftPreflightHint:
    raw_question: str
    sidecar_version: str  # e.g. "soft-preflight-v0.1"
    classifier_label: str  # e.g. SOFT_FALSE_PREMISE_SUSPECTED
    confidence: float  # 0.0-1.0, never used for hard decisions
    rationale: str  # one-line explanation from the model
    timestamp: int  # monotonic for replay debugging
    model_profile_id: str  # which model produced the hint
```

The hint always carries the `SOFT_` prefix on its `classifier_label`
so it can never be mistaken for a deterministic verdict by:

- The audit-line tail renderer in `arborist/cli.py:_render_warrant_tail`
- The verifier (which doesn't read sidecar fields anyway)
- Bench aggregations
- Operators inspecting bench JSONL or run-DAG blobs

## 3. Where it lives

```
arborist/qa/soft_preflight.py    — sidecar implementation
                                  (new module)
```

Mirrors `arborist/qa/inspect.py` (the existing read-only sidecar
diagnostic for span classification) — same architectural pattern:
soft signal, never enters proof path, never written to providence
cache, never bumps governance hash on use.

Wired into `query()` and `runner.ask()` as an optional pre-LLM call
gated on `soft_preflight_enabled` (default False). Adds ~200ms of
latency per call (one extra LLM round-trip on the local Hermes
endpoint).

## 4. Cache binding

Soft sidecar hints DO NOT enter `governance_policy_hash`. Two cache
rows with the same hard preflight contract but different soft hints
SHARE a cache_key — sidecar disagreement is not a cache split.

Soft hints DO surface on the run-DAG preflight payload's
`question_state.soft_preflight_hint` field (new optional sub-field).
The hint is hash-bound for audit replay, but two runs with different
sidecar outputs can still hit the same cache. This is intentional:
audit can see what soft signal the sidecar emitted; cache identity
remains driven by the hard policy.

## 5. Implementation sketch

```python
# arborist/qa/soft_preflight.py

def soft_preflight_question(
    question: str,
    *,
    chat_client: ChatClient,
    model_id: str,
    policy: dict,
) -> SoftPreflightHint:
    """Call the LLM to produce a soft preflight hint. Pure
    sidecar: caller MUST treat the output as advisory only.
    Never enters the verifier, never gates admissibility."""
    if not policy.get("soft_preflight_enabled", False):
        return _stub_hint("SOFT_DISABLED", "sidecar off")

    # Short, focused system prompt asking the model to classify
    # the question shape from a fixed enum:
    #   SOFT_WELL_FORMED
    #   SOFT_FALSE_PREMISE_SUSPECTED
    #   SOFT_CONTRADICTION_SUSPECTED
    #   SOFT_TIME_SENSITIVE
    #   SOFT_SCOPE_AMBIGUOUS
    #   SOFT_OUT_OF_CORPUS_LIKELY
    # ...
    # Return one of the enum tokens + a one-line rationale.
    # Constrained generation (max_tokens=128, temperature=0.0
    # for deterministic-ish output, top_p=1.0).
    raw = chat_client.chat_completion(
        messages=[
            {"role": "system", "content": SOFT_PREFLIGHT_SYSTEM_PROMPT},
            {"role": "user", "content": question},
        ],
        model=model_id,
        temperature=0.0,
        max_tokens=128,
    )
    label, rationale = _parse_soft_hint(raw)
    return SoftPreflightHint(
        raw_question=question,
        sidecar_version="soft-preflight-v0.1",
        classifier_label=f"SOFT_{label.upper()}",
        confidence=0.5,  # soft signals never claim hard confidence
        rationale=rationale,
        timestamp=int(time.monotonic_ns()),
        model_profile_id=model_id,
    )
```

## 6. Integration points

- `arborist/qa/query.py` — call `soft_preflight_question()` after
  `preflight_question()` (the hard call) but before the main LLM
  call. Soft hint surfaces on result dict as
  `soft_preflight_hint`; merges into the run-DAG `preflight`
  stage payload as a sub-field of `question_state`.
- `arborist/qa/runner.py` — same wiring.
- `arborist/cli.py` — new `--soft-preflight` flag (per-call
  override; default off matches policy default).
- `bench/qa_sweep.py` — bench rows pick up `soft_preflight_label`
  + `soft_preflight_confidence` as bounded-projection fields.
- `arborist/cli.py:_render_warrant_tail` — soft hints render as
  `· soft: <label>` on the audit-line (clearly differentiated from
  the hard `· false premise` etc. tails). Operator can tell at a
  glance whether a tail is hard or soft.

## 7. Bench plan

After the sidecar lands:

1. **Soft-only A/B** on the metacog-trigger fixture (#000010
   §13.1 + the `bench/qa_questions_metacog_subset.txt` from the
   2026-05-04 work). Measure: does the soft sidecar agree with
   the hard detectors on the cases where the hard detectors fire?
   Disagreement rate ≤ 10% signals the sidecar is reliable
   enough to surface to operators.
2. **Soft-on subtle-cases A/B** with hand-crafted fixtures of
   subtle false premises / multi-hop contradictions that DON'T
   trigger the hard detectors. Measure: does the sidecar add
   coverage on shapes the hard detectors miss?
3. **Latency cost** — the extra ~200ms per call multiplied by
   bench cycle counts. Bench cycle latency budget is
   currently ~17-35s/call (Hermes); adding 200ms is ~1% overhead.
   Measure to confirm.

## 8. What this ticket does NOT do

- **Does NOT enter the verifier proof path.** D1 preserved.
- **Does NOT bump `schema_version` / `canonicalization_version` /
  `chunking_version`**. Pure additive.
- **Does NOT change cache identity.** Soft hints don't fold into
  `governance_policy_hash`.
- **Does NOT replace the hard detectors.** Hard detectors remain
  the source of truth for `PREFLIGHT_OK` / `PREFLIGHT_BLOCKED`.
- **Does NOT promote a soft hint to hard** without deterministic
  support. The sidecar can claim `SOFT_FALSE_PREMISE_SUSPECTED`
  but the hard detector must agree before `false_premise_suspected`
  enters `logical_statuses`.

## 9. Open questions

- Use a separate cheap model for the sidecar (e.g. Hermes-3 itself,
  or a smaller model) vs the same model that produces the answer?
  Bench data needed.
- One-call vs streaming-batch? At 200ms/call across a 75-question
  bench (n=3), one-call adds ~22s; batched (e.g. 5 questions at
  once) could compress this significantly but complicates the
  per-row provenance binding.
- Confidence calibration — initial proposal is `0.5` literal for
  any soft hint. Future refinement might calibrate via empirical
  agreement-rate with the hard detectors.

## 10. Status

Closed · landed 2026-05-04 in the same commit batch as the
metacog-trigger fixture expansion + --show-preflight full clause
render + latency profile. Source-doc reference: ticket #000010
§18 + `~/Downloads/meta-cognition_for_hermes(1).txt` §18.

The deterministic substrate from #000010 + #000009 is the
provable foundation. SOFT_PREFLIGHT_HINT is an enrichment
layer that operates strictly within the substrate's "hard
hash never enters the proof path" rule (cf. whitepaper
§13 decision 8).

## 11. What landed

- **`arborist/qa/soft_preflight.py`** — `SoftPreflightHint`
  dataclass + `soft_preflight_question()` pure function.
  9 canonical labels (`SOFT_WELL_FORMED`,
  `SOFT_FALSE_PREMISE_SUSPECTED`, `SOFT_CONTRADICTION_SUSPECTED`,
  `SOFT_TIME_SENSITIVE`, `SOFT_SCOPE_AMBIGUOUS`,
  `SOFT_OUT_OF_CORPUS_LIKELY`, `SOFT_BROAD_QUANTIFIER`,
  `SOFT_MULTI_HOP_REASONING`, `SOFT_SUBJECTIVE`) plus stub
  states (`SOFT_DISABLED`, `SOFT_PARSE_FAIL`). Constrained-
  generation prompt asking the model to pick ONE label + a
  one-line rationale. Failure-closed across every parse path
  (raises → `SOFT_PARSE_FAIL`, drift → `SOFT_PARSE_FAIL`).
  `SOFT_PREFLIGHT_VERSION = "soft-preflight-v0.1"`.

- **`arborist/qa/query.py`** — wired into the post-classifier /
  pre-retrieval segment. One short LLM call (~200ms median),
  result surfaces as `soft_preflight_hint` on the result dict
  + `soft_preflight_ms` in the timings dict.

- **`arborist/qa/runner.py` + `arborist/qa/query.py`** —
  `soft_preflight_enabled: False` policy default; cache row
  identity unchanged (NOT folded into `_VERIFIER_POLICY_FIELDS`
  per §4 — soft hints don't gate cache identity).

- **`arborist/cli.py`** — `--soft-preflight` flag on
  `arborist query`. Audit-line tail renders soft hints as
  `· soft: <label>` (e.g. `· soft: time sensitive`) — distinct
  from hard tails so operators see the signal separation at a
  glance. SOFT_DISABLED / SOFT_PARSE_FAIL / SOFT_WELL_FORMED
  suppress (no actionable signal).

- **`tests/test_soft_preflight.py`** — 25 new tests pinning:
  default-OFF behavior, parse-failure modes, label
  normalization (SOFT_ prefix enforced even when model drops
  it), all 8 actionable labels round-trip, fail-closed on
  client exceptions, dataclass JSON round-trip,
  rationale-length cap.

## 12. Live verification

End-to-end smoke test on `When did Mr. Burns become Homer's
biological father?` with `--soft-preflight`:

```
EVIDENCE-WARRANTED · via claim_lattice · false premise · soft: time sensitive  1/1  16.4s
```

Both signals compose: hard `· false premise` from #000010
deterministic detector, soft `· soft: time sensitive` from
#000011 sidecar. The model classified a different shape than
the hard detector — by design (soft hints are independent
advisory signals, not redundant with the hard layer).

## 13. Bench plan (deferred)

Per §7 of this ticket — soft-only A/B against the metacog-
trigger fixture (`bench/qa_questions_metacog_subset.txt` —
expanded 2026-05-04 to 28 questions), measure agreement-rate
between soft sidecar and hard detectors. Disagreement ≤ 10%
signals the sidecar is reliable enough for default-on
consideration. This bench is queued but not run in the same
commit cycle as implementation.
