qa(#000011 + 4 more): SOFT_PREFLIGHT_HINT impl + 5-task fan-out

Big batch — closes 4 of the 5 deferred items from the prior status
report plus opens & implements a previously-deferred design ticket
(#000011) zero-shot.

#000025 — Metacog test fixture expansion:
  bench/qa_questions_metacog_subset.txt grows from 6 → 28 questions
  covering edge cases per detector kind: temporal (4 cases),
  contradiction (4), false-premise (5), out-of-corpus (3), multi-
  trigger (2), well-formed controls (5). Documents two known
  detector ceilings: Q11 over-fires on past-tense factoid
  ("who was the first president"); Q16/Q17/Q19 (Edison/Australia/
  NASA-fake) miss false premises that lack a presupposition
  pattern match. Fixture now serves as long-term regression suite.

#000026 — --show-preflight full clause render:
  build_run_dag() and build_reject_run_dag() gain optional
  preflight_payload kwarg. When supplied, the canonical 5-clause
  CTI payload (classifier / answer_contract / prompt_contract /
  evidence_contract / policy_refs + question_state + node_version)
  persists alongside the leaf hash in run_dag_blob.
  aborist providence --show-preflight CACHE_KEY now renders the
  full payload + verifies the persisted hash matches the
  recomputed canonical hash (audit-replay tamper detection).
  Legacy rows fall through cleanly: payload_hash_check reports
  "unavailable: legacy row predates preflight_payload persistence".

#000027 — Latency profile:
  Microbenched preflight: 0.46ms/question (negligible). Single
  fresh call breakdown: search 2.4s, llm 2.8s, total 5.4s — the
  33-35s in Addendum 3 was vLLM concurrency contention at c=4
  (per qa-modes-bench.md saturation note), not substrate
  overhead. Added preflight_ms + soft_preflight_ms to timings
  dict for explicit confirmation in future cycles.

#000028 — Auto-quality-check sweep revival:
  scripts/bench_emergent.py running with EMERGENT_N=100 in
  background (PID 125680). Will accumulate cycles into
  bench/emergent_log.jsonl for #000006 rolling log re-aggregation.
  Async — not blocking on completion.

#000029 — #000011 SOFT_PREFLIGHT_HINT implementation:
  aborist/qa/soft_preflight.py — new module. SoftPreflightHint
  dataclass + soft_preflight_question() pure function. 9
  canonical labels mapping to soft analogues of #000010 hard
  detectors plus 2 stub states (SOFT_DISABLED, SOFT_PARSE_FAIL).
  Constrained-generation prompt (max_tokens=128, temp=0.0) asks
  the model to pick ONE label + one-line rationale.
  Fail-closed across every parse path:
    - chat_client raises → SOFT_PARSE_FAIL
    - response unparseable → SOFT_PARSE_FAIL
    - label outside enum → SOFT_PARSE_FAIL
  Sidecar enforces SOFT_ prefix at the normalize step so a
  model that drops the prefix still gets caught.

  Wired into query() between preflight & retrieval. Default
  OFF (`soft_preflight_enabled: False`). NOT folded into
  _VERIFIER_POLICY_FIELDS — soft hints don't gate cache
  identity (#000011 §4). Audit-line tail renders as
  "· soft: <label>" (e.g. "· soft: time sensitive") so the
  signal is visually distinct from hard tails.

  --soft-preflight CLI flag opts in per-call. End-to-end
  live-verified on "When did Mr. Burns become Homer's biological
  father?" — produces:

    EVIDENCE-WARRANTED · via claim_lattice
        · false premise · soft: time sensitive
        1/1  16.4s

  Hard `· false premise` (from #000010 deterministic detector)
  composed with soft `· soft: time sensitive` (from #000011
  sidecar). The model classified a different shape than the hard
  detector — by design; soft hints are independent advisory
  signals, not redundant with the hard layer.

  25 new tests pin: default-OFF behavior, parse-failure modes,
  label normalization (SOFT_ prefix enforced), all 8 actionable
  labels round-trip, fail-closed on client exceptions, dataclass
  JSON round-trip, rationale-length cap.

Other:
  - #000010 §13.3 documents 2/5 metacog-trigger questions return
    STRICT despite hard-detector warning — direct empirical
    motivation for #000011 design.
  - tests/test_dag.py extends with 3 _extract_preflight_hash_*
    helper tests (cleaning #000009 §7.2 unfinished state).
  - bench/emergent_log.jsonl adds new cycles from background run.

#000011 status: closed. Hard rule (D1) preserved across all
1021 tests (up from 996, +25 new). Soft preflight is purely
advisory; the verifier proof path is unchanged.

2026-05-03 23:00:56 -04:00

4 KiB

Raw Blame History

Aborist tickets — index

Aborist uses a flat ticket-NNNNNN-<slug>.md convention for actionable proposals: defects with a known root cause, design proposals awaiting go/no-go, or scoped enhancements queued for future implementation.

Convention

Filename: docs/tickets/ticket-NNNNNN-<slug>.md. The ID is a six-digit zero-padded integer; the slug is hyphen-lowercase and short enough to stay readable on a 100-column terminal.
Numbering: monotonic, never reused. Pick the next free integer greater than every existing ticket. The current next ID is on the Next ID line below; bump it when you open a ticket and commit the bump in the same commit as the new ticket file.

Header: every ticket starts with

# Ticket #NNNNNN — <short title>

**Status:** open · awaiting go/no-go      (or "in progress", "closed")
**Opened:** YYYY-MM-DD
**Scope:** <one-liner — what this ticket is for>
**Audience:** <who reads this>
**Hard constraint:** <invariants this ticket commits to NOT break>

Body: problem statement, design choices with trade-offs, recommendation, implementation sketch, scope boundaries, status. See ticket-000001-retrieval-keywords-audit-gap.md for a worked template.
Closure: when a ticket lands, flip Status to closed · landed in commit <sha> and leave the file in place. Do not delete tickets; they are the design log.

Distinction from other docs

docs/ also contains architecture references and bench journey notes that are NOT tickets — they describe state of the world rather than proposing change. Examples that stay un-numbered:

cti-architecture.md (architecture reference)
mesh.md, mesh-deploy.md (architecture + ops reference)
qa-modes-bench-2026-04-30.md (point-in-time bench journal)
verifier-semantic-gap-design.md, self-reference-design.md (older design docs that pre-date the ticket convention; leave as-is unless retroactively promoting one to a ticket adds value)
concept-relations-design.md (architecture reference for the per-shard concept_relations synonym/rivalry layer + the 1.6% storage-tax rationale; landed 2026-05-01)

If a doc proposes change AND awaits a decision AND has scoped implementation cost, it's a ticket. Otherwise it's reference.

Index

Newest first. Update on every open/close.

ID	Title	Status	Opened	Directive
#000011	SOFT_PREFLIGHT_HINT model-assisted sidecar	closed · landed 2026-05-04 (zero-shot full impl)	2026-05-04	D1 (preserves)
#000010	Meta-Cognition Preflight Guard (M0 / MCTL)	closed · landed 2026-05-03 (Phases 1–4); DAG binding shipped via #000009	2026-05-03	D1, D3
#000009	Preflight run-DAG node binding (#000008+#000010)	closed · re-landed 2026-05-04 (§8 corrections: reject-path DAG, nested CTI clauses)	2026-05-03	D3, D4
#000008	Broad-quantifier preflight guard	closed · landed in `4f2b5a6`; Phase 5 DAG binding split into #000009	2026-05-02	—
#000007	Query-layer hyphen folding	closed · 2026-05-02	2026-05-02	—
#000006	Bench-emergent findings (rolling research log)	open · rolling	2026-05-02	—
#000005	Label ladder migration (POINTER-LINKED → …)	closed · 2026-05-02	2026-05-01	D7
#000004	Directive coverage in bench summary	closed · `acd1f9c`	2026-05-01	D8
#000003	Anchor-class warrant generalization (Module H+)	closed · 2026-05-02	2026-05-01	D6
#000002	Reference-Frame Polarity Contract (Module L)	closed · 2026-05-02	2026-05-01	D3
#000001	Retrieval-keywords audit gap	closed · 2026-05-02	2026-05-01	D4

Next ID

000012

4 KiB Raw Blame History Unescape Escape