qa(#000009): preflight stage binds into run_dag_root
Closes ticket #000009 zero-shot. Scope expanded to cover BOTH ticket #000008 (broad-quantifier) AND ticket #000010 (meta-cognition) preflight contracts in a single combined node — both share the same audit-replay gap and inserting two separate nodes between question and retrieval was operationally awkward. aborist/qa/dag.py: + preflight_node_hash() — combines QuestionState + quantifier classifier output + behavioral policy_state into one canonical SHA-256 hex. + build_run_dag() gains optional preflight_hash parameter. When supplied, inserts {"stage": "preflight", "hash": ...} at position 1 (between question and retrieval). Backward-compat: None → original 7/9-stage shapes preserved for legacy run_dag_root re-validation. Quote-mode: 7 → 8 stages with preflight. Pointer-mode CTI: 9 → 10 stages with preflight. aborist/qa/query.py + runner.py: Both build the preflight payload from question_state + quantifier dict + 10-field policy_state (guard_enabled, guard_apply_caps, guard_apply_caps_mode_gated, claim_cap_resolved, claim_cap_actually_applied, reminder_enabled, reminder_eligible, reject_broad_active, metacognition_enabled, block_on_contradiction). This means two cache rows that share the same question + same model output + same verifier verdict but DIFFERENT preflight policy state now produce different run_dag_root values. Audit replay can pin the policy decision per row. 9 new tests in tests/test_dag.py: - preflight_node_hash determinism - hash bumps on question_state change - hash bumps on policy_state change (the audit-replay payoff) - all-None defensive shape - 7→8 stage transition (quote mode) - 9→10 stage transition (pointer mode), preflight at index 1 - run_dag_root bumps when preflight_hash bumps - verify_run_dag round-trips through preflight stage Live verification: latest providence_cache row carries ['question', 'preflight', 'retrieval', 'evidence_map', 'prompt', 'raw_answer', 'parsed_claim_lattice', 'verify', 'render', 'final_label'] — preflight stage living in the persisted DAG. 987 tests passing (9 new); 36 skipped. Tickets: #000009 status: closed · landed 2026-05-03 (zero-shot) #000010 cross-ref updated: "DAG binding shipped via #000009" What's NOT in this ticket (logged in §7.2): - CLI flag for inspecting preflight node from cache_key - Bench harness preflight_hash field for cross-row comparison - SOFT_PREFLIGHT_HINT (model-assisted preflight sidecar)
This commit is contained in:
parent
f2bbe512db
commit
c36e85c86c
6 changed files with 367 additions and 16 deletions
|
|
@ -57,8 +57,8 @@ Newest first. Update on every open/close.
|
|||
|
||||
| ID | Title | Status | Opened | Directive |
|
||||
|----------|------------------------------------------------|-----------------------|------------|-----------|
|
||||
| #000010 | Meta-Cognition Preflight Guard (M0 / MCTL) | closed · landed 2026-05-03 (Phases 1–4); Phase 5 DAG join #000009 | 2026-05-03 | D1, D3 |
|
||||
| #000009 | Quantifier preflight run-DAG node binding | open · awaiting go/no-go | 2026-05-03 | D3, D4 |
|
||||
| #000010 | Meta-Cognition Preflight Guard (M0 / MCTL) | closed · landed 2026-05-03 (Phases 1–4); DAG binding shipped via #000009 | 2026-05-03 | D1, D3 |
|
||||
| #000009 | Preflight run-DAG node binding (#000008+#000010) | closed · landed 2026-05-03 (zero-shot) | 2026-05-03 | D3, D4 |
|
||||
| #000008 | Broad-quantifier preflight guard | closed · landed in `4f2b5a6`; Phase 5 DAG binding split into #000009 | 2026-05-02 | — |
|
||||
| #000007 | Query-layer hyphen folding | closed · 2026-05-02 | 2026-05-02 | — |
|
||||
| #000006 | Bench-emergent findings (rolling research log) | open · rolling | 2026-05-02 | — |
|
||||
|
|
|
|||
|
|
@ -1,11 +1,13 @@
|
|||
# Ticket #000009 — Quantifier preflight run-DAG node binding
|
||||
# Ticket #000009 — Preflight run-DAG node binding
|
||||
|
||||
**Status:** open · awaiting go/no-go
|
||||
**Status:** closed · landed 2026-05-03 (zero-shot)
|
||||
**Opened:** 2026-05-03
|
||||
**Scope:** Bind the broad-quantifier preflight contract from
|
||||
ticket #000008 into the per-run Merkle-DAG so the classifier output
|
||||
+ cap-application decision are hash-bound rather than just
|
||||
surface-level result-dict fields.
|
||||
**Scope (expanded 2026-05-03):** Bind BOTH ticket #000008's broad-
|
||||
quantifier preflight contract AND ticket #000010's meta-cognition
|
||||
QuestionState into the per-run Merkle-DAG. Both share the same
|
||||
audit-replay gap (per #000010 §12.6) and inserting two separate
|
||||
nodes between `question` and `retrieval` is operationally awkward;
|
||||
single combined `preflight` node carries both payloads.
|
||||
**Audience:** fox + future blackops shifts.
|
||||
**Hard constraint:** No `schema_version`, `canonicalization_version`,
|
||||
or `chunking_version` bumps. Same constraint #000008 §1 carried.
|
||||
|
|
@ -210,6 +212,71 @@ This ticket closes the loop.
|
|||
|
||||
## 7. Status
|
||||
|
||||
Open · awaiting go/no-go. Mechanism is well-understood from
|
||||
#000008 §9.5; this ticket is the audit-binding cleanup, not new
|
||||
behavior.
|
||||
**Closed · landed 2026-05-03 (zero-shot).** Mechanism shipped end-
|
||||
to-end:
|
||||
|
||||
- `aborist/qa/dag.py:preflight_node_hash()` — combines
|
||||
QuestionState + quantifier classifier output + behavioral
|
||||
policy_state into one canonical SHA-256 hex.
|
||||
- `aborist/qa/dag.py:build_run_dag()` — new optional
|
||||
`preflight_hash` parameter inserts a `preflight` stage at
|
||||
position 1 (between `question` and `retrieval`). Quote-mode
|
||||
shape becomes 8 stages; pointer-mode CTI shape becomes 10
|
||||
stages. Backward-compatible: when `preflight_hash` is None,
|
||||
shapes stay 7/9 exactly so legacy `run_dag_root` values
|
||||
re-validate.
|
||||
- `aborist/qa/query.py` + `aborist/qa/runner.py` — both build
|
||||
the preflight payload from `question_state`, the quantifier
|
||||
dict, and a 10-field `policy_state` capturing the *behavioral*
|
||||
decisions taken on this run (guard_enabled, apply_caps_active,
|
||||
claim_cap_resolved, claim_cap_actually_applied, reminder_enabled,
|
||||
reminder_eligible, reject_broad_active, metacognition_enabled,
|
||||
block_on_contradiction, mode-gating bits).
|
||||
|
||||
9 new tests in `tests/test_dag.py` pin: hash determinism,
|
||||
question_state/policy_state independence, all-None defensive
|
||||
shape, 7→8 / 9→10 stage transitions, root change on policy flip,
|
||||
verify_run_dag round-trip with preflight stage.
|
||||
|
||||
987 tests passing (9 new); 36 skipped.
|
||||
|
||||
### 7.1 What this enables (audit replay)
|
||||
|
||||
Two cache rows that have:
|
||||
|
||||
- The same question
|
||||
- The same model output
|
||||
- The same verifier verdict
|
||||
|
||||
But different behind-the-scenes preflight policy state (e.g. cap
|
||||
applied vs not, reminder injected vs not, reject-broad path
|
||||
taken vs not) now produce **different `run_dag_root`** values.
|
||||
|
||||
Audit replay can:
|
||||
|
||||
```
|
||||
hash(preflight node) = h_pre
|
||||
→ pin: classifier output, cap decision, reminder decision,
|
||||
reject decision, metacog gates
|
||||
|
||||
If h_pre changes between two cache_keys for the same question,
|
||||
that row reflects a different preflight policy.
|
||||
|
||||
Allows: regression bisection ("which day did the cap default
|
||||
flip break our STRICT-rate?"), policy A/B reconstruction
|
||||
("show me all rows where cap was applied vs not"), cross-model
|
||||
diff ("hermes vs qwen on the same question with the same
|
||||
preflight contract").
|
||||
```
|
||||
|
||||
### 7.2 What's NOT in this ticket
|
||||
|
||||
- **CLI flag for inspecting preflight node**: a future
|
||||
`aborist providence --show-preflight <cache_key>` would render
|
||||
the preflight payload from `run_dag_blob`. Out of scope here.
|
||||
- **Bench harness preflight-hash field**: bench rows could
|
||||
surface `preflight_hash` (12-char prefix like `cache_key`)
|
||||
for cross-row comparison. Out of scope; can add later if
|
||||
bench analysis needs it.
|
||||
- **`SOFT_PREFLIGHT_HINT` (model-assisted preflight)**:
|
||||
reserved per #000010 §18 / source doc. Hard rule preserved.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue