engineering/arborist
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

qa(#000009): §8 corrections — reject-path DAG + nested CTI clauses

Browse source 
Architectural feedback at ~/Downloads/RESPONSE-ticket-000009-... .txt
(2026-05-04) flagged five gaps in the c36e85c landing. Most
critical: reject-broad early-return path emitted no run_dag_blob,
so audit replay couldn't see that a rejection happened (let alone
under what policy state).

A — reject-path DAG (the critical gap):

  aborist/qa/dag.py: build_reject_run_dag() — 3-stage minimal DAG
  question → preflight → final_label. final_label payload carries
  rejection_reason + answer_text_hash so two rejections under
  different policy state produce different roots.

  query.py reject path now wires it: returns run_dag_root +
  run_dag_blob on the rejection result dict. Live-verified end-
  to-end on `make query Q="winners of all major sports?"
  REJECT_BROAD=1 BURN=1`.

  Audit replay rule: 3 stages always means reject path. Operators
  can read the stage list and tell instantly without parsing the
  payload.

B — nested CTI clauses:

  preflight_node_hash() payload restructured from flat 3-key to
  nested 5-clause:

    classifier        — quantifier classifier output (#000008)
    answer_contract   — guard / cap / reject / metacog state (per-run)
    prompt_contract   — reminder enabled / injected / template_id
    evidence_contract — exposure budget, line discipline
    policy_refs       — governance_policy_hash, model_profile_hash,
                        answer_mode (reference, not raw policy)

  Plus question_state (metacog) as its own clause and top-level
  stage + node_version. Single DAG stage; nested clauses inside
  for diff legibility (feedback §3).

C — node_version field:

  PREFLIGHT_NODE_VERSION = "preflight-node-v1" pinned in the
  payload so legacy runs without the node can be unambiguously
  labeled `unavailable_legacy_run` by audit tools (feedback §9).

D — reference hashes only:

  policy_refs uses governance_policy_hash + model_profile_hash
  rather than bundling raw policy booleans. Avoids
  double-committing already-hashed state (feedback §4).

E — reminder_template_id:

  prompt_contract.reminder_template_id = "broad-quantifier-bounded-v1"
  or "broad-quantifier-unbounded-v1" depending on scope_bound_hint,
  populated only when reminder actually fires.

F — stage name kept as `preflight` (not `quantifier_preflight`):

  Node carries both #000008 quantifier AND #000010 metacognition
  payloads. node_version disambiguates schema for audit tools.

G — docs/cti-architecture.md update deferred to a small follow-up.

Bug fixes:
  - free-variable shadowing on verifier_policy_hash /
    model_profile_hash / question_hash — local re-imports inside
    the reject branch shadowed module-top imports used elsewhere
    in query() / runner(); now use the module-top names.
  - reject path question_hash signature: takes `mode=` not
    `dedup_mode=` — fixed in the reject DAG builder caller.

Hash compatibility:
  Rows written between c36e85c and this commit have hash payloads
  matching the OLD flat 3-key shape. The persisted run_dag_blob
  captures the actual payload that was hashed, so those rows
  still verify via verify_run_dag(). New rows use the nested
  5-clause shape.

7 new tests in tests/test_dag.py:
  - hash sensitivity to answer_contract / prompt_contract /
    policy_refs flips (audit-replay payoff demonstrations)
  - PREFLIGHT_NODE_VERSION pinning
  - reject DAG: 3-stage shape, root changes with preflight hash,
    round-trips through verify_run_dag

993 tests passing (6 net new); 36 skipped.

Live verification:
  make query Q="winners of all major sports?" REJECT_BROAD=1 BURN=1
  → status=broad_quantifier_rejected, run_dag_root populated,
    blob carries 3-stage shape.

  make query Q="winners of all major sports?" BURN=1
  → 10-stage shape preserved (question → preflight → retrieval
    → ... → final_label).

Ticket #000009 status: closed · re-landed 2026-05-04 with §8
corrections.
This commit is contained in:
russell@unturf.com 2026-05-03 18:49:56 -04:00
parent c36e85c86c
commit 111dda6160
No known key found for this signature in database
6 changed files with 683 additions and 118 deletions
Download patch file Download diff file Expand all files Collapse all files

171
aborist/qa/dag.py
View file

@ -99,44 +99,103 @@ def localize_failure(
return "answer"
PREFLIGHT_NODE_VERSION = "preflight-node-v1"
def build_preflight_node_payload(
*,
question_state: dict | None = None,
quantifier: dict | None = None,
answer_contract: dict | None = None,
prompt_contract: dict | None = None,
evidence_contract: dict | None = None,
policy_refs: dict | None = None,
) -> dict:
"""Build the canonical nested-clause payload for the preflight
DAG stage. Returns a JSON-ready dict; pair with
:func:`preflight_node_hash` to get the SHA-256 hex.
Five-clause structure per ticket #000009 §8.2 / feedback §3:
- ``classifier`` — quantifier classifier output (#000008):
intensity, matched_token, explicit_count, scope_bound_hint,
is_broad, classifier_version, operational_shape.
- ``answer_contract`` — guard / cap / reject decisions taken
on this run.
- ``prompt_contract`` — reminder enabled / injected /
template_id (#000008 §10.5).
- ``evidence_contract`` — exposure budget, one-claim-per-line
discipline (#000010 §10.4).
- ``policy_refs`` — governance_policy_hash + model_profile_hash
+ answer_mode. Reference-by-hash rather than raw policy
bundles (feedback §4: avoid double-committing
already-hashed state).
Plus the metacog ``question_state`` from #000010 — that's its
own clause for now (logical_statuses, false_premise_hints,
contradiction_pairs). It's hashed separately by
`metacognition.preflight_policy_hash` already.
Any clause may be None / empty — the resulting payload is
still stable. Includes ``node_version`` so legacy runs without
the node can be unambiguously labeled `unavailable_legacy_run`
by audit tools.
"""
return {
"stage": "preflight",
"node_version": PREFLIGHT_NODE_VERSION,
"classifier": dict(quantifier) if quantifier else {},
"answer_contract": dict(answer_contract) if answer_contract else {},
"prompt_contract": dict(prompt_contract) if prompt_contract else {},
"evidence_contract": dict(evidence_contract) if evidence_contract else {},
"policy_refs": dict(policy_refs) if policy_refs else {},
# Metacognition QuestionState carries
# ``preflight_policy_hash`` internally so flipping a metacog
# detector invalidates this clause via that field. Stored
# nested so audit-replay can read all metacog signal in one
# place without descending into the quantifier classifier.
"question_state": dict(question_state) if question_state else {},
}
def preflight_node_hash(
*,
question_state: dict | None = None,
quantifier: dict | None = None,
policy_state: dict | None = None,
answer_contract: dict | None = None,
prompt_contract: dict | None = None,
evidence_contract: dict | None = None,
policy_refs: dict | None = None,
) -> str:
"""Hash the preflight decision into a stable hex string.
"""Hash the preflight decision into a stable SHA-256 hex string.
Bundles three sources of preflight provenance into one
canonical hash that contributes to ``run_dag_root`` via the
new ``preflight`` stage in :func:`build_run_dag`:
Returns the hash of the nested-clause payload built by
:func:`build_preflight_node_payload`. See that function for the
five-clause structure.
- ``question_state`` — `QuestionState.to_dict()` from
``aborist.qa.metacognition`` (#000010): logical_statuses,
question_shape, false_premise_hints, contradiction_pairs,
temporal_sensitivity, etc. Carries
``preflight_policy_hash`` so policy flips invalidate the
stage hash automatically.
- ``quantifier`` — classifier output from
``aborist.qa.quantifier.classify_question_quantifier``
(#000008): intensity, matched_token, explicit_count,
scope_bound_hint, classifier_version.
- ``policy_state`` — the *behavioral* decisions taken on this
run that aren't already in the above (e.g. whether the cap
was actually applied vs just looked up, whether the reminder
was injected, whether reject-broad path was taken). These
are what differentiate two cache rows with the same
classifier output but different downstream effects.
Audit-replay payoff: two cache rows that share the same
question + same model output + same verifier verdict but
different preflight policy state produce different hashes
here, which propagate to ``run_dag_root`` via
:func:`build_run_dag`.
Any of the three may be None (legacy / opt-out); the hash is
still stable. Returns SHA-256 hex.
Backward compatibility note: Pre-2026-05-04 (`c36e85c`) callers
used a flat 3-key payload (`question_state` / `quantifier` /
`policy_state`). Hashes computed with that callsite will NOT
match this restructured callsite — `run_dag_root` values for
rows written between `c36e85c` and the current commit are
treated as a discrete generation; they're still verifiable by
re-reading `run_dag_blob` (the persisted blob captures the
payload that was actually hashed).
"""
payload = {
"stage": "preflight",
"question_state": question_state or {},
"quantifier": quantifier or {},
"policy_state": policy_state or {},
}
payload = build_preflight_node_payload(
question_state=question_state,
quantifier=quantifier,
answer_contract=answer_contract,
prompt_contract=prompt_contract,
evidence_contract=evidence_contract,
policy_refs=policy_refs,
)
return _sha256_hex(_canonical_json(payload))
@ -305,6 +364,60 @@ def build_run_dag(
return {"root": root_hex, "nodes": nodes}
def build_reject_run_dag(
*,
question_hash: str,
preflight_hash: str,
rejection_reason: str,
answer_text: str,
audit_mode: str = "UNGROUNDED",
verifier_method: str = "claim_lattice_pointer",
violations: list[dict] | None = None,
) -> dict:
"""3-stage reject-broad run-DAG: ``question → preflight →
final_label``.
Ticket #000009 §8.2 / 2026-05-04 feedback §6.2: preflight
rejection currently early-returns from ``query()`` before the
standard ``build_run_dag()`` runs, so reject rows have no
auditable Merkle commitment. This builder fills that gap with
a minimal DAG shape that captures the rejection without
pretending retrieval / prompt / raw_model_output happened.
The returned shape is INTENTIONALLY shorter than the standard
7/9/8/10-stage shapes — `audit replay can read the stage
list` and tell instantly that this row is a preflight
rejection: 3 stages always means reject path.
`final_label` carries the rejection_reason + answer_text hash
so two rejections that differ only in their (rendered)
rationale string still produce different roots. The
rejection_reason is the canonical string from the violation
(`"preflight rejection — broad-quantifier query with
unbounded scope. ..."`), NOT the operator-facing rendered
answer_text — that lets policy template changes invalidate
the hash even if the operator-visible text is unchanged.
"""
final_label_payload = {
"audit_mode": audit_mode,
"verifier_method": verifier_method,
"lookup_path": "preflight",
"rejection_reason": rejection_reason,
"answer_text_hash": _sha256_hex(answer_text or ""),
}
if violations is not None:
final_label_payload["violations"] = violations
final_label_hash = _sha256_hex(_canonical_json(final_label_payload))
nodes = [
{"stage": "question", "hash": question_hash},
{"stage": "preflight", "hash": preflight_hash},
{"stage": "final_label", "hash": final_label_hash},
]
leaves = [bytes.fromhex(n["hash"]) for n in nodes]
root_hex = MerkleTree.build(leaves).root.hex()
return {"root": root_hex, "nodes": nodes}
def verify_run_dag(blob: str | dict) -> bool:
"""Recompute the Merkle root from ``blob`` and check it matches.

206
aborist/qa/query.py
View file

@ -1733,19 +1733,111 @@ def query(
# we already know the answer set is undefined. Result schema
# mirrors a normal UNGROUNDED row so bench/CLI rendering
# stays consistent.
# Ticket #000009 §8.2 / feedback §6.2: build a 3-stage
# reject-broad DAG so the rejection is Merkle-auditable.
# Without this, two rejections under different policy state
# would be indistinguishable in audit replay.
from aborist.qa.dag import (
build_reject_run_dag,
preflight_node_hash as _pre_hash,
)
# question_hash / verifier_policy_hash / model_profile_hash
# already imported at module top; do NOT re-import locally
# (would shadow free-variable uses elsewhere in this function).
_reject_qhash = question_hash(
question,
mode=policy.get("question_dedup", "equivalence_class"),
)
_reject_ghash = verifier_policy_hash(policy)
_reject_mhash = model_profile_hash(model_id, revision, quantization)
_reject_rationale = (
"preflight rejection — broad-quantifier query with "
"unbounded scope. Operator opted in via "
"quantifier_reject_broad policy."
)
_reject_violations = [{
"kind": "BROAD_QUANTIFIER_REJECTED",
"intensity": quantifier["intensity"],
"matched_token": quantifier["matched_token"],
"scope_bound_hint": quantifier["scope_bound_hint"],
"reason": _reject_rationale,
}]
_reject_answer_text = (
"BROAD-QUANTIFIER PREFLIGHT REJECTED · scope unbounded\n\n"
f"Question matched {quantifier['intensity']} intensity "
f"(\"{quantifier['matched_token']}\") with an under-"
"specified universe. Narrow the question (e.g. add a "
"year, league, country, or category) or run with "
"--allow-broad for exploratory enumeration."
)
_reject_preflight_hash = _pre_hash(
question_state=question_state.to_dict(),
quantifier=quantifier,
answer_contract={
"guard_enabled": quantifier_guard_on,
"mode_gated": quantifier_mode_gated,
"apply_caps_active": quantifier_apply_caps,
"apply_caps_mode_gated": quantifier_caps_mode_gated,
"claim_cap_resolved": claim_cap_lookup,
"claim_cap_applied": None, # cap never reaches verifier on reject
"manual_quotes_allowed": False,
"evidence_pointer_required": True,
"allow_unbounded_enumeration": False,
"reject_broad_active": True,
"metacognition_enabled": bool(
policy.get("metacognition_enabled", True)
),
"block_on_contradiction": bool(
policy.get("metacognition_block_on_contradiction", False)
),
},
prompt_contract={
# Rejection skips the LLM, so no reminder ever fires.
"reminder_enabled": bool(
policy.get("quantifier_reminder_enabled", False)
),
"reminder_injected": False,
"reminder_template_id": None,
},
evidence_contract={
"max_evidence_ids_exposed": int(policy.get(
"claim_lattice_max_pointers_per_claim", 2
)),
"one_claim_per_line": True,
},
policy_refs={
"governance_policy_hash": _reject_ghash,
"model_profile_hash": _reject_mhash,
"answer_mode": answer_mode,
},
)
_reject_run_dag = build_reject_run_dag(
question_hash=_reject_qhash,
preflight_hash=_reject_preflight_hash,
rejection_reason=_reject_rationale,
answer_text=_reject_answer_text,
audit_mode="UNGROUNDED",
verifier_method=(
"claim_lattice_pointer"
if answer_mode == "claim_lattice_pointer"
else "claim_lattice"
if answer_mode == "claim_lattice"
else "quote"
),
violations=_reject_violations,
)
return {
"status": "broad_quantifier_rejected",
"audit_mode": "UNGROUNDED",
"cache_key": None,
"lookup_path": "preflight",
"answer_text": (
"BROAD-QUANTIFIER PREFLIGHT REJECTED · scope unbounded\n\n"
f"Question matched {quantifier['intensity']} intensity "
f"(\"{quantifier['matched_token']}\") with an under-"
"specified universe. Narrow the question (e.g. add a "
"year, league, country, or category) or run with "
"--allow-broad for exploratory enumeration."
),
# Audit binding: reject path now carries its own
# 3-stage run_dag (question → preflight → final_label)
# so audit replay can read the rejection from
# run_dag_blob the same way it reads any other row.
"run_dag_root": _reject_run_dag["root"],
"run_dag_blob": json.dumps(_reject_run_dag, separators=(",", ":")),
"answer_text": _reject_answer_text,
"sources": [],
"n_quotes": 0,
"n_verified": 0,
@ -1756,17 +1848,7 @@ def query(
else "quote",
"unverified_quotes": [],
"partially_verified_quotes": [],
"violations": [{
"kind": "BROAD_QUANTIFIER_REJECTED",
"intensity": quantifier["intensity"],
"matched_token": quantifier["matched_token"],
"scope_bound_hint": quantifier["scope_bound_hint"],
"reason": (
"preflight rejection — broad-quantifier query with "
"unbounded scope. Operator opted in via "
"quantifier_reject_broad policy."
),
}],
"violations": _reject_violations,
"format_collapsed": None,
"raw_answer": None,
"quantifier_intensity": quantifier["intensity"],
@ -2663,36 +2745,54 @@ def query(
),
)
plan_hash = retrieval_plan_hash(plan)
# Ticket #000009 — preflight node binding. Combines #000008
# quantifier classifier output + #000010 metacognition
# QuestionState + the behavioral policy decisions (cap
# actually applied, reminder actually injected, reject path
# taken) into a single hash that contributes to run_dag_root.
# Audit replay can now distinguish two cache rows that have
# the same question + same model output but different
# preflight policy state.
# Ticket #000009 — preflight node binding (nested CTI clauses
# per ticket §8 / 2026-05-04 feedback). Single DAG stage with
# five nested clauses (classifier, answer_contract,
# prompt_contract, evidence_contract, policy_refs) + the
# metacognition QuestionState.
from aborist.qa.dag import preflight_node_hash
# verifier_policy_hash + model_profile_hash already imported
# at module top; reusing the existing names. Local re-imports
# would shadow earlier free-variable uses.
ghash_for_dag = verifier_policy_hash(policy)
claim_cap_actually_applied = (
claim_cap_lookup
if (quantifier_apply_caps
and quantifier_caps_mode_gated
and claim_cap_lookup is not None)
else None
)
# Reminder injection actually fires when guard is on AND
# mode-gated AND quantifier is broad AND policy enables it.
# Mirrors the gate in the runner.ask() / query() reminder
# block above.
reminder_eligible = (
quantifier_guard_on
and quantifier_mode_gated
and quantifier.get("is_broad", False)
)
reminder_enabled = bool(policy.get("quantifier_reminder_enabled", False))
reminder_injected = reminder_eligible and reminder_enabled
reminder_template_id = None
if reminder_injected:
reminder_template_id = (
"broad-quantifier-bounded-v1"
if quantifier.get("scope_bound_hint") == "bounded"
else "broad-quantifier-unbounded-v1"
)
preflight_hash = preflight_node_hash(
question_state=question_state.to_dict(),
quantifier=quantifier,
policy_state={
answer_contract={
"guard_enabled": quantifier_guard_on,
"guard_apply_caps": quantifier_apply_caps,
"guard_apply_caps_mode_gated": quantifier_caps_mode_gated,
"mode_gated": quantifier_mode_gated,
"apply_caps_active": quantifier_apply_caps,
"apply_caps_mode_gated": quantifier_caps_mode_gated,
"claim_cap_resolved": claim_cap_lookup,
"claim_cap_actually_applied": (
quantifier_apply_caps
and quantifier_caps_mode_gated
and claim_cap_lookup is not None
),
"reminder_enabled": bool(
policy.get("quantifier_reminder_enabled", False)
),
"reminder_eligible": (
quantifier_guard_on
and quantifier_mode_gated
and quantifier.get("is_broad", False)
),
"claim_cap_applied": claim_cap_actually_applied,
"manual_quotes_allowed": False,
"evidence_pointer_required": is_lattice_mode,
"allow_unbounded_enumeration": False,
"reject_broad_active": bool(
policy.get("quantifier_reject_broad", False)
),
@ -2700,9 +2800,27 @@ def query(
policy.get("metacognition_enabled", True)
),
"block_on_contradiction": bool(
policy.get("metacognition_block_on_contradiction", False)
policy.get(
"metacognition_block_on_contradiction", False
)
),
},
prompt_contract={
"reminder_enabled": reminder_enabled,
"reminder_injected": reminder_injected,
"reminder_template_id": reminder_template_id,
},
evidence_contract={
"max_evidence_ids_exposed": int(policy.get(
"claim_lattice_max_pointers_per_claim", 2
)),
"one_claim_per_line": is_lattice_mode,
},
policy_refs={
"governance_policy_hash": ghash_for_dag,
"model_profile_hash": mhash,
"answer_mode": answer_mode,
},
)
run_dag = build_run_dag(
question_hash=qhash,

67
aborist/qa/runner.py
View file

@ -852,29 +852,46 @@ def ask(
}
for i, cs in enumerate(verdict.get("claim_statuses") or [])
]
# Ticket #000009 — preflight node binding (mirror of query()).
# Ticket #000009 — preflight node binding (mirror of query();
# nested CTI clauses per ticket §8 / 2026-05-04 feedback).
from aborist.qa.dag import preflight_node_hash
# verifier_policy_hash + model_profile_hash imported at module
# top; do NOT re-import locally (free-variable shadowing).
ghash_for_dag = verifier_policy_hash(policy)
claim_cap_actually_applied = (
claim_cap_lookup
if (quantifier_apply_caps
and quantifier_caps_mode_gated
and claim_cap_lookup is not None)
else None
)
reminder_eligible = (
quantifier_guard_on
and quantifier_mode_gated
and quantifier.get("is_broad", False)
)
reminder_enabled = bool(policy.get("quantifier_reminder_enabled", False))
reminder_injected = reminder_eligible and reminder_enabled
reminder_template_id = None
if reminder_injected:
reminder_template_id = (
"broad-quantifier-bounded-v1"
if quantifier.get("scope_bound_hint") == "bounded"
else "broad-quantifier-unbounded-v1"
)
preflight_hash = preflight_node_hash(
question_state=question_state.to_dict(),
quantifier=quantifier,
policy_state={
answer_contract={
"guard_enabled": quantifier_guard_on,
"guard_apply_caps": quantifier_apply_caps,
"guard_apply_caps_mode_gated": quantifier_caps_mode_gated,
"mode_gated": quantifier_mode_gated,
"apply_caps_active": quantifier_apply_caps,
"apply_caps_mode_gated": quantifier_caps_mode_gated,
"claim_cap_resolved": claim_cap_lookup,
"claim_cap_actually_applied": (
quantifier_apply_caps
and quantifier_caps_mode_gated
and claim_cap_lookup is not None
),
"reminder_enabled": bool(
policy.get("quantifier_reminder_enabled", False)
),
"reminder_eligible": (
quantifier_guard_on
and quantifier_mode_gated
and quantifier.get("is_broad", False)
),
"claim_cap_applied": claim_cap_actually_applied,
"manual_quotes_allowed": False,
"evidence_pointer_required": is_lattice_mode,
"allow_unbounded_enumeration": False,
"reject_broad_active": bool(
policy.get("quantifier_reject_broad", False)
),
@ -885,6 +902,22 @@ def ask(
policy.get("metacognition_block_on_contradiction", False)
),
},
prompt_contract={
"reminder_enabled": reminder_enabled,
"reminder_injected": reminder_injected,
"reminder_template_id": reminder_template_id,
},
evidence_contract={
"max_evidence_ids_exposed": int(policy.get(
"claim_lattice_max_pointers_per_claim", 2
)),
"one_claim_per_line": is_lattice_mode,
},
policy_refs={
"governance_policy_hash": ghash_for_dag,
"model_profile_hash": mhash,
"answer_mode": answer_mode,
},
)
run_dag = build_run_dag(
question_hash=qhash,

2
docs/TICKETS.md
View file

@ -58,7 +58,7 @@ Newest first. Update on every open/close.
| ID | Title | Status | Opened | Directive |
|----------|------------------------------------------------|-----------------------|------------|-----------|
| #000010 | Meta-Cognition Preflight Guard (M0 / MCTL) | closed · landed 2026-05-03 (Phases 1–4); DAG binding shipped via #000009 | 2026-05-03 | D1, D3 |
| #000009 | Preflight run-DAG node binding (#000008+#000010) | closed · landed 2026-05-03 (zero-shot) | 2026-05-03 | D3, D4 |
| #000009 | Preflight run-DAG node binding (#000008+#000010) | closed · re-landed 2026-05-04 (§8 corrections: reject-path DAG, nested CTI clauses) | 2026-05-03 | D3, D4 |
| #000008 | Broad-quantifier preflight guard | closed · landed in `4f2b5a6`; Phase 5 DAG binding split into #000009 | 2026-05-02 | — |
| #000007 | Query-layer hyphen folding | closed · 2026-05-02 | 2026-05-02 | — |
| #000006 | Bench-emergent findings (rolling research log) | open · rolling | 2026-05-02 | — |

183
docs/tickets/ticket-000009-quantifier-preflight-dag-binding.md
View file

@ -1,6 +1,10 @@
# Ticket #000009 — Preflight run-DAG node binding
**Status:** closed · landed 2026-05-03 (zero-shot)
**Status:** closed · re-landed 2026-05-04 with §8 corrections
(reject-path DAG, nested CTI clauses, node_version, reminder
template id, policy_refs). Original landing 2026-05-03 in
`c36e85c`; §8 corrections per architectural feedback at
`~/Downloads/RESPONSE-ticket-000009-quantifier-preflight-dag-binding.txt`.
**Opened:** 2026-05-03
**Scope (expanded 2026-05-03):** Bind BOTH ticket #000008's broad-
quantifier preflight contract AND ticket #000010's meta-cognition
@ -280,3 +284,180 @@ preflight contract").
bench analysis needs it.
- **`SOFT_PREFLIGHT_HINT` (model-assisted preflight)**:
reserved per #000010 §18 / source doc. Hard rule preserved.
## 8. Architectural review + corrections (2026-05-04)
Source: `~/Downloads/RESPONSE-ticket-000009-quantifier-preflight-dag-binding.txt`
(2026-05-04, Asia/Kuala_Lumpur). Verdict: "Go. Land #000009."
Recommendations against the as-shipped (`c36e85c`) implementation:
### 8.1 Gaps surfaced
| # | Gap | Severity |
|---|--------------------------------------------------------|----------|
| A | **Reject-broad path emits NO run_dag_blob.** Preflight rejection currently early-returns from `query()` before `build_run_dag()` runs. Audit replay can't see that a rejection happened. | critical |
| B | Payload is flat (`question_state` / `quantifier` / `policy_state`). Feedback §3 recommends nested CTI clauses (`classifier`, `answer_contract`, `prompt_contract`, `evidence_contract`, `policy_refs`) for diff legibility. | structural |
| C | No `node_version` field. Legacy runs without the stage can't be unambiguously labeled `unavailable_legacy_run`. | clarity |
| D | Payload bundles raw policy booleans instead of referencing `governance_policy_hash` / `model_profile_hash`. Bloats the hash and double-commits already-hashed state. | clarity |
| E | No `reminder_template_id` field for the prompt contract. | minor |
| F | Stage named `preflight` (because metacog also lives there) vs feedback's `quantifier_preflight`. Defensible either way; we'll document the choice. | naming |
| G | `docs/cti-architecture.md` not updated. | docs |
### 8.2 Landing plan
1. **A — reject-path DAG.** Build a minimal `question → preflight →
final_label` DAG for the reject-broad early-return path. Reject
rows currently have no `run_dag_root` / `run_dag_blob`; this
gives them one with the preflight node and a final-label that
carries `BROAD_QUANTIFIER_REJECTED`. **Audit replay parity with
normal runs.**
2. **B — restructure payload.** Move from flat 3-key to nested
5-clause:
```
classifier — quantifier classifier output
answer_contract — guard / cap / reject / allow_broad state
prompt_contract — reminder enabled / injected / template_id
evidence_contract — exposure budget, line-discipline
policy_refs — governance_policy_hash, model_profile_hash,
answer_mode (reference, not raw policy)
```
Plus a top-level `node_version: "preflight-node-v1"` for legacy
disambiguation (C).
3. **D — reference hashes only.** Replace bundled policy booleans
with `policy_refs.governance_policy_hash` +
`policy_refs.model_profile_hash`. Keep the *behavioral*
decisions (claim_cap_applied, reminder_injected, etc.) in
`answer_contract` / `prompt_contract` since those are
per-run-state-of-the-world, not policy.
4. **E — reminder_template_id.** When reminder fires, record which
template (`broad-quantifier-bounded-v1` /
`broad-quantifier-unbounded-v1`).
5. **F — keep `preflight` name** because the node carries BOTH
#000008 quantifier classifier AND #000010 metacognition
QuestionState. `quantifier_preflight` would be misleading. The
`node_version` field disambiguates schema.
6. **G — update docs/cti-architecture.md.**
### 8.3 What stays as-is
The feedback's recommendations match what's already shipped on:
- Single DAG stage (not three separate nodes) — ✓
- Stage placement between `question` and `retrieval` — ✓
- Backward-compat (None preflight_hash → 7/9-stage shape) — ✓
- `governance_policy_hash` invalidates cache rows on policy
flip — ✓ (already in #000008 / #000010)
- No verifier / retrieval / schema changes — ✓
- Symmetric wiring in `query()` and `runner.ask()` — ✓
### 8.4 Out of scope (per feedback §12)
> Do not use #000009 to sneak in any of this:
>
> - new verifier rules
> - new quantifier classifier categories
> - retrieval changes
> - new schema/cache columns
> - new audit_mode token
> - NLI / semantic entailment
> - prompt-template behavior changes
> - default policy flips
Confirmed. Corrections stay strictly in the audit-binding lane.
### 8.5 Corrections landed (commit pending after this update)
**A — reject-path DAG** (the critical gap):
`aborist/qa/dag.py:build_reject_run_dag()` ships. Builds a 3-stage
`question → preflight → final_label` DAG for the reject-broad
early-return path. `query()` now wires it in and returns
`run_dag_root` + `run_dag_blob` on the rejection result dict.
Live-verified on `make query Q="winners of all major sports?"
REJECT_BROAD=1 BURN=1`:
```
"status": "broad_quantifier_rejected",
"run_dag_root": "86a03380…",
"run_dag_blob": {"nodes": [
{"stage": "question", "hash": "…"},
{"stage": "preflight", "hash": "…"},
{"stage": "final_label", "hash": "…"}
]}
```
3-stage shape always means reject path; audit replay can read the
stage list and tell instantly without parsing the payload.
**B — nested CTI clauses + C — node_version + D — reference
hashes + E — reminder_template_id**:
`preflight_node_hash()` payload restructured from flat 3-key
(`question_state` / `quantifier` / `policy_state`) to nested
5-clause:
```
classifier — quantifier classifier output
answer_contract — guard / cap / reject / metacog state (per-run)
prompt_contract — reminder enabled / injected / template_id
evidence_contract — exposure budget, line discipline
policy_refs — governance_policy_hash, model_profile_hash,
answer_mode (reference, not raw policy)
```
Plus top-level `stage`, `node_version: "preflight-node-v1"`, and
`question_state` (metacog, kept its own clause for now since
QuestionState carries its own internal `preflight_policy_hash`).
`reminder_template_id` is `"broad-quantifier-bounded-v1"` or
`"broad-quantifier-unbounded-v1"` depending on
`scope_bound_hint`, populated only when reminder actually fires.
`policy_refs.governance_policy_hash` is the
`verifier_policy_hash(policy)` already used for cache identity —
reusing the existing hash rather than re-canonicalizing all the
policy fields. `model_profile_hash` follows the same pattern.
**F — stage name kept as `preflight`** (not `quantifier_preflight`)
since the node carries both #000008 quantifier classifier AND
#000010 metacognition QuestionState. `node_version` field
disambiguates schema for audit tools.
**G — `docs/cti-architecture.md` update**: deferred to a small
follow-up commit. The architecture description in the ticket §8
serves as the canonical reference until then.
Hash compatibility note: rows written between commit `c36e85c`
(initial #000009 landing) and this commit have hash payloads
matching the OLD flat 3-key shape. The persisted
`run_dag_blob` captures the actual payload that was hashed, so
those rows still verify via `verify_run_dag()`. New rows after
this commit use the nested 5-clause shape. Operators reading
the `run_dag_blob` directly see the structure either way.
### 8.6 New tests
`tests/test_dag.py`:
- `test_preflight_node_hash_changes_with_answer_contract` —
apply_caps flip in answer_contract → distinct node hash.
- `test_preflight_node_hash_changes_with_prompt_contract` —
reminder injection flip → distinct hash.
- `test_preflight_node_hash_changes_with_policy_refs` —
governance_policy_hash flip → distinct hash.
- `test_preflight_node_hash_includes_node_version` — pins
`PREFLIGHT_NODE_VERSION = "preflight-node-v1"`.
- `test_reject_run_dag_three_stage_shape` — reject DAG always
3 stages (question → preflight → final_label).
- `test_reject_run_dag_root_changes_with_preflight_hash` —
audit-replay payoff for reject path.
- `test_reject_run_dag_round_trips_through_verify` — 3-stage
shape verifies the same way as 7/9/8/10-stage shapes.
993 tests passing (6 net new); 36 skipped.
### 8.7 Status — re-closed
Closed · landed via this commit on top of `c36e85c`. Audit-binding
gap from feedback §6.2 (reject path) closed; nested-clause payload
landed per feedback §3; node_version + reminder_template_id +
policy_refs landed per feedback §4 + §9.

172
tests/test_dag.py
View file

@ -179,56 +179,105 @@ def test_verify_dag_accepts_json_string():
# ---------------------------------------------------------------- Ticket #000009: preflight stage
def test_preflight_node_hash_is_deterministic():
"""Same inputs → same hex string, byte-for-byte."""
"""Same nested-clause inputs → same hex string, byte-for-byte."""
qs = {"logical_statuses": ["well_formed"], "preflight_result": "PREFLIGHT_OK"}
quant = {"intensity": "SINGULAR", "is_broad": False}
pol = {"guard_enabled": True, "guard_apply_caps": False}
a = preflight_node_hash(question_state=qs, quantifier=quant, policy_state=pol)
b = preflight_node_hash(question_state=qs, quantifier=quant, policy_state=pol)
answer_contract = {"guard_enabled": True, "apply_caps_active": False}
a = preflight_node_hash(
question_state=qs, quantifier=quant,
answer_contract=answer_contract,
)
b = preflight_node_hash(
question_state=qs, quantifier=quant,
answer_contract=answer_contract,
)
assert a == b
assert len(a) == 64 # SHA-256 hex
def test_preflight_node_hash_changes_with_question_state():
base_quant = {"intensity": "SINGULAR", "is_broad": False}
pol = {"guard_enabled": True}
answer_contract = {"guard_enabled": True}
h_a = preflight_node_hash(
question_state={"preflight_result": "PREFLIGHT_OK"},
quantifier=base_quant, policy_state=pol,
quantifier=base_quant, answer_contract=answer_contract,
)
h_b = preflight_node_hash(
question_state={"preflight_result": "PREFLIGHT_PARTIAL"},
quantifier=base_quant, policy_state=pol,
quantifier=base_quant, answer_contract=answer_contract,
)
assert h_a != h_b
def test_preflight_node_hash_changes_with_policy_state():
"""Apply-caps flip MUST bump the preflight node hash so audit
replay can distinguish guard-on vs guard-off rows that
otherwise share the same classifier output."""
def test_preflight_node_hash_changes_with_answer_contract():
"""Apply-caps flip in answer_contract MUST bump the node hash
so audit replay can distinguish guard-on vs guard-off rows
that otherwise share the same classifier output."""
qs = {"logical_statuses": ["broad_quantifier_unbounded"]}
quant = {"intensity": "ALL", "is_broad": True}
h_off = preflight_node_hash(
question_state=qs, quantifier=quant,
policy_state={"guard_apply_caps": False},
answer_contract={"apply_caps_active": False},
)
h_on = preflight_node_hash(
question_state=qs, quantifier=quant,
policy_state={"guard_apply_caps": True},
answer_contract={"apply_caps_active": True},
)
assert h_off != h_on
def test_preflight_node_hash_changes_with_prompt_contract():
"""Reminder-injection flip MUST bump the node hash."""
qs = {"logical_statuses": ["broad_quantifier_unbounded"]}
quant = {"intensity": "ALL", "is_broad": True}
h_off = preflight_node_hash(
question_state=qs, quantifier=quant,
prompt_contract={"reminder_injected": False},
)
h_on = preflight_node_hash(
question_state=qs, quantifier=quant,
prompt_contract={
"reminder_injected": True,
"reminder_template_id": "broad-quantifier-unbounded-v1",
},
)
assert h_off != h_on
def test_preflight_node_hash_changes_with_policy_refs():
"""policy_refs.governance_policy_hash flip → distinct node
hash. Required for cache-key/run-DAG-root joint invalidation."""
qs = {"logical_statuses": ["well_formed"]}
h_a = preflight_node_hash(
question_state=qs,
policy_refs={"governance_policy_hash": "a" * 64},
)
h_b = preflight_node_hash(
question_state=qs,
policy_refs={"governance_policy_hash": "b" * 64},
)
assert h_a != h_b
def test_preflight_node_hash_includes_node_version():
"""node_version field is part of the hashed payload so a future
schema bump (preflight-node-v2 etc.) invalidates legacy nodes."""
from aborist.qa.dag import (
PREFLIGHT_NODE_VERSION,
build_preflight_node_payload,
)
payload = build_preflight_node_payload(
question_state={"x": 1},
)
assert payload["node_version"] == PREFLIGHT_NODE_VERSION
assert PREFLIGHT_NODE_VERSION == "preflight-node-v1"
def test_preflight_node_hash_handles_all_none():
"""Defensive — all three components may be None during gradual
rollout. Hash stays stable."""
a = preflight_node_hash(
question_state=None, quantifier=None, policy_state=None,
)
b = preflight_node_hash(
question_state=None, quantifier=None, policy_state=None,
)
"""Defensive — all clauses may be None during gradual rollout.
Hash stays stable."""
a = preflight_node_hash()
b = preflight_node_hash()
assert a == b
assert len(a) == 64
@ -252,7 +301,7 @@ def test_dag_with_preflight_inserts_eight_stage_shape():
pre_hash = preflight_node_hash(
question_state={"preflight_result": "PREFLIGHT_OK"},
quantifier={"intensity": "SINGULAR", "is_broad": False},
policy_state={"guard_enabled": True},
answer_contract={"guard_enabled": True},
)
out = build_run_dag(**_kw(preflight_hash=pre_hash))
stages = [n["stage"] for n in out["nodes"]]
@ -268,7 +317,7 @@ def test_dag_with_preflight_lattice_mode_ten_stages():
pre_hash = preflight_node_hash(
question_state={"preflight_result": "PREFLIGHT_OK"},
quantifier={"intensity": "ALL", "is_broad": True},
policy_state={"guard_enabled": True},
answer_contract={"guard_enabled": True},
)
out = build_run_dag(**_kw(
preflight_hash=pre_hash,
@ -291,12 +340,12 @@ def test_dag_root_changes_when_preflight_hash_changes():
pre_a = preflight_node_hash(
question_state={"preflight_result": "PREFLIGHT_OK"},
quantifier={"intensity": "SINGULAR"},
policy_state={"guard_apply_caps": False},
answer_contract={"apply_caps_active": False},
)
pre_b = preflight_node_hash(
question_state={"preflight_result": "PREFLIGHT_OK"},
quantifier={"intensity": "SINGULAR"},
policy_state={"guard_apply_caps": True},
answer_contract={"apply_caps_active": True},
)
a = build_run_dag(**_kw(preflight_hash=pre_a))
b = build_run_dag(**_kw(preflight_hash=pre_b))
@ -310,8 +359,79 @@ def test_dag_with_preflight_round_trips_through_verify():
pre_hash = preflight_node_hash(
question_state={"preflight_result": "PREFLIGHT_PARTIAL"},
quantifier={"intensity": "ALL", "is_broad": True},
policy_state={"guard_apply_caps": True, "claim_cap_resolved": 8},
answer_contract={"apply_caps_active": True, "claim_cap_resolved": 8},
)
out = build_run_dag(**_kw(preflight_hash=pre_hash))
blob = json.dumps(out, separators=(",", ":"))
assert verify_run_dag(blob) is True
# ---------------------------------------------------------------- reject-broad DAG (§8.2 A)
def test_reject_run_dag_three_stage_shape():
"""Reject-broad early-return path produces a 3-stage DAG:
question → preflight → final_label. Audit replay can read the
stage list and tell instantly that this row is a preflight
rejection (3 stages = reject path)."""
from aborist.qa.dag import build_reject_run_dag, preflight_node_hash
pre = preflight_node_hash(
question_state={"preflight_result": "PREFLIGHT_BLOCKED"},
quantifier={"intensity": "ALL", "is_broad": True,
"scope_bound_hint": "unbounded"},
answer_contract={"reject_broad_active": True},
)
out = build_reject_run_dag(
question_hash="a" * 64,
preflight_hash=pre,
rejection_reason="preflight rejection — broad-unbounded.",
answer_text="BROAD-QUANTIFIER PREFLIGHT REJECTED",
violations=[{"kind": "BROAD_QUANTIFIER_REJECTED"}],
)
stages = [n["stage"] for n in out["nodes"]]
assert stages == ["question", "preflight", "final_label"]
assert len(stages) == 3
def test_reject_run_dag_root_changes_with_preflight_hash():
"""Two reject runs that differ only in the preflight payload
(e.g. different policy state at rejection time) must produce
different run_dag_root values."""
from aborist.qa.dag import build_reject_run_dag, preflight_node_hash
pre_a = preflight_node_hash(
question_state={"preflight_result": "PREFLIGHT_BLOCKED"},
quantifier={"intensity": "ALL"},
policy_refs={"governance_policy_hash": "a" * 64},
)
pre_b = preflight_node_hash(
question_state={"preflight_result": "PREFLIGHT_BLOCKED"},
quantifier={"intensity": "ALL"},
policy_refs={"governance_policy_hash": "b" * 64},
)
a = build_reject_run_dag(
question_hash="d" * 64, preflight_hash=pre_a,
rejection_reason="r", answer_text="x",
)
b = build_reject_run_dag(
question_hash="d" * 64, preflight_hash=pre_b,
rejection_reason="r", answer_text="x",
)
assert a["root"] != b["root"]
def test_reject_run_dag_round_trips_through_verify():
"""3-stage reject DAG must verify the same way as the standard
7/9/8/10-stage shapes."""
import json
from aborist.qa.dag import build_reject_run_dag, preflight_node_hash
pre = preflight_node_hash(
question_state={"preflight_result": "PREFLIGHT_BLOCKED"},
quantifier={"intensity": "ALL"},
)
out = build_reject_run_dag(
question_hash="a" * 64,
preflight_hash=pre,
rejection_reason="preflight rejection",
answer_text="REJECTED",
)
blob = json.dumps(out, separators=(",", ":"))
assert verify_run_dag(blob) is True