qa(#000009): §8 corrections — reject-path DAG + nested CTI clauses
Architectural feedback at ~/Downloads/RESPONSE-ticket-000009-... .txt (2026-05-04) flagged five gaps in thec36e85clanding. Most critical: reject-broad early-return path emitted no run_dag_blob, so audit replay couldn't see that a rejection happened (let alone under what policy state). A — reject-path DAG (the critical gap): aborist/qa/dag.py: build_reject_run_dag() — 3-stage minimal DAG question → preflight → final_label. final_label payload carries rejection_reason + answer_text_hash so two rejections under different policy state produce different roots. query.py reject path now wires it: returns run_dag_root + run_dag_blob on the rejection result dict. Live-verified end- to-end on `make query Q="winners of all major sports?" REJECT_BROAD=1 BURN=1`. Audit replay rule: 3 stages always means reject path. Operators can read the stage list and tell instantly without parsing the payload. B — nested CTI clauses: preflight_node_hash() payload restructured from flat 3-key to nested 5-clause: classifier — quantifier classifier output (#000008) answer_contract — guard / cap / reject / metacog state (per-run) prompt_contract — reminder enabled / injected / template_id evidence_contract — exposure budget, line discipline policy_refs — governance_policy_hash, model_profile_hash, answer_mode (reference, not raw policy) Plus question_state (metacog) as its own clause and top-level stage + node_version. Single DAG stage; nested clauses inside for diff legibility (feedback §3). C — node_version field: PREFLIGHT_NODE_VERSION = "preflight-node-v1" pinned in the payload so legacy runs without the node can be unambiguously labeled `unavailable_legacy_run` by audit tools (feedback §9). D — reference hashes only: policy_refs uses governance_policy_hash + model_profile_hash rather than bundling raw policy booleans. Avoids double-committing already-hashed state (feedback §4). E — reminder_template_id: prompt_contract.reminder_template_id = "broad-quantifier-bounded-v1" or "broad-quantifier-unbounded-v1" depending on scope_bound_hint, populated only when reminder actually fires. F — stage name kept as `preflight` (not `quantifier_preflight`): Node carries both #000008 quantifier AND #000010 metacognition payloads. node_version disambiguates schema for audit tools. G — docs/cti-architecture.md update deferred to a small follow-up. Bug fixes: - free-variable shadowing on verifier_policy_hash / model_profile_hash / question_hash — local re-imports inside the reject branch shadowed module-top imports used elsewhere in query() / runner(); now use the module-top names. - reject path question_hash signature: takes `mode=` not `dedup_mode=` — fixed in the reject DAG builder caller. Hash compatibility: Rows written betweenc36e85cand this commit have hash payloads matching the OLD flat 3-key shape. The persisted run_dag_blob captures the actual payload that was hashed, so those rows still verify via verify_run_dag(). New rows use the nested 5-clause shape. 7 new tests in tests/test_dag.py: - hash sensitivity to answer_contract / prompt_contract / policy_refs flips (audit-replay payoff demonstrations) - PREFLIGHT_NODE_VERSION pinning - reject DAG: 3-stage shape, root changes with preflight hash, round-trips through verify_run_dag 993 tests passing (6 net new); 36 skipped. Live verification: make query Q="winners of all major sports?" REJECT_BROAD=1 BURN=1 → status=broad_quantifier_rejected, run_dag_root populated, blob carries 3-stage shape. make query Q="winners of all major sports?" BURN=1 → 10-stage shape preserved (question → preflight → retrieval → ... → final_label). Ticket #000009 status: closed · re-landed 2026-05-04 with §8 corrections.
This commit is contained in:
parent
c36e85c86c
commit
111dda6160
6 changed files with 683 additions and 118 deletions
|
|
@ -99,44 +99,103 @@ def localize_failure(
|
||||||
return "answer"
|
return "answer"
|
||||||
|
|
||||||
|
|
||||||
|
PREFLIGHT_NODE_VERSION = "preflight-node-v1"
|
||||||
|
|
||||||
|
|
||||||
|
def build_preflight_node_payload(
|
||||||
|
*,
|
||||||
|
question_state: dict | None = None,
|
||||||
|
quantifier: dict | None = None,
|
||||||
|
answer_contract: dict | None = None,
|
||||||
|
prompt_contract: dict | None = None,
|
||||||
|
evidence_contract: dict | None = None,
|
||||||
|
policy_refs: dict | None = None,
|
||||||
|
) -> dict:
|
||||||
|
"""Build the canonical nested-clause payload for the preflight
|
||||||
|
DAG stage. Returns a JSON-ready dict; pair with
|
||||||
|
:func:`preflight_node_hash` to get the SHA-256 hex.
|
||||||
|
|
||||||
|
Five-clause structure per ticket #000009 §8.2 / feedback §3:
|
||||||
|
|
||||||
|
- ``classifier`` — quantifier classifier output (#000008):
|
||||||
|
intensity, matched_token, explicit_count, scope_bound_hint,
|
||||||
|
is_broad, classifier_version, operational_shape.
|
||||||
|
- ``answer_contract`` — guard / cap / reject decisions taken
|
||||||
|
on this run.
|
||||||
|
- ``prompt_contract`` — reminder enabled / injected /
|
||||||
|
template_id (#000008 §10.5).
|
||||||
|
- ``evidence_contract`` — exposure budget, one-claim-per-line
|
||||||
|
discipline (#000010 §10.4).
|
||||||
|
- ``policy_refs`` — governance_policy_hash + model_profile_hash
|
||||||
|
+ answer_mode. Reference-by-hash rather than raw policy
|
||||||
|
bundles (feedback §4: avoid double-committing
|
||||||
|
already-hashed state).
|
||||||
|
|
||||||
|
Plus the metacog ``question_state`` from #000010 — that's its
|
||||||
|
own clause for now (logical_statuses, false_premise_hints,
|
||||||
|
contradiction_pairs). It's hashed separately by
|
||||||
|
`metacognition.preflight_policy_hash` already.
|
||||||
|
|
||||||
|
Any clause may be None / empty — the resulting payload is
|
||||||
|
still stable. Includes ``node_version`` so legacy runs without
|
||||||
|
the node can be unambiguously labeled `unavailable_legacy_run`
|
||||||
|
by audit tools.
|
||||||
|
"""
|
||||||
|
return {
|
||||||
|
"stage": "preflight",
|
||||||
|
"node_version": PREFLIGHT_NODE_VERSION,
|
||||||
|
"classifier": dict(quantifier) if quantifier else {},
|
||||||
|
"answer_contract": dict(answer_contract) if answer_contract else {},
|
||||||
|
"prompt_contract": dict(prompt_contract) if prompt_contract else {},
|
||||||
|
"evidence_contract": dict(evidence_contract) if evidence_contract else {},
|
||||||
|
"policy_refs": dict(policy_refs) if policy_refs else {},
|
||||||
|
# Metacognition QuestionState carries
|
||||||
|
# ``preflight_policy_hash`` internally so flipping a metacog
|
||||||
|
# detector invalidates this clause via that field. Stored
|
||||||
|
# nested so audit-replay can read all metacog signal in one
|
||||||
|
# place without descending into the quantifier classifier.
|
||||||
|
"question_state": dict(question_state) if question_state else {},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def preflight_node_hash(
|
def preflight_node_hash(
|
||||||
*,
|
*,
|
||||||
question_state: dict | None = None,
|
question_state: dict | None = None,
|
||||||
quantifier: dict | None = None,
|
quantifier: dict | None = None,
|
||||||
policy_state: dict | None = None,
|
answer_contract: dict | None = None,
|
||||||
|
prompt_contract: dict | None = None,
|
||||||
|
evidence_contract: dict | None = None,
|
||||||
|
policy_refs: dict | None = None,
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Hash the preflight decision into a stable hex string.
|
"""Hash the preflight decision into a stable SHA-256 hex string.
|
||||||
|
|
||||||
Bundles three sources of preflight provenance into one
|
Returns the hash of the nested-clause payload built by
|
||||||
canonical hash that contributes to ``run_dag_root`` via the
|
:func:`build_preflight_node_payload`. See that function for the
|
||||||
new ``preflight`` stage in :func:`build_run_dag`:
|
five-clause structure.
|
||||||
|
|
||||||
- ``question_state`` — `QuestionState.to_dict()` from
|
Audit-replay payoff: two cache rows that share the same
|
||||||
``aborist.qa.metacognition`` (#000010): logical_statuses,
|
question + same model output + same verifier verdict but
|
||||||
question_shape, false_premise_hints, contradiction_pairs,
|
different preflight policy state produce different hashes
|
||||||
temporal_sensitivity, etc. Carries
|
here, which propagate to ``run_dag_root`` via
|
||||||
``preflight_policy_hash`` so policy flips invalidate the
|
:func:`build_run_dag`.
|
||||||
stage hash automatically.
|
|
||||||
- ``quantifier`` — classifier output from
|
|
||||||
``aborist.qa.quantifier.classify_question_quantifier``
|
|
||||||
(#000008): intensity, matched_token, explicit_count,
|
|
||||||
scope_bound_hint, classifier_version.
|
|
||||||
- ``policy_state`` — the *behavioral* decisions taken on this
|
|
||||||
run that aren't already in the above (e.g. whether the cap
|
|
||||||
was actually applied vs just looked up, whether the reminder
|
|
||||||
was injected, whether reject-broad path was taken). These
|
|
||||||
are what differentiate two cache rows with the same
|
|
||||||
classifier output but different downstream effects.
|
|
||||||
|
|
||||||
Any of the three may be None (legacy / opt-out); the hash is
|
Backward compatibility note: Pre-2026-05-04 (`c36e85c`) callers
|
||||||
still stable. Returns SHA-256 hex.
|
used a flat 3-key payload (`question_state` / `quantifier` /
|
||||||
|
`policy_state`). Hashes computed with that callsite will NOT
|
||||||
|
match this restructured callsite — `run_dag_root` values for
|
||||||
|
rows written between `c36e85c` and the current commit are
|
||||||
|
treated as a discrete generation; they're still verifiable by
|
||||||
|
re-reading `run_dag_blob` (the persisted blob captures the
|
||||||
|
payload that was actually hashed).
|
||||||
"""
|
"""
|
||||||
payload = {
|
payload = build_preflight_node_payload(
|
||||||
"stage": "preflight",
|
question_state=question_state,
|
||||||
"question_state": question_state or {},
|
quantifier=quantifier,
|
||||||
"quantifier": quantifier or {},
|
answer_contract=answer_contract,
|
||||||
"policy_state": policy_state or {},
|
prompt_contract=prompt_contract,
|
||||||
}
|
evidence_contract=evidence_contract,
|
||||||
|
policy_refs=policy_refs,
|
||||||
|
)
|
||||||
return _sha256_hex(_canonical_json(payload))
|
return _sha256_hex(_canonical_json(payload))
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -305,6 +364,60 @@ def build_run_dag(
|
||||||
return {"root": root_hex, "nodes": nodes}
|
return {"root": root_hex, "nodes": nodes}
|
||||||
|
|
||||||
|
|
||||||
|
def build_reject_run_dag(
|
||||||
|
*,
|
||||||
|
question_hash: str,
|
||||||
|
preflight_hash: str,
|
||||||
|
rejection_reason: str,
|
||||||
|
answer_text: str,
|
||||||
|
audit_mode: str = "UNGROUNDED",
|
||||||
|
verifier_method: str = "claim_lattice_pointer",
|
||||||
|
violations: list[dict] | None = None,
|
||||||
|
) -> dict:
|
||||||
|
"""3-stage reject-broad run-DAG: ``question → preflight →
|
||||||
|
final_label``.
|
||||||
|
|
||||||
|
Ticket #000009 §8.2 / 2026-05-04 feedback §6.2: preflight
|
||||||
|
rejection currently early-returns from ``query()`` before the
|
||||||
|
standard ``build_run_dag()`` runs, so reject rows have no
|
||||||
|
auditable Merkle commitment. This builder fills that gap with
|
||||||
|
a minimal DAG shape that captures the rejection without
|
||||||
|
pretending retrieval / prompt / raw_model_output happened.
|
||||||
|
|
||||||
|
The returned shape is INTENTIONALLY shorter than the standard
|
||||||
|
7/9/8/10-stage shapes — `audit replay can read the stage
|
||||||
|
list` and tell instantly that this row is a preflight
|
||||||
|
rejection: 3 stages always means reject path.
|
||||||
|
|
||||||
|
`final_label` carries the rejection_reason + answer_text hash
|
||||||
|
so two rejections that differ only in their (rendered)
|
||||||
|
rationale string still produce different roots. The
|
||||||
|
rejection_reason is the canonical string from the violation
|
||||||
|
(`"preflight rejection — broad-quantifier query with
|
||||||
|
unbounded scope. ..."`), NOT the operator-facing rendered
|
||||||
|
answer_text — that lets policy template changes invalidate
|
||||||
|
the hash even if the operator-visible text is unchanged.
|
||||||
|
"""
|
||||||
|
final_label_payload = {
|
||||||
|
"audit_mode": audit_mode,
|
||||||
|
"verifier_method": verifier_method,
|
||||||
|
"lookup_path": "preflight",
|
||||||
|
"rejection_reason": rejection_reason,
|
||||||
|
"answer_text_hash": _sha256_hex(answer_text or ""),
|
||||||
|
}
|
||||||
|
if violations is not None:
|
||||||
|
final_label_payload["violations"] = violations
|
||||||
|
final_label_hash = _sha256_hex(_canonical_json(final_label_payload))
|
||||||
|
nodes = [
|
||||||
|
{"stage": "question", "hash": question_hash},
|
||||||
|
{"stage": "preflight", "hash": preflight_hash},
|
||||||
|
{"stage": "final_label", "hash": final_label_hash},
|
||||||
|
]
|
||||||
|
leaves = [bytes.fromhex(n["hash"]) for n in nodes]
|
||||||
|
root_hex = MerkleTree.build(leaves).root.hex()
|
||||||
|
return {"root": root_hex, "nodes": nodes}
|
||||||
|
|
||||||
|
|
||||||
def verify_run_dag(blob: str | dict) -> bool:
|
def verify_run_dag(blob: str | dict) -> bool:
|
||||||
"""Recompute the Merkle root from ``blob`` and check it matches.
|
"""Recompute the Merkle root from ``blob`` and check it matches.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1733,19 +1733,111 @@ def query(
|
||||||
# we already know the answer set is undefined. Result schema
|
# we already know the answer set is undefined. Result schema
|
||||||
# mirrors a normal UNGROUNDED row so bench/CLI rendering
|
# mirrors a normal UNGROUNDED row so bench/CLI rendering
|
||||||
# stays consistent.
|
# stays consistent.
|
||||||
|
# Ticket #000009 §8.2 / feedback §6.2: build a 3-stage
|
||||||
|
# reject-broad DAG so the rejection is Merkle-auditable.
|
||||||
|
# Without this, two rejections under different policy state
|
||||||
|
# would be indistinguishable in audit replay.
|
||||||
|
from aborist.qa.dag import (
|
||||||
|
build_reject_run_dag,
|
||||||
|
preflight_node_hash as _pre_hash,
|
||||||
|
)
|
||||||
|
# question_hash / verifier_policy_hash / model_profile_hash
|
||||||
|
# already imported at module top; do NOT re-import locally
|
||||||
|
# (would shadow free-variable uses elsewhere in this function).
|
||||||
|
_reject_qhash = question_hash(
|
||||||
|
question,
|
||||||
|
mode=policy.get("question_dedup", "equivalence_class"),
|
||||||
|
)
|
||||||
|
_reject_ghash = verifier_policy_hash(policy)
|
||||||
|
_reject_mhash = model_profile_hash(model_id, revision, quantization)
|
||||||
|
_reject_rationale = (
|
||||||
|
"preflight rejection — broad-quantifier query with "
|
||||||
|
"unbounded scope. Operator opted in via "
|
||||||
|
"quantifier_reject_broad policy."
|
||||||
|
)
|
||||||
|
_reject_violations = [{
|
||||||
|
"kind": "BROAD_QUANTIFIER_REJECTED",
|
||||||
|
"intensity": quantifier["intensity"],
|
||||||
|
"matched_token": quantifier["matched_token"],
|
||||||
|
"scope_bound_hint": quantifier["scope_bound_hint"],
|
||||||
|
"reason": _reject_rationale,
|
||||||
|
}]
|
||||||
|
_reject_answer_text = (
|
||||||
|
"BROAD-QUANTIFIER PREFLIGHT REJECTED · scope unbounded\n\n"
|
||||||
|
f"Question matched {quantifier['intensity']} intensity "
|
||||||
|
f"(\"{quantifier['matched_token']}\") with an under-"
|
||||||
|
"specified universe. Narrow the question (e.g. add a "
|
||||||
|
"year, league, country, or category) or run with "
|
||||||
|
"--allow-broad for exploratory enumeration."
|
||||||
|
)
|
||||||
|
_reject_preflight_hash = _pre_hash(
|
||||||
|
question_state=question_state.to_dict(),
|
||||||
|
quantifier=quantifier,
|
||||||
|
answer_contract={
|
||||||
|
"guard_enabled": quantifier_guard_on,
|
||||||
|
"mode_gated": quantifier_mode_gated,
|
||||||
|
"apply_caps_active": quantifier_apply_caps,
|
||||||
|
"apply_caps_mode_gated": quantifier_caps_mode_gated,
|
||||||
|
"claim_cap_resolved": claim_cap_lookup,
|
||||||
|
"claim_cap_applied": None, # cap never reaches verifier on reject
|
||||||
|
"manual_quotes_allowed": False,
|
||||||
|
"evidence_pointer_required": True,
|
||||||
|
"allow_unbounded_enumeration": False,
|
||||||
|
"reject_broad_active": True,
|
||||||
|
"metacognition_enabled": bool(
|
||||||
|
policy.get("metacognition_enabled", True)
|
||||||
|
),
|
||||||
|
"block_on_contradiction": bool(
|
||||||
|
policy.get("metacognition_block_on_contradiction", False)
|
||||||
|
),
|
||||||
|
},
|
||||||
|
prompt_contract={
|
||||||
|
# Rejection skips the LLM, so no reminder ever fires.
|
||||||
|
"reminder_enabled": bool(
|
||||||
|
policy.get("quantifier_reminder_enabled", False)
|
||||||
|
),
|
||||||
|
"reminder_injected": False,
|
||||||
|
"reminder_template_id": None,
|
||||||
|
},
|
||||||
|
evidence_contract={
|
||||||
|
"max_evidence_ids_exposed": int(policy.get(
|
||||||
|
"claim_lattice_max_pointers_per_claim", 2
|
||||||
|
)),
|
||||||
|
"one_claim_per_line": True,
|
||||||
|
},
|
||||||
|
policy_refs={
|
||||||
|
"governance_policy_hash": _reject_ghash,
|
||||||
|
"model_profile_hash": _reject_mhash,
|
||||||
|
"answer_mode": answer_mode,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
_reject_run_dag = build_reject_run_dag(
|
||||||
|
question_hash=_reject_qhash,
|
||||||
|
preflight_hash=_reject_preflight_hash,
|
||||||
|
rejection_reason=_reject_rationale,
|
||||||
|
answer_text=_reject_answer_text,
|
||||||
|
audit_mode="UNGROUNDED",
|
||||||
|
verifier_method=(
|
||||||
|
"claim_lattice_pointer"
|
||||||
|
if answer_mode == "claim_lattice_pointer"
|
||||||
|
else "claim_lattice"
|
||||||
|
if answer_mode == "claim_lattice"
|
||||||
|
else "quote"
|
||||||
|
),
|
||||||
|
violations=_reject_violations,
|
||||||
|
)
|
||||||
return {
|
return {
|
||||||
"status": "broad_quantifier_rejected",
|
"status": "broad_quantifier_rejected",
|
||||||
"audit_mode": "UNGROUNDED",
|
"audit_mode": "UNGROUNDED",
|
||||||
"cache_key": None,
|
"cache_key": None,
|
||||||
"lookup_path": "preflight",
|
"lookup_path": "preflight",
|
||||||
"answer_text": (
|
# Audit binding: reject path now carries its own
|
||||||
"BROAD-QUANTIFIER PREFLIGHT REJECTED · scope unbounded\n\n"
|
# 3-stage run_dag (question → preflight → final_label)
|
||||||
f"Question matched {quantifier['intensity']} intensity "
|
# so audit replay can read the rejection from
|
||||||
f"(\"{quantifier['matched_token']}\") with an under-"
|
# run_dag_blob the same way it reads any other row.
|
||||||
"specified universe. Narrow the question (e.g. add a "
|
"run_dag_root": _reject_run_dag["root"],
|
||||||
"year, league, country, or category) or run with "
|
"run_dag_blob": json.dumps(_reject_run_dag, separators=(",", ":")),
|
||||||
"--allow-broad for exploratory enumeration."
|
"answer_text": _reject_answer_text,
|
||||||
),
|
|
||||||
"sources": [],
|
"sources": [],
|
||||||
"n_quotes": 0,
|
"n_quotes": 0,
|
||||||
"n_verified": 0,
|
"n_verified": 0,
|
||||||
|
|
@ -1756,17 +1848,7 @@ def query(
|
||||||
else "quote",
|
else "quote",
|
||||||
"unverified_quotes": [],
|
"unverified_quotes": [],
|
||||||
"partially_verified_quotes": [],
|
"partially_verified_quotes": [],
|
||||||
"violations": [{
|
"violations": _reject_violations,
|
||||||
"kind": "BROAD_QUANTIFIER_REJECTED",
|
|
||||||
"intensity": quantifier["intensity"],
|
|
||||||
"matched_token": quantifier["matched_token"],
|
|
||||||
"scope_bound_hint": quantifier["scope_bound_hint"],
|
|
||||||
"reason": (
|
|
||||||
"preflight rejection — broad-quantifier query with "
|
|
||||||
"unbounded scope. Operator opted in via "
|
|
||||||
"quantifier_reject_broad policy."
|
|
||||||
),
|
|
||||||
}],
|
|
||||||
"format_collapsed": None,
|
"format_collapsed": None,
|
||||||
"raw_answer": None,
|
"raw_answer": None,
|
||||||
"quantifier_intensity": quantifier["intensity"],
|
"quantifier_intensity": quantifier["intensity"],
|
||||||
|
|
@ -2663,36 +2745,54 @@ def query(
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
plan_hash = retrieval_plan_hash(plan)
|
plan_hash = retrieval_plan_hash(plan)
|
||||||
# Ticket #000009 — preflight node binding. Combines #000008
|
# Ticket #000009 — preflight node binding (nested CTI clauses
|
||||||
# quantifier classifier output + #000010 metacognition
|
# per ticket §8 / 2026-05-04 feedback). Single DAG stage with
|
||||||
# QuestionState + the behavioral policy decisions (cap
|
# five nested clauses (classifier, answer_contract,
|
||||||
# actually applied, reminder actually injected, reject path
|
# prompt_contract, evidence_contract, policy_refs) + the
|
||||||
# taken) into a single hash that contributes to run_dag_root.
|
# metacognition QuestionState.
|
||||||
# Audit replay can now distinguish two cache rows that have
|
|
||||||
# the same question + same model output but different
|
|
||||||
# preflight policy state.
|
|
||||||
from aborist.qa.dag import preflight_node_hash
|
from aborist.qa.dag import preflight_node_hash
|
||||||
|
# verifier_policy_hash + model_profile_hash already imported
|
||||||
|
# at module top; reusing the existing names. Local re-imports
|
||||||
|
# would shadow earlier free-variable uses.
|
||||||
|
ghash_for_dag = verifier_policy_hash(policy)
|
||||||
|
claim_cap_actually_applied = (
|
||||||
|
claim_cap_lookup
|
||||||
|
if (quantifier_apply_caps
|
||||||
|
and quantifier_caps_mode_gated
|
||||||
|
and claim_cap_lookup is not None)
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
# Reminder injection actually fires when guard is on AND
|
||||||
|
# mode-gated AND quantifier is broad AND policy enables it.
|
||||||
|
# Mirrors the gate in the runner.ask() / query() reminder
|
||||||
|
# block above.
|
||||||
|
reminder_eligible = (
|
||||||
|
quantifier_guard_on
|
||||||
|
and quantifier_mode_gated
|
||||||
|
and quantifier.get("is_broad", False)
|
||||||
|
)
|
||||||
|
reminder_enabled = bool(policy.get("quantifier_reminder_enabled", False))
|
||||||
|
reminder_injected = reminder_eligible and reminder_enabled
|
||||||
|
reminder_template_id = None
|
||||||
|
if reminder_injected:
|
||||||
|
reminder_template_id = (
|
||||||
|
"broad-quantifier-bounded-v1"
|
||||||
|
if quantifier.get("scope_bound_hint") == "bounded"
|
||||||
|
else "broad-quantifier-unbounded-v1"
|
||||||
|
)
|
||||||
preflight_hash = preflight_node_hash(
|
preflight_hash = preflight_node_hash(
|
||||||
question_state=question_state.to_dict(),
|
question_state=question_state.to_dict(),
|
||||||
quantifier=quantifier,
|
quantifier=quantifier,
|
||||||
policy_state={
|
answer_contract={
|
||||||
"guard_enabled": quantifier_guard_on,
|
"guard_enabled": quantifier_guard_on,
|
||||||
"guard_apply_caps": quantifier_apply_caps,
|
"mode_gated": quantifier_mode_gated,
|
||||||
"guard_apply_caps_mode_gated": quantifier_caps_mode_gated,
|
"apply_caps_active": quantifier_apply_caps,
|
||||||
|
"apply_caps_mode_gated": quantifier_caps_mode_gated,
|
||||||
"claim_cap_resolved": claim_cap_lookup,
|
"claim_cap_resolved": claim_cap_lookup,
|
||||||
"claim_cap_actually_applied": (
|
"claim_cap_applied": claim_cap_actually_applied,
|
||||||
quantifier_apply_caps
|
"manual_quotes_allowed": False,
|
||||||
and quantifier_caps_mode_gated
|
"evidence_pointer_required": is_lattice_mode,
|
||||||
and claim_cap_lookup is not None
|
"allow_unbounded_enumeration": False,
|
||||||
),
|
|
||||||
"reminder_enabled": bool(
|
|
||||||
policy.get("quantifier_reminder_enabled", False)
|
|
||||||
),
|
|
||||||
"reminder_eligible": (
|
|
||||||
quantifier_guard_on
|
|
||||||
and quantifier_mode_gated
|
|
||||||
and quantifier.get("is_broad", False)
|
|
||||||
),
|
|
||||||
"reject_broad_active": bool(
|
"reject_broad_active": bool(
|
||||||
policy.get("quantifier_reject_broad", False)
|
policy.get("quantifier_reject_broad", False)
|
||||||
),
|
),
|
||||||
|
|
@ -2700,9 +2800,27 @@ def query(
|
||||||
policy.get("metacognition_enabled", True)
|
policy.get("metacognition_enabled", True)
|
||||||
),
|
),
|
||||||
"block_on_contradiction": bool(
|
"block_on_contradiction": bool(
|
||||||
policy.get("metacognition_block_on_contradiction", False)
|
policy.get(
|
||||||
|
"metacognition_block_on_contradiction", False
|
||||||
|
)
|
||||||
),
|
),
|
||||||
},
|
},
|
||||||
|
prompt_contract={
|
||||||
|
"reminder_enabled": reminder_enabled,
|
||||||
|
"reminder_injected": reminder_injected,
|
||||||
|
"reminder_template_id": reminder_template_id,
|
||||||
|
},
|
||||||
|
evidence_contract={
|
||||||
|
"max_evidence_ids_exposed": int(policy.get(
|
||||||
|
"claim_lattice_max_pointers_per_claim", 2
|
||||||
|
)),
|
||||||
|
"one_claim_per_line": is_lattice_mode,
|
||||||
|
},
|
||||||
|
policy_refs={
|
||||||
|
"governance_policy_hash": ghash_for_dag,
|
||||||
|
"model_profile_hash": mhash,
|
||||||
|
"answer_mode": answer_mode,
|
||||||
|
},
|
||||||
)
|
)
|
||||||
run_dag = build_run_dag(
|
run_dag = build_run_dag(
|
||||||
question_hash=qhash,
|
question_hash=qhash,
|
||||||
|
|
|
||||||
|
|
@ -852,29 +852,46 @@ def ask(
|
||||||
}
|
}
|
||||||
for i, cs in enumerate(verdict.get("claim_statuses") or [])
|
for i, cs in enumerate(verdict.get("claim_statuses") or [])
|
||||||
]
|
]
|
||||||
# Ticket #000009 — preflight node binding (mirror of query()).
|
# Ticket #000009 — preflight node binding (mirror of query();
|
||||||
|
# nested CTI clauses per ticket §8 / 2026-05-04 feedback).
|
||||||
from aborist.qa.dag import preflight_node_hash
|
from aborist.qa.dag import preflight_node_hash
|
||||||
|
# verifier_policy_hash + model_profile_hash imported at module
|
||||||
|
# top; do NOT re-import locally (free-variable shadowing).
|
||||||
|
ghash_for_dag = verifier_policy_hash(policy)
|
||||||
|
claim_cap_actually_applied = (
|
||||||
|
claim_cap_lookup
|
||||||
|
if (quantifier_apply_caps
|
||||||
|
and quantifier_caps_mode_gated
|
||||||
|
and claim_cap_lookup is not None)
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
reminder_eligible = (
|
||||||
|
quantifier_guard_on
|
||||||
|
and quantifier_mode_gated
|
||||||
|
and quantifier.get("is_broad", False)
|
||||||
|
)
|
||||||
|
reminder_enabled = bool(policy.get("quantifier_reminder_enabled", False))
|
||||||
|
reminder_injected = reminder_eligible and reminder_enabled
|
||||||
|
reminder_template_id = None
|
||||||
|
if reminder_injected:
|
||||||
|
reminder_template_id = (
|
||||||
|
"broad-quantifier-bounded-v1"
|
||||||
|
if quantifier.get("scope_bound_hint") == "bounded"
|
||||||
|
else "broad-quantifier-unbounded-v1"
|
||||||
|
)
|
||||||
preflight_hash = preflight_node_hash(
|
preflight_hash = preflight_node_hash(
|
||||||
question_state=question_state.to_dict(),
|
question_state=question_state.to_dict(),
|
||||||
quantifier=quantifier,
|
quantifier=quantifier,
|
||||||
policy_state={
|
answer_contract={
|
||||||
"guard_enabled": quantifier_guard_on,
|
"guard_enabled": quantifier_guard_on,
|
||||||
"guard_apply_caps": quantifier_apply_caps,
|
"mode_gated": quantifier_mode_gated,
|
||||||
"guard_apply_caps_mode_gated": quantifier_caps_mode_gated,
|
"apply_caps_active": quantifier_apply_caps,
|
||||||
|
"apply_caps_mode_gated": quantifier_caps_mode_gated,
|
||||||
"claim_cap_resolved": claim_cap_lookup,
|
"claim_cap_resolved": claim_cap_lookup,
|
||||||
"claim_cap_actually_applied": (
|
"claim_cap_applied": claim_cap_actually_applied,
|
||||||
quantifier_apply_caps
|
"manual_quotes_allowed": False,
|
||||||
and quantifier_caps_mode_gated
|
"evidence_pointer_required": is_lattice_mode,
|
||||||
and claim_cap_lookup is not None
|
"allow_unbounded_enumeration": False,
|
||||||
),
|
|
||||||
"reminder_enabled": bool(
|
|
||||||
policy.get("quantifier_reminder_enabled", False)
|
|
||||||
),
|
|
||||||
"reminder_eligible": (
|
|
||||||
quantifier_guard_on
|
|
||||||
and quantifier_mode_gated
|
|
||||||
and quantifier.get("is_broad", False)
|
|
||||||
),
|
|
||||||
"reject_broad_active": bool(
|
"reject_broad_active": bool(
|
||||||
policy.get("quantifier_reject_broad", False)
|
policy.get("quantifier_reject_broad", False)
|
||||||
),
|
),
|
||||||
|
|
@ -885,6 +902,22 @@ def ask(
|
||||||
policy.get("metacognition_block_on_contradiction", False)
|
policy.get("metacognition_block_on_contradiction", False)
|
||||||
),
|
),
|
||||||
},
|
},
|
||||||
|
prompt_contract={
|
||||||
|
"reminder_enabled": reminder_enabled,
|
||||||
|
"reminder_injected": reminder_injected,
|
||||||
|
"reminder_template_id": reminder_template_id,
|
||||||
|
},
|
||||||
|
evidence_contract={
|
||||||
|
"max_evidence_ids_exposed": int(policy.get(
|
||||||
|
"claim_lattice_max_pointers_per_claim", 2
|
||||||
|
)),
|
||||||
|
"one_claim_per_line": is_lattice_mode,
|
||||||
|
},
|
||||||
|
policy_refs={
|
||||||
|
"governance_policy_hash": ghash_for_dag,
|
||||||
|
"model_profile_hash": mhash,
|
||||||
|
"answer_mode": answer_mode,
|
||||||
|
},
|
||||||
)
|
)
|
||||||
run_dag = build_run_dag(
|
run_dag = build_run_dag(
|
||||||
question_hash=qhash,
|
question_hash=qhash,
|
||||||
|
|
|
||||||
|
|
@ -58,7 +58,7 @@ Newest first. Update on every open/close.
|
||||||
| ID | Title | Status | Opened | Directive |
|
| ID | Title | Status | Opened | Directive |
|
||||||
|----------|------------------------------------------------|-----------------------|------------|-----------|
|
|----------|------------------------------------------------|-----------------------|------------|-----------|
|
||||||
| #000010 | Meta-Cognition Preflight Guard (M0 / MCTL) | closed · landed 2026-05-03 (Phases 1–4); DAG binding shipped via #000009 | 2026-05-03 | D1, D3 |
|
| #000010 | Meta-Cognition Preflight Guard (M0 / MCTL) | closed · landed 2026-05-03 (Phases 1–4); DAG binding shipped via #000009 | 2026-05-03 | D1, D3 |
|
||||||
| #000009 | Preflight run-DAG node binding (#000008+#000010) | closed · landed 2026-05-03 (zero-shot) | 2026-05-03 | D3, D4 |
|
| #000009 | Preflight run-DAG node binding (#000008+#000010) | closed · re-landed 2026-05-04 (§8 corrections: reject-path DAG, nested CTI clauses) | 2026-05-03 | D3, D4 |
|
||||||
| #000008 | Broad-quantifier preflight guard | closed · landed in `4f2b5a6`; Phase 5 DAG binding split into #000009 | 2026-05-02 | — |
|
| #000008 | Broad-quantifier preflight guard | closed · landed in `4f2b5a6`; Phase 5 DAG binding split into #000009 | 2026-05-02 | — |
|
||||||
| #000007 | Query-layer hyphen folding | closed · 2026-05-02 | 2026-05-02 | — |
|
| #000007 | Query-layer hyphen folding | closed · 2026-05-02 | 2026-05-02 | — |
|
||||||
| #000006 | Bench-emergent findings (rolling research log) | open · rolling | 2026-05-02 | — |
|
| #000006 | Bench-emergent findings (rolling research log) | open · rolling | 2026-05-02 | — |
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,10 @@
|
||||||
# Ticket #000009 — Preflight run-DAG node binding
|
# Ticket #000009 — Preflight run-DAG node binding
|
||||||
|
|
||||||
**Status:** closed · landed 2026-05-03 (zero-shot)
|
**Status:** closed · re-landed 2026-05-04 with §8 corrections
|
||||||
|
(reject-path DAG, nested CTI clauses, node_version, reminder
|
||||||
|
template id, policy_refs). Original landing 2026-05-03 in
|
||||||
|
`c36e85c`; §8 corrections per architectural feedback at
|
||||||
|
`~/Downloads/RESPONSE-ticket-000009-quantifier-preflight-dag-binding.txt`.
|
||||||
**Opened:** 2026-05-03
|
**Opened:** 2026-05-03
|
||||||
**Scope (expanded 2026-05-03):** Bind BOTH ticket #000008's broad-
|
**Scope (expanded 2026-05-03):** Bind BOTH ticket #000008's broad-
|
||||||
quantifier preflight contract AND ticket #000010's meta-cognition
|
quantifier preflight contract AND ticket #000010's meta-cognition
|
||||||
|
|
@ -280,3 +284,180 @@ preflight contract").
|
||||||
bench analysis needs it.
|
bench analysis needs it.
|
||||||
- **`SOFT_PREFLIGHT_HINT` (model-assisted preflight)**:
|
- **`SOFT_PREFLIGHT_HINT` (model-assisted preflight)**:
|
||||||
reserved per #000010 §18 / source doc. Hard rule preserved.
|
reserved per #000010 §18 / source doc. Hard rule preserved.
|
||||||
|
|
||||||
|
## 8. Architectural review + corrections (2026-05-04)
|
||||||
|
|
||||||
|
Source: `~/Downloads/RESPONSE-ticket-000009-quantifier-preflight-dag-binding.txt`
|
||||||
|
(2026-05-04, Asia/Kuala_Lumpur). Verdict: "Go. Land #000009."
|
||||||
|
Recommendations against the as-shipped (`c36e85c`) implementation:
|
||||||
|
|
||||||
|
### 8.1 Gaps surfaced
|
||||||
|
|
||||||
|
| # | Gap | Severity |
|
||||||
|
|---|--------------------------------------------------------|----------|
|
||||||
|
| A | **Reject-broad path emits NO run_dag_blob.** Preflight rejection currently early-returns from `query()` before `build_run_dag()` runs. Audit replay can't see that a rejection happened. | critical |
|
||||||
|
| B | Payload is flat (`question_state` / `quantifier` / `policy_state`). Feedback §3 recommends nested CTI clauses (`classifier`, `answer_contract`, `prompt_contract`, `evidence_contract`, `policy_refs`) for diff legibility. | structural |
|
||||||
|
| C | No `node_version` field. Legacy runs without the stage can't be unambiguously labeled `unavailable_legacy_run`. | clarity |
|
||||||
|
| D | Payload bundles raw policy booleans instead of referencing `governance_policy_hash` / `model_profile_hash`. Bloats the hash and double-commits already-hashed state. | clarity |
|
||||||
|
| E | No `reminder_template_id` field for the prompt contract. | minor |
|
||||||
|
| F | Stage named `preflight` (because metacog also lives there) vs feedback's `quantifier_preflight`. Defensible either way; we'll document the choice. | naming |
|
||||||
|
| G | `docs/cti-architecture.md` not updated. | docs |
|
||||||
|
|
||||||
|
### 8.2 Landing plan
|
||||||
|
|
||||||
|
1. **A — reject-path DAG.** Build a minimal `question → preflight →
|
||||||
|
final_label` DAG for the reject-broad early-return path. Reject
|
||||||
|
rows currently have no `run_dag_root` / `run_dag_blob`; this
|
||||||
|
gives them one with the preflight node and a final-label that
|
||||||
|
carries `BROAD_QUANTIFIER_REJECTED`. **Audit replay parity with
|
||||||
|
normal runs.**
|
||||||
|
2. **B — restructure payload.** Move from flat 3-key to nested
|
||||||
|
5-clause:
|
||||||
|
```
|
||||||
|
classifier — quantifier classifier output
|
||||||
|
answer_contract — guard / cap / reject / allow_broad state
|
||||||
|
prompt_contract — reminder enabled / injected / template_id
|
||||||
|
evidence_contract — exposure budget, line-discipline
|
||||||
|
policy_refs — governance_policy_hash, model_profile_hash,
|
||||||
|
answer_mode (reference, not raw policy)
|
||||||
|
```
|
||||||
|
Plus a top-level `node_version: "preflight-node-v1"` for legacy
|
||||||
|
disambiguation (C).
|
||||||
|
3. **D — reference hashes only.** Replace bundled policy booleans
|
||||||
|
with `policy_refs.governance_policy_hash` +
|
||||||
|
`policy_refs.model_profile_hash`. Keep the *behavioral*
|
||||||
|
decisions (claim_cap_applied, reminder_injected, etc.) in
|
||||||
|
`answer_contract` / `prompt_contract` since those are
|
||||||
|
per-run-state-of-the-world, not policy.
|
||||||
|
4. **E — reminder_template_id.** When reminder fires, record which
|
||||||
|
template (`broad-quantifier-bounded-v1` /
|
||||||
|
`broad-quantifier-unbounded-v1`).
|
||||||
|
5. **F — keep `preflight` name** because the node carries BOTH
|
||||||
|
#000008 quantifier classifier AND #000010 metacognition
|
||||||
|
QuestionState. `quantifier_preflight` would be misleading. The
|
||||||
|
`node_version` field disambiguates schema.
|
||||||
|
6. **G — update docs/cti-architecture.md.**
|
||||||
|
|
||||||
|
### 8.3 What stays as-is
|
||||||
|
|
||||||
|
The feedback's recommendations match what's already shipped on:
|
||||||
|
|
||||||
|
- Single DAG stage (not three separate nodes) — ✓
|
||||||
|
- Stage placement between `question` and `retrieval` — ✓
|
||||||
|
- Backward-compat (None preflight_hash → 7/9-stage shape) — ✓
|
||||||
|
- `governance_policy_hash` invalidates cache rows on policy
|
||||||
|
flip — ✓ (already in #000008 / #000010)
|
||||||
|
- No verifier / retrieval / schema changes — ✓
|
||||||
|
- Symmetric wiring in `query()` and `runner.ask()` — ✓
|
||||||
|
|
||||||
|
### 8.4 Out of scope (per feedback §12)
|
||||||
|
|
||||||
|
> Do not use #000009 to sneak in any of this:
|
||||||
|
>
|
||||||
|
> - new verifier rules
|
||||||
|
> - new quantifier classifier categories
|
||||||
|
> - retrieval changes
|
||||||
|
> - new schema/cache columns
|
||||||
|
> - new audit_mode token
|
||||||
|
> - NLI / semantic entailment
|
||||||
|
> - prompt-template behavior changes
|
||||||
|
> - default policy flips
|
||||||
|
|
||||||
|
Confirmed. Corrections stay strictly in the audit-binding lane.
|
||||||
|
|
||||||
|
### 8.5 Corrections landed (commit pending after this update)
|
||||||
|
|
||||||
|
**A — reject-path DAG** (the critical gap):
|
||||||
|
|
||||||
|
`aborist/qa/dag.py:build_reject_run_dag()` ships. Builds a 3-stage
|
||||||
|
`question → preflight → final_label` DAG for the reject-broad
|
||||||
|
early-return path. `query()` now wires it in and returns
|
||||||
|
`run_dag_root` + `run_dag_blob` on the rejection result dict.
|
||||||
|
Live-verified on `make query Q="winners of all major sports?"
|
||||||
|
REJECT_BROAD=1 BURN=1`:
|
||||||
|
|
||||||
|
```
|
||||||
|
"status": "broad_quantifier_rejected",
|
||||||
|
"run_dag_root": "86a03380…",
|
||||||
|
"run_dag_blob": {"nodes": [
|
||||||
|
{"stage": "question", "hash": "…"},
|
||||||
|
{"stage": "preflight", "hash": "…"},
|
||||||
|
{"stage": "final_label", "hash": "…"}
|
||||||
|
]}
|
||||||
|
```
|
||||||
|
|
||||||
|
3-stage shape always means reject path; audit replay can read the
|
||||||
|
stage list and tell instantly without parsing the payload.
|
||||||
|
|
||||||
|
**B — nested CTI clauses + C — node_version + D — reference
|
||||||
|
hashes + E — reminder_template_id**:
|
||||||
|
|
||||||
|
`preflight_node_hash()` payload restructured from flat 3-key
|
||||||
|
(`question_state` / `quantifier` / `policy_state`) to nested
|
||||||
|
5-clause:
|
||||||
|
|
||||||
|
```
|
||||||
|
classifier — quantifier classifier output
|
||||||
|
answer_contract — guard / cap / reject / metacog state (per-run)
|
||||||
|
prompt_contract — reminder enabled / injected / template_id
|
||||||
|
evidence_contract — exposure budget, line discipline
|
||||||
|
policy_refs — governance_policy_hash, model_profile_hash,
|
||||||
|
answer_mode (reference, not raw policy)
|
||||||
|
```
|
||||||
|
|
||||||
|
Plus top-level `stage`, `node_version: "preflight-node-v1"`, and
|
||||||
|
`question_state` (metacog, kept its own clause for now since
|
||||||
|
QuestionState carries its own internal `preflight_policy_hash`).
|
||||||
|
|
||||||
|
`reminder_template_id` is `"broad-quantifier-bounded-v1"` or
|
||||||
|
`"broad-quantifier-unbounded-v1"` depending on
|
||||||
|
`scope_bound_hint`, populated only when reminder actually fires.
|
||||||
|
|
||||||
|
`policy_refs.governance_policy_hash` is the
|
||||||
|
`verifier_policy_hash(policy)` already used for cache identity —
|
||||||
|
reusing the existing hash rather than re-canonicalizing all the
|
||||||
|
policy fields. `model_profile_hash` follows the same pattern.
|
||||||
|
|
||||||
|
**F — stage name kept as `preflight`** (not `quantifier_preflight`)
|
||||||
|
since the node carries both #000008 quantifier classifier AND
|
||||||
|
#000010 metacognition QuestionState. `node_version` field
|
||||||
|
disambiguates schema for audit tools.
|
||||||
|
|
||||||
|
**G — `docs/cti-architecture.md` update**: deferred to a small
|
||||||
|
follow-up commit. The architecture description in the ticket §8
|
||||||
|
serves as the canonical reference until then.
|
||||||
|
|
||||||
|
Hash compatibility note: rows written between commit `c36e85c`
|
||||||
|
(initial #000009 landing) and this commit have hash payloads
|
||||||
|
matching the OLD flat 3-key shape. The persisted
|
||||||
|
`run_dag_blob` captures the actual payload that was hashed, so
|
||||||
|
those rows still verify via `verify_run_dag()`. New rows after
|
||||||
|
this commit use the nested 5-clause shape. Operators reading
|
||||||
|
the `run_dag_blob` directly see the structure either way.
|
||||||
|
|
||||||
|
### 8.6 New tests
|
||||||
|
|
||||||
|
`tests/test_dag.py`:
|
||||||
|
- `test_preflight_node_hash_changes_with_answer_contract` —
|
||||||
|
apply_caps flip in answer_contract → distinct node hash.
|
||||||
|
- `test_preflight_node_hash_changes_with_prompt_contract` —
|
||||||
|
reminder injection flip → distinct hash.
|
||||||
|
- `test_preflight_node_hash_changes_with_policy_refs` —
|
||||||
|
governance_policy_hash flip → distinct hash.
|
||||||
|
- `test_preflight_node_hash_includes_node_version` — pins
|
||||||
|
`PREFLIGHT_NODE_VERSION = "preflight-node-v1"`.
|
||||||
|
- `test_reject_run_dag_three_stage_shape` — reject DAG always
|
||||||
|
3 stages (question → preflight → final_label).
|
||||||
|
- `test_reject_run_dag_root_changes_with_preflight_hash` —
|
||||||
|
audit-replay payoff for reject path.
|
||||||
|
- `test_reject_run_dag_round_trips_through_verify` — 3-stage
|
||||||
|
shape verifies the same way as 7/9/8/10-stage shapes.
|
||||||
|
|
||||||
|
993 tests passing (6 net new); 36 skipped.
|
||||||
|
|
||||||
|
### 8.7 Status — re-closed
|
||||||
|
|
||||||
|
Closed · landed via this commit on top of `c36e85c`. Audit-binding
|
||||||
|
gap from feedback §6.2 (reject path) closed; nested-clause payload
|
||||||
|
landed per feedback §3; node_version + reminder_template_id +
|
||||||
|
policy_refs landed per feedback §4 + §9.
|
||||||
|
|
|
||||||
|
|
@ -179,56 +179,105 @@ def test_verify_dag_accepts_json_string():
|
||||||
# ---------------------------------------------------------------- Ticket #000009: preflight stage
|
# ---------------------------------------------------------------- Ticket #000009: preflight stage
|
||||||
|
|
||||||
def test_preflight_node_hash_is_deterministic():
|
def test_preflight_node_hash_is_deterministic():
|
||||||
"""Same inputs → same hex string, byte-for-byte."""
|
"""Same nested-clause inputs → same hex string, byte-for-byte."""
|
||||||
qs = {"logical_statuses": ["well_formed"], "preflight_result": "PREFLIGHT_OK"}
|
qs = {"logical_statuses": ["well_formed"], "preflight_result": "PREFLIGHT_OK"}
|
||||||
quant = {"intensity": "SINGULAR", "is_broad": False}
|
quant = {"intensity": "SINGULAR", "is_broad": False}
|
||||||
pol = {"guard_enabled": True, "guard_apply_caps": False}
|
answer_contract = {"guard_enabled": True, "apply_caps_active": False}
|
||||||
a = preflight_node_hash(question_state=qs, quantifier=quant, policy_state=pol)
|
a = preflight_node_hash(
|
||||||
b = preflight_node_hash(question_state=qs, quantifier=quant, policy_state=pol)
|
question_state=qs, quantifier=quant,
|
||||||
|
answer_contract=answer_contract,
|
||||||
|
)
|
||||||
|
b = preflight_node_hash(
|
||||||
|
question_state=qs, quantifier=quant,
|
||||||
|
answer_contract=answer_contract,
|
||||||
|
)
|
||||||
assert a == b
|
assert a == b
|
||||||
assert len(a) == 64 # SHA-256 hex
|
assert len(a) == 64 # SHA-256 hex
|
||||||
|
|
||||||
|
|
||||||
def test_preflight_node_hash_changes_with_question_state():
|
def test_preflight_node_hash_changes_with_question_state():
|
||||||
base_quant = {"intensity": "SINGULAR", "is_broad": False}
|
base_quant = {"intensity": "SINGULAR", "is_broad": False}
|
||||||
pol = {"guard_enabled": True}
|
answer_contract = {"guard_enabled": True}
|
||||||
h_a = preflight_node_hash(
|
h_a = preflight_node_hash(
|
||||||
question_state={"preflight_result": "PREFLIGHT_OK"},
|
question_state={"preflight_result": "PREFLIGHT_OK"},
|
||||||
quantifier=base_quant, policy_state=pol,
|
quantifier=base_quant, answer_contract=answer_contract,
|
||||||
)
|
)
|
||||||
h_b = preflight_node_hash(
|
h_b = preflight_node_hash(
|
||||||
question_state={"preflight_result": "PREFLIGHT_PARTIAL"},
|
question_state={"preflight_result": "PREFLIGHT_PARTIAL"},
|
||||||
quantifier=base_quant, policy_state=pol,
|
quantifier=base_quant, answer_contract=answer_contract,
|
||||||
)
|
)
|
||||||
assert h_a != h_b
|
assert h_a != h_b
|
||||||
|
|
||||||
|
|
||||||
def test_preflight_node_hash_changes_with_policy_state():
|
def test_preflight_node_hash_changes_with_answer_contract():
|
||||||
"""Apply-caps flip MUST bump the preflight node hash so audit
|
"""Apply-caps flip in answer_contract MUST bump the node hash
|
||||||
replay can distinguish guard-on vs guard-off rows that
|
so audit replay can distinguish guard-on vs guard-off rows
|
||||||
otherwise share the same classifier output."""
|
that otherwise share the same classifier output."""
|
||||||
qs = {"logical_statuses": ["broad_quantifier_unbounded"]}
|
qs = {"logical_statuses": ["broad_quantifier_unbounded"]}
|
||||||
quant = {"intensity": "ALL", "is_broad": True}
|
quant = {"intensity": "ALL", "is_broad": True}
|
||||||
h_off = preflight_node_hash(
|
h_off = preflight_node_hash(
|
||||||
question_state=qs, quantifier=quant,
|
question_state=qs, quantifier=quant,
|
||||||
policy_state={"guard_apply_caps": False},
|
answer_contract={"apply_caps_active": False},
|
||||||
)
|
)
|
||||||
h_on = preflight_node_hash(
|
h_on = preflight_node_hash(
|
||||||
question_state=qs, quantifier=quant,
|
question_state=qs, quantifier=quant,
|
||||||
policy_state={"guard_apply_caps": True},
|
answer_contract={"apply_caps_active": True},
|
||||||
)
|
)
|
||||||
assert h_off != h_on
|
assert h_off != h_on
|
||||||
|
|
||||||
|
|
||||||
|
def test_preflight_node_hash_changes_with_prompt_contract():
|
||||||
|
"""Reminder-injection flip MUST bump the node hash."""
|
||||||
|
qs = {"logical_statuses": ["broad_quantifier_unbounded"]}
|
||||||
|
quant = {"intensity": "ALL", "is_broad": True}
|
||||||
|
h_off = preflight_node_hash(
|
||||||
|
question_state=qs, quantifier=quant,
|
||||||
|
prompt_contract={"reminder_injected": False},
|
||||||
|
)
|
||||||
|
h_on = preflight_node_hash(
|
||||||
|
question_state=qs, quantifier=quant,
|
||||||
|
prompt_contract={
|
||||||
|
"reminder_injected": True,
|
||||||
|
"reminder_template_id": "broad-quantifier-unbounded-v1",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
assert h_off != h_on
|
||||||
|
|
||||||
|
|
||||||
|
def test_preflight_node_hash_changes_with_policy_refs():
|
||||||
|
"""policy_refs.governance_policy_hash flip → distinct node
|
||||||
|
hash. Required for cache-key/run-DAG-root joint invalidation."""
|
||||||
|
qs = {"logical_statuses": ["well_formed"]}
|
||||||
|
h_a = preflight_node_hash(
|
||||||
|
question_state=qs,
|
||||||
|
policy_refs={"governance_policy_hash": "a" * 64},
|
||||||
|
)
|
||||||
|
h_b = preflight_node_hash(
|
||||||
|
question_state=qs,
|
||||||
|
policy_refs={"governance_policy_hash": "b" * 64},
|
||||||
|
)
|
||||||
|
assert h_a != h_b
|
||||||
|
|
||||||
|
|
||||||
|
def test_preflight_node_hash_includes_node_version():
|
||||||
|
"""node_version field is part of the hashed payload so a future
|
||||||
|
schema bump (preflight-node-v2 etc.) invalidates legacy nodes."""
|
||||||
|
from aborist.qa.dag import (
|
||||||
|
PREFLIGHT_NODE_VERSION,
|
||||||
|
build_preflight_node_payload,
|
||||||
|
)
|
||||||
|
payload = build_preflight_node_payload(
|
||||||
|
question_state={"x": 1},
|
||||||
|
)
|
||||||
|
assert payload["node_version"] == PREFLIGHT_NODE_VERSION
|
||||||
|
assert PREFLIGHT_NODE_VERSION == "preflight-node-v1"
|
||||||
|
|
||||||
|
|
||||||
def test_preflight_node_hash_handles_all_none():
|
def test_preflight_node_hash_handles_all_none():
|
||||||
"""Defensive — all three components may be None during gradual
|
"""Defensive — all clauses may be None during gradual rollout.
|
||||||
rollout. Hash stays stable."""
|
Hash stays stable."""
|
||||||
a = preflight_node_hash(
|
a = preflight_node_hash()
|
||||||
question_state=None, quantifier=None, policy_state=None,
|
b = preflight_node_hash()
|
||||||
)
|
|
||||||
b = preflight_node_hash(
|
|
||||||
question_state=None, quantifier=None, policy_state=None,
|
|
||||||
)
|
|
||||||
assert a == b
|
assert a == b
|
||||||
assert len(a) == 64
|
assert len(a) == 64
|
||||||
|
|
||||||
|
|
@ -252,7 +301,7 @@ def test_dag_with_preflight_inserts_eight_stage_shape():
|
||||||
pre_hash = preflight_node_hash(
|
pre_hash = preflight_node_hash(
|
||||||
question_state={"preflight_result": "PREFLIGHT_OK"},
|
question_state={"preflight_result": "PREFLIGHT_OK"},
|
||||||
quantifier={"intensity": "SINGULAR", "is_broad": False},
|
quantifier={"intensity": "SINGULAR", "is_broad": False},
|
||||||
policy_state={"guard_enabled": True},
|
answer_contract={"guard_enabled": True},
|
||||||
)
|
)
|
||||||
out = build_run_dag(**_kw(preflight_hash=pre_hash))
|
out = build_run_dag(**_kw(preflight_hash=pre_hash))
|
||||||
stages = [n["stage"] for n in out["nodes"]]
|
stages = [n["stage"] for n in out["nodes"]]
|
||||||
|
|
@ -268,7 +317,7 @@ def test_dag_with_preflight_lattice_mode_ten_stages():
|
||||||
pre_hash = preflight_node_hash(
|
pre_hash = preflight_node_hash(
|
||||||
question_state={"preflight_result": "PREFLIGHT_OK"},
|
question_state={"preflight_result": "PREFLIGHT_OK"},
|
||||||
quantifier={"intensity": "ALL", "is_broad": True},
|
quantifier={"intensity": "ALL", "is_broad": True},
|
||||||
policy_state={"guard_enabled": True},
|
answer_contract={"guard_enabled": True},
|
||||||
)
|
)
|
||||||
out = build_run_dag(**_kw(
|
out = build_run_dag(**_kw(
|
||||||
preflight_hash=pre_hash,
|
preflight_hash=pre_hash,
|
||||||
|
|
@ -291,12 +340,12 @@ def test_dag_root_changes_when_preflight_hash_changes():
|
||||||
pre_a = preflight_node_hash(
|
pre_a = preflight_node_hash(
|
||||||
question_state={"preflight_result": "PREFLIGHT_OK"},
|
question_state={"preflight_result": "PREFLIGHT_OK"},
|
||||||
quantifier={"intensity": "SINGULAR"},
|
quantifier={"intensity": "SINGULAR"},
|
||||||
policy_state={"guard_apply_caps": False},
|
answer_contract={"apply_caps_active": False},
|
||||||
)
|
)
|
||||||
pre_b = preflight_node_hash(
|
pre_b = preflight_node_hash(
|
||||||
question_state={"preflight_result": "PREFLIGHT_OK"},
|
question_state={"preflight_result": "PREFLIGHT_OK"},
|
||||||
quantifier={"intensity": "SINGULAR"},
|
quantifier={"intensity": "SINGULAR"},
|
||||||
policy_state={"guard_apply_caps": True},
|
answer_contract={"apply_caps_active": True},
|
||||||
)
|
)
|
||||||
a = build_run_dag(**_kw(preflight_hash=pre_a))
|
a = build_run_dag(**_kw(preflight_hash=pre_a))
|
||||||
b = build_run_dag(**_kw(preflight_hash=pre_b))
|
b = build_run_dag(**_kw(preflight_hash=pre_b))
|
||||||
|
|
@ -310,8 +359,79 @@ def test_dag_with_preflight_round_trips_through_verify():
|
||||||
pre_hash = preflight_node_hash(
|
pre_hash = preflight_node_hash(
|
||||||
question_state={"preflight_result": "PREFLIGHT_PARTIAL"},
|
question_state={"preflight_result": "PREFLIGHT_PARTIAL"},
|
||||||
quantifier={"intensity": "ALL", "is_broad": True},
|
quantifier={"intensity": "ALL", "is_broad": True},
|
||||||
policy_state={"guard_apply_caps": True, "claim_cap_resolved": 8},
|
answer_contract={"apply_caps_active": True, "claim_cap_resolved": 8},
|
||||||
)
|
)
|
||||||
out = build_run_dag(**_kw(preflight_hash=pre_hash))
|
out = build_run_dag(**_kw(preflight_hash=pre_hash))
|
||||||
blob = json.dumps(out, separators=(",", ":"))
|
blob = json.dumps(out, separators=(",", ":"))
|
||||||
assert verify_run_dag(blob) is True
|
assert verify_run_dag(blob) is True
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------- reject-broad DAG (§8.2 A)
|
||||||
|
|
||||||
|
def test_reject_run_dag_three_stage_shape():
|
||||||
|
"""Reject-broad early-return path produces a 3-stage DAG:
|
||||||
|
question → preflight → final_label. Audit replay can read the
|
||||||
|
stage list and tell instantly that this row is a preflight
|
||||||
|
rejection (3 stages = reject path)."""
|
||||||
|
from aborist.qa.dag import build_reject_run_dag, preflight_node_hash
|
||||||
|
pre = preflight_node_hash(
|
||||||
|
question_state={"preflight_result": "PREFLIGHT_BLOCKED"},
|
||||||
|
quantifier={"intensity": "ALL", "is_broad": True,
|
||||||
|
"scope_bound_hint": "unbounded"},
|
||||||
|
answer_contract={"reject_broad_active": True},
|
||||||
|
)
|
||||||
|
out = build_reject_run_dag(
|
||||||
|
question_hash="a" * 64,
|
||||||
|
preflight_hash=pre,
|
||||||
|
rejection_reason="preflight rejection — broad-unbounded.",
|
||||||
|
answer_text="BROAD-QUANTIFIER PREFLIGHT REJECTED",
|
||||||
|
violations=[{"kind": "BROAD_QUANTIFIER_REJECTED"}],
|
||||||
|
)
|
||||||
|
stages = [n["stage"] for n in out["nodes"]]
|
||||||
|
assert stages == ["question", "preflight", "final_label"]
|
||||||
|
assert len(stages) == 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_reject_run_dag_root_changes_with_preflight_hash():
|
||||||
|
"""Two reject runs that differ only in the preflight payload
|
||||||
|
(e.g. different policy state at rejection time) must produce
|
||||||
|
different run_dag_root values."""
|
||||||
|
from aborist.qa.dag import build_reject_run_dag, preflight_node_hash
|
||||||
|
pre_a = preflight_node_hash(
|
||||||
|
question_state={"preflight_result": "PREFLIGHT_BLOCKED"},
|
||||||
|
quantifier={"intensity": "ALL"},
|
||||||
|
policy_refs={"governance_policy_hash": "a" * 64},
|
||||||
|
)
|
||||||
|
pre_b = preflight_node_hash(
|
||||||
|
question_state={"preflight_result": "PREFLIGHT_BLOCKED"},
|
||||||
|
quantifier={"intensity": "ALL"},
|
||||||
|
policy_refs={"governance_policy_hash": "b" * 64},
|
||||||
|
)
|
||||||
|
a = build_reject_run_dag(
|
||||||
|
question_hash="d" * 64, preflight_hash=pre_a,
|
||||||
|
rejection_reason="r", answer_text="x",
|
||||||
|
)
|
||||||
|
b = build_reject_run_dag(
|
||||||
|
question_hash="d" * 64, preflight_hash=pre_b,
|
||||||
|
rejection_reason="r", answer_text="x",
|
||||||
|
)
|
||||||
|
assert a["root"] != b["root"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_reject_run_dag_round_trips_through_verify():
|
||||||
|
"""3-stage reject DAG must verify the same way as the standard
|
||||||
|
7/9/8/10-stage shapes."""
|
||||||
|
import json
|
||||||
|
from aborist.qa.dag import build_reject_run_dag, preflight_node_hash
|
||||||
|
pre = preflight_node_hash(
|
||||||
|
question_state={"preflight_result": "PREFLIGHT_BLOCKED"},
|
||||||
|
quantifier={"intensity": "ALL"},
|
||||||
|
)
|
||||||
|
out = build_reject_run_dag(
|
||||||
|
question_hash="a" * 64,
|
||||||
|
preflight_hash=pre,
|
||||||
|
rejection_reason="preflight rejection",
|
||||||
|
answer_text="REJECTED",
|
||||||
|
)
|
||||||
|
blob = json.dumps(out, separators=(",", ":"))
|
||||||
|
assert verify_run_dag(blob) is True
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue