arborist/tests
russell@unturf.com 111dda6160
qa(#000009): §8 corrections — reject-path DAG + nested CTI clauses
Architectural feedback at ~/Downloads/RESPONSE-ticket-000009-... .txt
(2026-05-04) flagged five gaps in the c36e85c landing. Most
critical: reject-broad early-return path emitted no run_dag_blob,
so audit replay couldn't see that a rejection happened (let alone
under what policy state).

A — reject-path DAG (the critical gap):

  aborist/qa/dag.py: build_reject_run_dag() — 3-stage minimal DAG
  question → preflight → final_label. final_label payload carries
  rejection_reason + answer_text_hash so two rejections under
  different policy state produce different roots.

  query.py reject path now wires it: returns run_dag_root +
  run_dag_blob on the rejection result dict. Live-verified end-
  to-end on `make query Q="winners of all major sports?"
  REJECT_BROAD=1 BURN=1`.

  Audit replay rule: 3 stages always means reject path. Operators
  can read the stage list and tell instantly without parsing the
  payload.

B — nested CTI clauses:

  preflight_node_hash() payload restructured from flat 3-key to
  nested 5-clause:

    classifier        — quantifier classifier output (#000008)
    answer_contract   — guard / cap / reject / metacog state (per-run)
    prompt_contract   — reminder enabled / injected / template_id
    evidence_contract — exposure budget, line discipline
    policy_refs       — governance_policy_hash, model_profile_hash,
                        answer_mode (reference, not raw policy)

  Plus question_state (metacog) as its own clause and top-level
  stage + node_version. Single DAG stage; nested clauses inside
  for diff legibility (feedback §3).

C — node_version field:

  PREFLIGHT_NODE_VERSION = "preflight-node-v1" pinned in the
  payload so legacy runs without the node can be unambiguously
  labeled `unavailable_legacy_run` by audit tools (feedback §9).

D — reference hashes only:

  policy_refs uses governance_policy_hash + model_profile_hash
  rather than bundling raw policy booleans. Avoids
  double-committing already-hashed state (feedback §4).

E — reminder_template_id:

  prompt_contract.reminder_template_id = "broad-quantifier-bounded-v1"
  or "broad-quantifier-unbounded-v1" depending on scope_bound_hint,
  populated only when reminder actually fires.

F — stage name kept as `preflight` (not `quantifier_preflight`):

  Node carries both #000008 quantifier AND #000010 metacognition
  payloads. node_version disambiguates schema for audit tools.

G — docs/cti-architecture.md update deferred to a small follow-up.

Bug fixes:
  - free-variable shadowing on verifier_policy_hash /
    model_profile_hash / question_hash — local re-imports inside
    the reject branch shadowed module-top imports used elsewhere
    in query() / runner(); now use the module-top names.
  - reject path question_hash signature: takes `mode=` not
    `dedup_mode=` — fixed in the reject DAG builder caller.

Hash compatibility:
  Rows written between c36e85c and this commit have hash payloads
  matching the OLD flat 3-key shape. The persisted run_dag_blob
  captures the actual payload that was hashed, so those rows
  still verify via verify_run_dag(). New rows use the nested
  5-clause shape.

7 new tests in tests/test_dag.py:
  - hash sensitivity to answer_contract / prompt_contract /
    policy_refs flips (audit-replay payoff demonstrations)
  - PREFLIGHT_NODE_VERSION pinning
  - reject DAG: 3-stage shape, root changes with preflight hash,
    round-trips through verify_run_dag

993 tests passing (6 net new); 36 skipped.

Live verification:
  make query Q="winners of all major sports?" REJECT_BROAD=1 BURN=1
  → status=broad_quantifier_rejected, run_dag_root populated,
    blob carries 3-stage shape.

  make query Q="winners of all major sports?" BURN=1
  → 10-stage shape preserved (question → preflight → retrieval
    → ... → final_label).

Ticket #000009 status: closed · re-landed 2026-05-04 with §8
corrections.
2026-05-03 18:49:56 -04:00
..
crawler crawler: skip feeds + sitemaps at ingest (they're discovery, not knowledge) 2026-04-29 05:55:28 -04:00
fixtures verify: strip wikitext from context before substring matching 2026-04-28 15:48:07 -04:00
__init__.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_bench_qa_sweep.py qa(#000010): Phases 2-4 land — wired, governed, labeled, benched 2026-05-03 18:22:18 -04:00
test_burn.py cli: 'aborist burn-kindergarten' — mass-burn fresh providence rows 2026-04-29 16:56:59 -04:00
test_burn_doc.py cli: extend 'aborist burn' to documents and cores 2026-04-28 17:32:40 -04:00
test_claim_lattice.py qa(verify): FORMAT_COLLAPSED soft-demote + open #000008 (broad-quantifier preflight) 2026-05-02 16:43:41 -04:00
test_cli_render.py qa(verify): FORMAT_COLLAPSED soft-demote + open #000008 (broad-quantifier preflight) 2026-05-02 16:43:41 -04:00
test_compress.py storage cheats + TF-IDF retrieval fix 2026-04-27 17:24:51 -04:00
test_concepts.py qa(concepts): corpus-derived concept_relations table replaces frozensets 2026-05-01 21:14:53 -04:00
test_dag.py qa(#000009): §8 corrections — reject-path DAG + nested CTI clauses 2026-05-03 18:49:56 -04:00
test_directives.py qa(provenance): #000001 land — retrieval_plan_hash binds D4 2026-05-01 19:26:01 -04:00
test_distill.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_distill_recursive.py recursive distillation: core -> depth+1 core 2026-04-27 07:54:30 -04:00
test_evict.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_frame.py qa(frame): #000002 land — reference-frame polarity contract (D3 → ✓) 2026-05-01 20:18:13 -04:00
test_grok_source.py add Grok export source: conversations + media posts 2026-04-27 13:49:42 -04:00
test_html_source.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_ingest.py verify: layered strategies + entity policies, rename VISUAL → UNGROUNDED 2026-04-28 16:58:31 -04:00
test_inspect.py qa(inspect): wordlist union (US+UK) + configurable supplemental dictionaries 2026-05-02 14:27:24 -04:00
test_journal.py journal: emit unfirehose/1.0 JSONL for queries + bench cycles 2026-05-02 15:19:39 -04:00
test_keys.py arch: layer-cake docs + verifier_policy_hash + old-maps principle 2026-05-01 12:10:29 -04:00
test_llm_context_base.py qa: pipe wikitext-base-v1 into LLM context (was verifier-only) 2026-04-28 17:08:49 -04:00
test_merkle.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_mesh.py mesh: cryptographic foundation, off by default 2026-04-27 19:00:24 -04:00
test_mesh_aead.py mesh: optional AEAD body encryption under per-epoch shared secret 2026-04-28 17:34:39 -04:00
test_mesh_chain.py mesh: per-peer audit chain-of-claims tracking on the wire 2026-04-28 19:41:25 -04:00
test_mesh_cli.py mesh: sync default-holds records younger than 1 hour (kindergarten window) 2026-04-29 16:07:24 -04:00
test_mesh_cli_pull.py mesh: 'pull' CLI verb — fetch document body from a peer 2026-04-28 17:33:27 -04:00
test_mesh_wire.py mesh: HTTP gossip wire — signed envelopes + 5 message types 2026-04-28 16:57:04 -04:00
test_mesh_wire_e2e.py mesh: HTTP gossip wire — signed envelopes + 5 message types 2026-04-28 16:57:04 -04:00
test_metacognition.py qa(#000010): Phases 2-4 land — wired, governed, labeled, benched 2026-05-03 18:22:18 -04:00
test_migration_audit_mode.py verify: layered strategies + entity policies, rename VISUAL → UNGROUNDED 2026-04-28 16:58:31 -04:00
test_providence_source.py docs: consolidate self-reference design — flat MVP + fact-Core distillation 2026-05-02 12:39:42 -04:00
test_qa.py qa: post-LLM faithfulness verifier sets STRICT/HYBRID/VISUAL audit_mode 2026-04-28 15:45:08 -04:00
test_qa_quality_live.py qa(verifier): promote deflection sidecar to soft-demote violation 2026-05-01 23:44:32 -04:00
test_quantifier_caps.py qa(#000008): §12.10 n=5 verification + §12.11 defaults flipped (Option A) 2026-05-03 17:25:38 -04:00
test_quantifier_classifier.py qa(#000008): classifier fix — count-question short-circuit + bounded fixtures 2026-05-03 08:29:36 -04:00
test_quantifier_phase4.py qa(#000008): Phase 4 — CLI flags + violation tails + reject-broad 2026-05-03 07:41:13 -04:00
test_quantifier_reminder.py qa(#000008): Phase 3 — broad-quantifier reminder mechanism (default off) 2026-05-03 07:33:40 -04:00
test_query.py qa(query): #000007 land — query-layer hyphen folding 2026-05-02 14:12:46 -04:00
test_reclassify.py verify: layered strategies + entity policies, rename VISUAL → UNGROUNDED 2026-04-28 16:58:31 -04:00
test_repair.py qa: chain-segment failure localization + re-prompt repair tier 2026-04-29 19:27:33 -04:00
test_resume.py resumable ingest + per-shard audit chain integrity 2026-04-27 11:29:27 -04:00
test_snapshot.py add corpus-level snapshots: single-hash identity for the forest 2026-04-27 21:29:10 -04:00
test_tfidf.py add TF-IDF keyword distiller 2026-04-27 07:55:53 -04:00
test_vcs_source.py add git and mercurial repo sources for self-play 2026-04-27 18:17:39 -04:00
test_verify.py qa: per-claim status taxonomy on verifier + repair-action plans on sidecar 2026-04-29 18:37:52 -04:00
test_verify_json.py qa(verify): Rule 9 — SUBJECT_TOKENS_ABSENT premise-parroting demote 2026-05-02 15:51:58 -04:00
test_versioned_ingest.py storage cheats + TF-IDF retrieval fix 2026-04-27 17:24:51 -04:00
test_warrant.py qa(warrant): #000003 land — anchor-class generalization (D6 → ✓) 2026-05-01 19:16:06 -04:00
test_wikipedia_old.py prepare full Wikipedia 2003-05-16 ingest: cur + old (revisions) 2026-04-27 08:10:42 -04:00
test_wikipedia_xml.py add Phase IV Wikipedia XML + abstract sources 2026-04-27 17:24:28 -04:00
test_wikitext.py verify: layered strategies + entity policies, rename VISUAL → UNGROUNDED 2026-04-28 16:58:31 -04:00