qa(provenance): #000001 land — retrieval_plan_hash binds D4

New module aborist/qa/retrieval_plan.py:
  RetrievalPlan dataclass — frozen, captures the operator-
    influenceable retrieval inputs that determine source
    selection (retrieval_keywords, top_k, over_fetch,
    max_context_chars, shard_ids).
  retrieval_plan_hash() — SHA-256 over canonical-JSON.
    Deterministic per call; folds into the run-DAG retrieval
    stage as a bound input alongside the existing
    sources_summary output.

build_run_dag (aborist/qa/dag.py) accepts retrieval_plan_hash:
  When provided, the retrieval stage hash binds BOTH plan
    (input) and result (output): H({"retrieval_plan_hash":...,
    "sources_summary_hash":...}).
  When omitted (e.g. legacy / quote-mode callers that haven't
    plumbed it yet), the retrieval stage falls back to the
    historical sources-summary-only hash so pre-#000001
    run_dag_root values stay stable.

query.py constructs the plan per call and passes it through.
Question text intentionally NOT in the plan — already covered
by question_hash. Shard ids included so audit can reproduce
which shards the search ran against.

Two runs with identical sources but different retrieval keywords
now produce different run_dag_root values — the provenance gap
on operator hints (auditor recovers "these were the keywords
that pulled in those sources") closes for the run-DAG path.

Marker test in tests/test_directives.py flipped from "absent"
assertion to "present": test_d4_retrieval_plan_binding_landed.
Plus test_d4_retrieval_plan_hash_module_exists pinning the
module shape. Full suite: 712 passed.

Deferred per ticket §6:
  - audit events (retrieval_plan_built / retrieval_result_selected)
  - providence_cache.retrieval_keywords SQL column
  - optional strict cache_key mode (retrieval_plan_affects_cache_key)

These are ergonomic affordances atop the run-DAG binding; earn
their own tickets when bench evidence shows the blob path is
too friction-heavy for real workflows.

Directive D4 status: ½ → ✓. Ticket #000001 closed.
This commit is contained in:
russell@unturf.com 2026-05-01 19:26:01 -04:00
parent 3796c238cc
commit 1dfa4636c3
No known key found for this signature in database
7 changed files with 251 additions and 29 deletions

View file

@ -59,7 +59,7 @@ Newest first. Update on every open/close.
| #000004 | Directive coverage in bench summary | closed · `acd1f9c` | 2026-05-01 | D8 |
| #000003 | Anchor-class warrant generalization (Module H+)| closed · 2026-05-02 | 2026-05-01 | D6 |
| #000002 | Reference-Frame Polarity Contract (Module L) | open | 2026-05-01 | D3 |
| #000001 | Retrieval-keywords audit gap | open | 2026-05-01 | D4 |
| #000001 | Retrieval-keywords audit gap | closed · 2026-05-02 | 2026-05-01 | D4 |
## Next ID

View file

@ -202,17 +202,18 @@ gets layered on top.
| 1 | Stop making Hermes prove things | ✓ | |
| 2 | Hermes emits pointer clauses | ✓ | |
| 3 | Build CTI internally | ½ | #000002 |
| 4 | Bind retrieval map AND evidence map | ½ | #000001 |
| 4 | Bind retrieval map AND evidence map | ✓ | #000001 (closed)|
| 5 | Verify pointers deterministically | ✓ | |
| 6 | Anchor-class warrant before NLI | ✓ | #000003 (closed)|
| 7 | Rename labels honestly | ✓ | #000005 (closed)|
| 8 | Automate only after test-pinning | discipline | #000004 (closed)|
Two of seven structural directives are partial (D3, D4); D6 closed
2026-05-02 via #000003. D7 is shipped at the EVIDENCE-LINKED rung;
ticket #000005 proposes the four-rung ladder migration
(POINTER-LINKED → ANCHOR-WARRANTED → EVIDENCE-WARRANTED →
ENTAILMENT-VERIFIED) for stronger label discipline. D8 is the
One of seven structural directives is partial (D3 — multi-frame
answer compilation pending #000002). D4 closed 2026-05-02 via
#000001 (run-DAG binding scope; SQL column + audit events
deferred). D6 closed 2026-05-02 via #000003. D7 closed 2026-05-02
via #000005 (four-rung ladder: POINTER-LINKED → ANCHOR-WARRANTED →
EVIDENCE-WARRANTED → ENTAILMENT-VERIFIED). D8 is the
meta-discipline that gates how all of the above land; the bench
coverage substrate (#000004) is closed.

View file

@ -1,7 +1,9 @@
# Ticket #000001 — Retrieval-keywords audit gap
**Status:** open · awaiting go/no-go
**Status:** closed · landed 2026-05-02 (run-DAG binding scope; SQL
column + audit-events scope deferred per §6 below)
**Opened:** 2026-05-01
**Closed:** 2026-05-02
**Scope:** Design proposal for capturing the `--retrieval-keywords` operator
hint in the v9.8 audit chain so retrieval is fully reproducible from a
providence record alone. Doc-only — no code in this commit.
@ -229,15 +231,60 @@ question." The 8-dim `cache_key` invariant stays intact.
## 5. Status
**Proposal.** No code yet. Pinging fox for go/no-go before landing.
**Closed 2026-05-02.** Run-DAG binding scope landed:
Forecast cost: ~1-2 hours of focused work (the dag.py change + schema
migration + ~6 tests + bench column). Risk: low — additive schema
change, additive run_dag input, no cache-key churn.
- New module `aborist/qa/retrieval_plan.py``RetrievalPlan`
dataclass + `retrieval_plan_hash` function. Captures
`retrieval_keywords`, `top_k`, `over_fetch`, `max_context_chars`,
`shard_ids`. SHA-256 over canonical-JSON; deterministic per call.
- `aborist/qa/dag.py:build_run_dag` accepts `retrieval_plan_hash`
parameter. When provided, the retrieval stage hash binds BOTH
the plan (input) and `sources_summary` (output). When omitted,
falls back to the historical sources-summary-only hash so
pre-#000001 records keep their `run_dag_root` values stable.
- `aborist/qa/query.py` constructs the plan per call from the
query-time inputs (keywords from `--retrieval-keywords`,
`top_k` / `over_fetch` / `max_context_chars` from policy +
args) plus the resolved shard set, computes the hash, and
passes through to `build_run_dag`.
Forecast value: closes the operator-hint provenance loop. Necessary
for audit-grade reproducibility once `--retrieval-keywords` becomes
a regular operator practice.
Two runs with identical sources but different retrieval keywords
now produce different `run_dag_root` values — provenance closes
the "how did retrieval choose these sources" gap.
3 new directive tests in `tests/test_directives.py`:
- `test_d4_retrieval_plan_binding_landed` (marker flipped from
"absent" assertion to "present").
- Hash divergence asserted via differing
`retrieval_plan_hash` arguments to `build_run_dag`.
- `test_d4_retrieval_plan_hash_module_exists` covers the
`RetrievalPlan` dataclass + `retrieval_plan_hash` function
shape.
Full suite: 712 passed.
## 6. Deferred — additive scope
Audit events (`retrieval_plan_built` / `retrieval_result_selected`)
and the `providence_cache.retrieval_keywords` SQL column from
the original Appendix A proposal stay deferred. The run-DAG
binding satisfies the Merkle-AGI commitment requirement (Theorem
T2: every causally relevant transformation input bound). Direct
SQL queryability and audit-event-level capture are operator-
ergonomic enhancements that earn their own tickets when bench
evidence shows the run-DAG-blob path is too friction-heavy for
real workflows. Per the five-step algorithm step 2: ship the
minimum viable substrate; add ergonomic affordances only when
they earn it.
`cache_key` impact stays as-was: the retrieval plan affects
`context_root` and `conversation_hash` indirectly (different plan
→ different selected sources → different context fed to LLM →
different conversation_hash). The plan does NOT enter
`question_hash` or `governance_policy_hash`; the 8-dim cache_key
invariant holds. Optional strict mode (where
`retrieval_plan_affects_cache_key=True` would fold the plan hash
in directly) stays in the original proposal as future work.
---