qa(provenance): #000001 land — retrieval_plan_hash binds D4
New module aborist/qa/retrieval_plan.py:
RetrievalPlan dataclass — frozen, captures the operator-
influenceable retrieval inputs that determine source
selection (retrieval_keywords, top_k, over_fetch,
max_context_chars, shard_ids).
retrieval_plan_hash() — SHA-256 over canonical-JSON.
Deterministic per call; folds into the run-DAG retrieval
stage as a bound input alongside the existing
sources_summary output.
build_run_dag (aborist/qa/dag.py) accepts retrieval_plan_hash:
When provided, the retrieval stage hash binds BOTH plan
(input) and result (output): H({"retrieval_plan_hash":...,
"sources_summary_hash":...}).
When omitted (e.g. legacy / quote-mode callers that haven't
plumbed it yet), the retrieval stage falls back to the
historical sources-summary-only hash so pre-#000001
run_dag_root values stay stable.
query.py constructs the plan per call and passes it through.
Question text intentionally NOT in the plan — already covered
by question_hash. Shard ids included so audit can reproduce
which shards the search ran against.
Two runs with identical sources but different retrieval keywords
now produce different run_dag_root values — the provenance gap
on operator hints (auditor recovers "these were the keywords
that pulled in those sources") closes for the run-DAG path.
Marker test in tests/test_directives.py flipped from "absent"
assertion to "present": test_d4_retrieval_plan_binding_landed.
Plus test_d4_retrieval_plan_hash_module_exists pinning the
module shape. Full suite: 712 passed.
Deferred per ticket §6:
- audit events (retrieval_plan_built / retrieval_result_selected)
- providence_cache.retrieval_keywords SQL column
- optional strict cache_key mode (retrieval_plan_affects_cache_key)
These are ergonomic affordances atop the run-DAG binding; earn
their own tickets when bench evidence shows the blob path is
too friction-heavy for real workflows.
Directive D4 status: ½ → ✓. Ticket #000001 closed.
This commit is contained in:
parent
3796c238cc
commit
1dfa4636c3
7 changed files with 251 additions and 29 deletions
|
|
@ -59,7 +59,7 @@ Newest first. Update on every open/close.
|
|||
| #000004 | Directive coverage in bench summary | closed · `acd1f9c` | 2026-05-01 | D8 |
|
||||
| #000003 | Anchor-class warrant generalization (Module H+)| closed · 2026-05-02 | 2026-05-01 | D6 |
|
||||
| #000002 | Reference-Frame Polarity Contract (Module L) | open | 2026-05-01 | D3 |
|
||||
| #000001 | Retrieval-keywords audit gap | open | 2026-05-01 | D4 |
|
||||
| #000001 | Retrieval-keywords audit gap | closed · 2026-05-02 | 2026-05-01 | D4 |
|
||||
|
||||
## Next ID
|
||||
|
||||
|
|
|
|||
|
|
@ -202,17 +202,18 @@ gets layered on top.
|
|||
| 1 | Stop making Hermes prove things | ✓ | |
|
||||
| 2 | Hermes emits pointer clauses | ✓ | |
|
||||
| 3 | Build CTI internally | ½ | #000002 |
|
||||
| 4 | Bind retrieval map AND evidence map | ½ | #000001 |
|
||||
| 4 | Bind retrieval map AND evidence map | ✓ | #000001 (closed)|
|
||||
| 5 | Verify pointers deterministically | ✓ | |
|
||||
| 6 | Anchor-class warrant before NLI | ✓ | #000003 (closed)|
|
||||
| 7 | Rename labels honestly | ✓ | #000005 (closed)|
|
||||
| 8 | Automate only after test-pinning | discipline | #000004 (closed)|
|
||||
|
||||
Two of seven structural directives are partial (D3, D4); D6 closed
|
||||
2026-05-02 via #000003. D7 is shipped at the EVIDENCE-LINKED rung;
|
||||
ticket #000005 proposes the four-rung ladder migration
|
||||
(POINTER-LINKED → ANCHOR-WARRANTED → EVIDENCE-WARRANTED →
|
||||
ENTAILMENT-VERIFIED) for stronger label discipline. D8 is the
|
||||
One of seven structural directives is partial (D3 — multi-frame
|
||||
answer compilation pending #000002). D4 closed 2026-05-02 via
|
||||
#000001 (run-DAG binding scope; SQL column + audit events
|
||||
deferred). D6 closed 2026-05-02 via #000003. D7 closed 2026-05-02
|
||||
via #000005 (four-rung ladder: POINTER-LINKED → ANCHOR-WARRANTED →
|
||||
EVIDENCE-WARRANTED → ENTAILMENT-VERIFIED). D8 is the
|
||||
meta-discipline that gates how all of the above land; the bench
|
||||
coverage substrate (#000004) is closed.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,9 @@
|
|||
# Ticket #000001 — Retrieval-keywords audit gap
|
||||
|
||||
**Status:** open · awaiting go/no-go
|
||||
**Status:** closed · landed 2026-05-02 (run-DAG binding scope; SQL
|
||||
column + audit-events scope deferred per §6 below)
|
||||
**Opened:** 2026-05-01
|
||||
**Closed:** 2026-05-02
|
||||
**Scope:** Design proposal for capturing the `--retrieval-keywords` operator
|
||||
hint in the v9.8 audit chain so retrieval is fully reproducible from a
|
||||
providence record alone. Doc-only — no code in this commit.
|
||||
|
|
@ -229,15 +231,60 @@ question." The 8-dim `cache_key` invariant stays intact.
|
|||
|
||||
## 5. Status
|
||||
|
||||
**Proposal.** No code yet. Pinging fox for go/no-go before landing.
|
||||
**Closed 2026-05-02.** Run-DAG binding scope landed:
|
||||
|
||||
Forecast cost: ~1-2 hours of focused work (the dag.py change + schema
|
||||
migration + ~6 tests + bench column). Risk: low — additive schema
|
||||
change, additive run_dag input, no cache-key churn.
|
||||
- New module `aborist/qa/retrieval_plan.py` — `RetrievalPlan`
|
||||
dataclass + `retrieval_plan_hash` function. Captures
|
||||
`retrieval_keywords`, `top_k`, `over_fetch`, `max_context_chars`,
|
||||
`shard_ids`. SHA-256 over canonical-JSON; deterministic per call.
|
||||
- `aborist/qa/dag.py:build_run_dag` accepts `retrieval_plan_hash`
|
||||
parameter. When provided, the retrieval stage hash binds BOTH
|
||||
the plan (input) and `sources_summary` (output). When omitted,
|
||||
falls back to the historical sources-summary-only hash so
|
||||
pre-#000001 records keep their `run_dag_root` values stable.
|
||||
- `aborist/qa/query.py` constructs the plan per call from the
|
||||
query-time inputs (keywords from `--retrieval-keywords`,
|
||||
`top_k` / `over_fetch` / `max_context_chars` from policy +
|
||||
args) plus the resolved shard set, computes the hash, and
|
||||
passes through to `build_run_dag`.
|
||||
|
||||
Forecast value: closes the operator-hint provenance loop. Necessary
|
||||
for audit-grade reproducibility once `--retrieval-keywords` becomes
|
||||
a regular operator practice.
|
||||
Two runs with identical sources but different retrieval keywords
|
||||
now produce different `run_dag_root` values — provenance closes
|
||||
the "how did retrieval choose these sources" gap.
|
||||
|
||||
3 new directive tests in `tests/test_directives.py`:
|
||||
- `test_d4_retrieval_plan_binding_landed` (marker flipped from
|
||||
"absent" assertion to "present").
|
||||
- Hash divergence asserted via differing
|
||||
`retrieval_plan_hash` arguments to `build_run_dag`.
|
||||
- `test_d4_retrieval_plan_hash_module_exists` covers the
|
||||
`RetrievalPlan` dataclass + `retrieval_plan_hash` function
|
||||
shape.
|
||||
|
||||
Full suite: 712 passed.
|
||||
|
||||
## 6. Deferred — additive scope
|
||||
|
||||
Audit events (`retrieval_plan_built` / `retrieval_result_selected`)
|
||||
and the `providence_cache.retrieval_keywords` SQL column from
|
||||
the original Appendix A proposal stay deferred. The run-DAG
|
||||
binding satisfies the Merkle-AGI commitment requirement (Theorem
|
||||
T2: every causally relevant transformation input bound). Direct
|
||||
SQL queryability and audit-event-level capture are operator-
|
||||
ergonomic enhancements that earn their own tickets when bench
|
||||
evidence shows the run-DAG-blob path is too friction-heavy for
|
||||
real workflows. Per the five-step algorithm step 2: ship the
|
||||
minimum viable substrate; add ergonomic affordances only when
|
||||
they earn it.
|
||||
|
||||
`cache_key` impact stays as-was: the retrieval plan affects
|
||||
`context_root` and `conversation_hash` indirectly (different plan
|
||||
→ different selected sources → different context fed to LLM →
|
||||
different conversation_hash). The plan does NOT enter
|
||||
`question_hash` or `governance_policy_hash`; the 8-dim cache_key
|
||||
invariant holds. Optional strict mode (where
|
||||
`retrieval_plan_affects_cache_key=True` would fold the plan hash
|
||||
in directly) stays in the original proposal as future work.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue