docs: add tool-action-dag-design.md research path (pre-ticket)
Captures design for an action-provenance DAG layer downstream of final_label - five new stages (action_plan -> tool_call -> tool_output -> postcondition_check -> action_label) - and why this stays a research doc rather than an open ticket today. Three options analyzed: Option A in-tree action DAG (identity drift), Option B sidecar package (recommended; preserves the verified-answer-cache identity by chaining a separate action_root that cross-links into run_dag_root), Option C out-of-scope. Promotion criteria spelled out so the doc graduates to a ticket when the first agent use case shows up. Cross-links #000001 (upstream provenance gap), #000022 (LossReport, same axiom one stage upstream), #000012 (v8 selection could later score action histories), and the 2026-05-07 arborist-vs-donto comparison. TICKETS.md gains a pointer in "Distinction from other docs" so future shifts find the doc.
This commit is contained in:
parent
fbd99a8d76
commit
a66ab10351
2 changed files with 245 additions and 0 deletions
|
|
@ -47,6 +47,10 @@ proposing change. Examples that stay un-numbered:
|
|||
- `concept-relations-design.md` (architecture reference for the
|
||||
per-shard `concept_relations` synonym/rivalry layer + the 1.6%
|
||||
storage-tax rationale; landed 2026-05-01)
|
||||
- `tool-action-dag-design.md` (research path for an action-
|
||||
provenance layer downstream of `final_label`. Captured pre-ticket
|
||||
per the 2026-05-07 arborist-vs-donto comparison; promotes to a
|
||||
ticket when an actual agent use case surfaces)
|
||||
|
||||
If a doc proposes change AND awaits a decision AND has scoped
|
||||
implementation cost, it's a ticket. Otherwise it's reference.
|
||||
|
|
|
|||
241
docs/tool-action-dag-design.md
Normal file
241
docs/tool-action-dag-design.md
Normal file
|
|
@ -0,0 +1,241 @@
|
|||
# Tool/action DAG — research path
|
||||
|
||||
**Status:** research path · not a ticket · awaiting first agent use case
|
||||
**Opened:** 2026-05-07
|
||||
**Promotion trigger:** when an actual agent (an LLM that takes
|
||||
real-world actions through arborist's verified answers) is in flight
|
||||
or about to land. Until then this stays a doc.
|
||||
**Audience:** fox + future blackops shifts evaluating whether arborist
|
||||
should host an action-provenance layer.
|
||||
|
||||
## Why this is a doc, not a ticket
|
||||
|
||||
Per `docs/TICKETS.md`: a doc proposes change AND awaits a decision
|
||||
AND has scoped implementation cost → it's a ticket. Otherwise it's
|
||||
reference. This proposal has the first two but not the third — the
|
||||
implementation cost is conditional on a use case that doesn't exist
|
||||
in tree today (`grep` for `tool_call`, `action_plan`, etc. returns
|
||||
zero hits across `arborist/` and `docs/tickets/` as of 2026-05-07).
|
||||
Five-step algorithm step 2 says: don't optimize a process that
|
||||
shouldn't exist yet. So we capture the design now, ticket it later
|
||||
when an agent shows up.
|
||||
|
||||
## Problem statement
|
||||
|
||||
Today arborist's run-DAG ends at `final_label`. Everything
|
||||
downstream of "the LLM gave us a STRICT pointer-linked answer" is
|
||||
invisible to the audit chain. When arborist serves a question-
|
||||
answerer LLM upstream of human decision-making, that's fine — the
|
||||
human is the side-effect surface and lives outside arborist's
|
||||
audit scope.
|
||||
|
||||
When arborist serves an *agent* — an LLM that calls tools, makes
|
||||
API requests, modifies state on the strength of a verified answer —
|
||||
the audit chain has a downstream gap analogous to the one #000001
|
||||
closed for retrieval keywords:
|
||||
|
||||
- `cache_key` (8-dim): no action footprint
|
||||
- `run_dag_root`: no action footprint
|
||||
- `audit_events`: no action footprint
|
||||
- `providence_cache`: no action footprint
|
||||
|
||||
The map (verified answer) is committed; the territory reached
|
||||
(tools dispatched, side effects produced) is not.
|
||||
|
||||
### Concrete failure shapes
|
||||
|
||||
**Case A — silent action.** An agent reads `STRICT` from arborist,
|
||||
calls `delete_user(id=42)`, and the deletion is permanent. Audit
|
||||
replay can recover the question, the sources, the verified answer —
|
||||
but no record that anything was done with it.
|
||||
|
||||
**Case B — wrong tool.** LLM answers correctly, then dispatches
|
||||
the wrong tool (typo'd argument, schema drift). The verified answer
|
||||
was right; the action was wrong. Today there's no separation between
|
||||
"the answer was right" and "the action that the answer authorized
|
||||
was right."
|
||||
|
||||
**Case C — partial rollback.** Action sequence partially completes,
|
||||
host process crashes, restart finds inconsistent world state. No
|
||||
committed rollback plan to replay.
|
||||
|
||||
## Proposed DAG extension
|
||||
|
||||
Five new stages chained after `final_label`:
|
||||
|
||||
```
|
||||
final_label
|
||||
→ action_plan # LLM-proposed sequence, committed BEFORE any execution
|
||||
→ tool_call # one node per dispatch; committed at moment of dispatch
|
||||
→ tool_output # one node per response; committed when host returns it
|
||||
→ postcondition_check # did the declared postcondition hold?
|
||||
→ action_label # EXECUTED / FAILED / ROLLED_BACK / QUARANTINED
|
||||
```
|
||||
|
||||
`action_label` mirrors `audit_mode` — a binary verdict; soft signals
|
||||
stay sidecars. Same verifier-stays-binary discipline as the existing
|
||||
trichotomy.
|
||||
|
||||
## Three design options
|
||||
|
||||
### Option A — in-tree action DAG
|
||||
|
||||
Schema lives in arborist's SQLite. Host process registers a
|
||||
tool-dispatch callback; arborist commits the call before the
|
||||
callback runs and commits the output when it returns. Failures
|
||||
(timeout, exception) commit a node with `tool_output.status='failed'`
|
||||
so audit replay sees the attempt.
|
||||
|
||||
**Pros:** one audit chain, one Merkle root, one verifier path.
|
||||
|
||||
**Cons:** arborist's surface area expands to include a tool-dispatch
|
||||
contract. Pass 1 of the Donto comparison framed arborist as
|
||||
"upstream of an LLM" — Option A makes it both upstream and
|
||||
downstream. Identity drift.
|
||||
|
||||
### Option B — sidecar package (recommended)
|
||||
|
||||
Separate `arborist-action` package consumes arborist's `run_dag_root`
|
||||
and chains its own `action_root`. Two roots, audit chain spans both
|
||||
via a typed cross-link (e.g., `action_root.parent_run_dag_root` is
|
||||
a foreign key into the answer's run-DAG).
|
||||
|
||||
**Pros:**
|
||||
- Preserves arborist's Pass-1 identity. The verified-answer cache
|
||||
stays a verified-answer cache; agent-side audit lives next door.
|
||||
- Two-root model is the same shape the existing mesh code uses to
|
||||
chain across peers.
|
||||
- Operators who don't want action-DAG don't install it. Operators
|
||||
who do install it get the cross-link "for free."
|
||||
- Schema migrations live in the action package, not in arborist
|
||||
core — `chain-check-shards` and the existing audit chain stay
|
||||
unchanged.
|
||||
|
||||
**Cons:**
|
||||
- Two roots to verify, two packages to keep in sync.
|
||||
- Cross-link contract has its own drift surface (the foreign-key
|
||||
shape, what the action package commits to, what arborist
|
||||
commits to).
|
||||
|
||||
### Option C — out of scope
|
||||
|
||||
Document the boundary. Operators wanting action provenance build
|
||||
it in their host process and emit their own audit chain. Arborist
|
||||
stays a pure question→answer cache.
|
||||
|
||||
**Pros:** smallest blast radius. No new code, no new identity
|
||||
question.
|
||||
|
||||
**Cons:** every operator that hits this need re-derives the same
|
||||
shape. Provenance discipline becomes per-operator self-discipline,
|
||||
which is exactly the gap arborist refuses to accept on the
|
||||
question→answer side (cf. #000022 LossReport on the adapter side).
|
||||
|
||||
## Recommended landing shape (when promoted to ticket)
|
||||
|
||||
Option B, scoped to:
|
||||
|
||||
1. New package `arborist-action` (separate pyproject, separate
|
||||
import root) with its own SQLite schema and Merkle conventions
|
||||
matching `arborist/merkle.py`.
|
||||
2. Cross-link contract: `action_root` payload includes
|
||||
`parent_run_dag_root` and `parent_cache_key`. Verifier is
|
||||
"those values resolve to a live providence-cache row whose
|
||||
`audit_mode` is in the policy-allowed set."
|
||||
3. Tool dispatch happens in the host process. Action package
|
||||
commits `tool_call` *before* dispatch and `tool_output` when
|
||||
the host returns. Failures (timeout, exception) commit a
|
||||
`tool_output` row with `status='failed'`.
|
||||
4. Rollback plans are committed at `action_plan` time, replay-able
|
||||
from the audit chain. Actually running rollback is host-process
|
||||
work.
|
||||
5. `action_label` mirrors `audit_mode` — `EXECUTED` / `FAILED` /
|
||||
`ROLLED_BACK` / `QUARANTINED`. Binary verdict. Soft signals
|
||||
(e.g., "did the action have unintended side effects?") stay
|
||||
sidecars in the action package, never enter the proof path.
|
||||
|
||||
## Out of scope (whenever this ships)
|
||||
|
||||
- **Tool dispatch.** Arborist (core or action package) never calls
|
||||
a tool. It commits intent, call, and response. Host process
|
||||
executes.
|
||||
- **Semantic validation of tool outputs.** Verifier stays binary.
|
||||
`postcondition_check` is structural ("did this declared invariant
|
||||
hold?"), not semantic ("was this the right action?").
|
||||
- **Multi-step planning logic.** Whatever plan the LLM produced
|
||||
gets committed; the action package doesn't re-plan, doesn't
|
||||
optimize, doesn't constrain.
|
||||
- **Rollback execution.** Plan commit only. Running it is host
|
||||
work.
|
||||
- **The arborist 8-dim cache_key, the verifier, retrieval, schema
|
||||
/ canonicalization / chunking versions.** Untouched.
|
||||
|
||||
## The identity question
|
||||
|
||||
The Donto comparison (Pass 1) was explicit:
|
||||
|
||||
> Arborist is upstream of an LLM. Donto sits next to humans
|
||||
> curating contested knowledge.
|
||||
|
||||
Adding an action-DAG layer makes arborist (or its sidecar)
|
||||
*also* downstream of an LLM — committing what the LLM did, not
|
||||
just what it said. Three readings:
|
||||
|
||||
1. **Identity drift.** Arborist becomes a hybrid: cache for
|
||||
verified answers + ledger for agentic actions. Operators who
|
||||
wanted "pure verified-answer cache" now share a tree with
|
||||
action provenance even if they don't install the sidecar.
|
||||
2. **Natural extension.** The Merkle-AGI principle that says
|
||||
"every cognition-affecting input must be committed" also says
|
||||
"every action taken under a verified answer must be committed."
|
||||
Same axiom, downstream side. Option B (sidecar) honors the
|
||||
axiom without forcing the identity shift on operators who
|
||||
don't want it.
|
||||
3. **Premature.** No agent is hitting arborist today. Build
|
||||
action-DAG when the first real agent runs through, not before.
|
||||
Reading 3 is consistent with five-step algorithm step 2.
|
||||
|
||||
Reading 3 is the current default. Readings 1 and 2 reactivate when
|
||||
the first real agent surfaces.
|
||||
|
||||
## Promotion criteria
|
||||
|
||||
This doc graduates to a ticket when **any** of:
|
||||
|
||||
- An operator-in-tree (or a clearly named upstream consumer) has
|
||||
an LLM in flight that takes real-world actions on the strength
|
||||
of arborist verified answers.
|
||||
- A second LossReport-shaped audit gap surfaces (e.g., a Donto-style
|
||||
comparison naming a different downstream missing-provenance
|
||||
axis) and the cumulative case for "arborist needs a downstream
|
||||
provenance story" outweighs the identity-drift cost.
|
||||
- A verified-answer cache gets used for an irreversible action
|
||||
in a real production incident, and post-mortem can't recover
|
||||
what happened. (Don't wait for this. The first two are cheaper
|
||||
signals.)
|
||||
|
||||
Until then: the doc captures the design so the work is ready when
|
||||
the use case is.
|
||||
|
||||
## References
|
||||
|
||||
- `/home/fox/Downloads/arborist_vs_donto.pdf` §6.1 ("Pass 1 — Purpose
|
||||
and audience"). Names arborist's audience as "developer who
|
||||
wants `ask(question) → (answer, proof)`." This doc explicitly
|
||||
lives downstream of that contract.
|
||||
- `docs/tickets/ticket-000001-retrieval-keywords-audit-gap.md` —
|
||||
the same provenance-gap shape on the *upstream* (retrieval)
|
||||
side. Closing the analogous gap on the action side is the
|
||||
symmetric move.
|
||||
- `docs/tickets/ticket-000022-adapter-loss-report.md` — same
|
||||
axiom ("every causally relevant transformation must surface
|
||||
what it dropped or did") applied to adapter normalization.
|
||||
Action-DAG would extend the axiom one stage further downstream.
|
||||
- `docs/tickets/ticket-000012-selection-consensus-protocol.md` —
|
||||
v8 selection protocol. If action-DAG ships, accepted mutations
|
||||
in v8 fork-choice should be able to reference action histories
|
||||
as evidence (e.g., "this policy patch reduced rollback rate by
|
||||
X%"). Cross-link surface, not implementation dependency.
|
||||
- `arborist/qa/dag.py` — the existing run-DAG construction; the
|
||||
action package would chain into the resulting `run_dag_root`
|
||||
via Option B's foreign key, not modify this module.
|
||||
Loading…
Add table
Add a link
Reference in a new issue