Two fan-out streams. Both bear directly on the ticket's "search
latency on real shards" headline finding.
## Lazy concept_relations loading
Phase 1 (migration memoization) cut SQLite executes 65% but warm-
cache wall barely moved. cProfile pinned the next hotspot:
synonym_expand 2.8 s × 2 calls + _load_token_idf 0.3 s. The eager
loader dumped all ~290 K concept_relations rows on first call —
the price of being able to answer ANY future question without
re-querying. Wrong tradeoff for single-query CLI use.
Refactored arborist/concepts/query.py:
- New _load_neighbors_for(shards_dir, tokens) — targeted
WHERE token IN (...) OR target IN (...) query. Returns just the
direct synonym neighborhood for the given tokens (~300 rows for
a typical 5-token question, vs 290 K for the full table).
- New _load_rivalry_rows(shards_dir) — process-wide cache of the
~2-row rivalry-relation set; near-zero cost.
- New _load_idf_for(shards_dir, tokens) — IDF only fetched when
expansion exceeds max_total (the cap). Most queries never reach
the truncation branch and skip IDF entirely.
- Per-token process-wide neighbor cache so multi-query bench scripts
don't re-query tokens already seen.
- synonym_expand and rivalry_excluded refactored to use the lazy
loaders. Eager _load_indices / _get_indices kept for any
back-compat caller; not used by hot paths.
- invalidate_cache() clears all three caches.
All 14 concept tests pass unchanged — the contract is preserved.
Re-profile of `who wrote virt-back?` against ~38 GB of real shards
(warm cache):
metric pre-fix post-Phase-1 post-lazy-concepts
wall_ms 14,500 13,400 9,300 (-36%)
search_ms 9,900 10,600 5,000 (-49%)
SQLite executes 10,623 3,687 3,708 ~same
synonym_expand 2,966 2,840 ~0 (lazy hit)
Search target was <5 s; we hit 5.0 s on the warm path. Cold cache
should drop further (the 290 K-row dump was disk-bound).
## Real-shard baseline artifact (Phase 2)
bench/scripts/real_shard_baseline.py — runs an 8-question fixture
through the full query() pipeline and emits:
- bench/results/real-shard-baseline.json (durable; commit_sha,
shards_fingerprint, per-query rows, summary)
- bench/results/real-shard-baseline.md (human-readable summary)
Question set in bench/fixtures/real-shard-baseline-v1.jsonl:
virt-back, France, Mac OS X, Linux, Microsoft, AMD/Intel
(rivalry path), and two canonical-projection cases (math + logic
preflight short-circuit).
First baseline run (commit e78814c plus this fan-out, BURN=1):
audit_mode n notes
STRICT 4 (virt-back, France, Mac OS X, Linux)
HYBRID 2 (Microsoft founder, AMD/Intel)
CANONICAL 2 (0.1+0.2, A IMPL B; <1 ms each)
wall median 4.1 s (range 0.6 ms – 7.2 s)
primary used 4 / 8
`who wrote virt-back?` lands at 6.3 s wall, audit STRICT, primary
source #1, cited evidence still includes a copyright footer
(reviewer's warrant-quality finding — deferred to follow-up ticket
since the latency fix was the gating concern).
Hard constraint preserved: baselines NEVER gate CI. The artifact
is for confirming a fix moved the needle, seeding ForkScore
comparisons, and noting findings worth tickets.
`make bench-real-shard` wires it. Honors ARBORIST_SHARDS_DIR.
## Test status
Full suite: 1306 passed, 36 skipped (no regressions from the
concept refactor; 14 concept tests cover the lazy/eager
equivalence).
`connect()` used to run executescript(SCHEMA_SQL) + 7 forward-
migration probes on every open. Profile of `who wrote virt-back?`
on 38 GB of real shards (warm cache) showed 588 connect() calls
per query, each running the full probe sequence — 10,623 total
SQLite executes. Migrations are forward-only and idempotent within
a code version, so once we've run them on a path in this process
there's no work to do on subsequent opens.
Cache shape: `set[str]` keyed by `str(Path(p).resolve())`.
Migration block runs once per (path, process); subsequent calls on
the same shard skip it entirely. Per-connection PRAGMAs
(foreign_keys=ON, synchronous=NORMAL, cache_size, temp_store,
mmap_size) still run every time — SQLite scopes foreign_keys
per-connection and our schema's FK CASCADE behavior depends on it.
That's why `PRAGMA foreign_keys = ON` moved out of the cached
SCHEMA_SQL block into the always-run pragma section.
Cache invalidation: explicit only.
`store.invalidate_migration_cache(path)` for callers who replace a
shard at the same path (snapshot-restore flows). `_clear_migration_
cache()` for tests. We don't auto-detect file replacement —
(dev, inode) is unreliable under tmpfs inode reuse, and (mtime,
size) drifts naturally as SQLite operates on the file (WAL
checkpoints, page growth). Path-only with explicit invalidation
is the honest contract.
Re-profile (same query, same shards, warm cache):
metric before after
_migrate_* (each function) 586 7 ← per shard
executescript 586 7
SQLite executes 10,623 3,687 (-65%)
wall (warm) 14.5 s 13.4 s
The warm-cache wall delta is small because the probes were many-
but-cheap; residual cost lives in FTS5 search (6.7 s) and
synonym_expand (2.8 s, both separate concerns). The 65% execute
drop is the cold-cache win — each redundant executescript() had
been triggering disk reads at the 75 s scale the reviewer reported.
Tests (6, all green): first connect runs all 7 probes, second
connect runs zero, schema integrity preserved across re-opens,
explicit invalidation re-probes, distinct paths each get one probe,
clear-cache helper works.
Full suite: 1306 passed, 36 skipped. Found and fixed an FK CASCADE
regression mid-implementation: PRAGMA foreign_keys = ON was inside
SCHEMA_SQL, so memoization was silently turning it off on subsequent
opens. test_burn_doc.py caught it. Moved to the per-connection
pragma block.
Ticket #000026 status: Phase 1 landed; Phase 2 (baseline artifact)
and Phase 3 (warrant-quality finding) queued.
Profiled the `virt-back` query against ~38 GB of real shards
(2026-05-08, warm cache). Total wall 14.5 s; LLM 2.7 s; search
9.9 s — that's the latency budget breakdown.
Headline finding: 588 SQLite `connect()` calls per query, each
running 7 forward-migration probes on already-fully-migrated
shards. 10,623 SQLite executes total. Reviewer's 75 s cold-cache
report tracks the same shape; warm cache only masks part of it.
Ticket bundles:
- Phase 1 (latency fix): per-process migration memoization. Smallest
patch; no schema impact; ~30 LOC + regression test. Target <5 s
search on real shards.
- Phase 2 (baseline artifact): bench/scripts/real_shard_baseline.py
+ bench/results/real-shard-baseline.json + make bench-real-shard.
Captures wall/search/LLM timings, primary-source-at-rank-1, used
flags, capital ledger + memory deltas, ForkScore preview.
- Phase 3 (findings): fold the warrant-quality observation
(copyright-footer vs author-metadata) into the baseline report
as a finding. Implementation deferred to a follow-up ticket.
Out of scope captured explicitly: authorship warrant ladder
(separate ticket post-latency), CI re-enable (small commit, not
a ticket), public docs refresh (adjacent, separable), ForkScore
live wiring (#000012 Phase 1b).
Index updated. Next ID 000027.
Reviewer note (2026-05-08): the review claimed the index showed
"5 open, 24 closed" but no count line ever existed in TICKETS.md;
the 6 active rows (5 open + 1 rolling) are correctly listed. No
fix needed.
`make query Q="0.1 + 0.2"` used to return `no_sources` because
arithmetic-shaped input has no FTS5 hits in any text shard. Two
surfaces close that gap.
**`arborist canon <key> "<input>"`** — direct π* call, no shards,
no LLM, no audit chain. Pure projection:
$ arborist canon arithmetic@v1 "0.1 + 0.2" → 3/10
$ arborist canon logic-kernel@v1 "A IMPL B" → (NOT A OR B)
$ arborist canon --list → registry contents
$ arborist canon --json arithmetic@v1 "0.1+0.2" → SHA-256 envelope
**Math/logic preflight in `arborist query`** — pure-arithmetic and
pure-propositional questions short-circuit RAG and answer through
arithmetic@v1 / logic-kernel@v1 directly. Synthetic
`audit_mode=CANONICAL_PROJECTION`, renders as
`CANONICAL · via <pi_star_ref>`:
$ arborist query "0.1 + 0.2"
0.1 + 0.2
CANONICAL · via arithmetic@v1 0.0s (projected)
3/10
$ arborist query "(NOT B) IMPL (NOT A)"
(NOT B) IMPL (NOT A)
CANONICAL · via logic-kernel@v1 0.0s (projected)
(NOT A OR B)
Sniff is conservative: pure-arithmetic shape (digits + ops, no
letters) or pure-propositional shape (uppercase atoms + reserved
keywords only). Natural-language wrapping ("what is 0.1+0.2?")
falls through to RAG. PiStarError on a shape match also falls
through — preflight is best-effort, never blocking.
Disable per-call: `--no-canonical-preflight` flag,
`policy["canonical_projection_preflight"]=False`.
No schema changes: CANONICAL_PROJECTION is a render-layer audit_mode
token. No providence_cache writes, no audit_events, no
governance_policy_hash bump. The canonical bytes ARE the answer;
SHA-256 of the bytes is the equivalence-class identity (already
committed via the π* registry).
Side housekeeping: arborist/pi_star/__init__.py docstring caught up
with reality — six concrete π*'s ship today, only tabular-pinned@v1
remains as a stub.
31 new tests (preflight sniff + dispatch, query short-circuit,
contrapositive equivalence-class collapse, CLI subcommand exit codes
and JSON envelope, --no-canonical-preflight policy gate). Full
suite: 1300 passed, 36 skipped.
Completes the bridge: every 5F sub-battery now has an embedded
(Phase 1a) AND a live (Phase 1b.2) path. Per-task detail.source
distinguishes synthetic from live signal.
| Sub-battery | Live surface | Live fixtures |
|---|---|---|
| Formulate | qa.parse_claims.parse_pointer_claims | 15 (prior) |
| Feedback Loop | store.append_audit + memory.snapshot (temp shard) | 12 (prior) |
| Function | parse_pointer_claims + shape evaluators | 12 |
| Finetuning | selfmodel.store_snapshot + claims_for round-trip | 10 |
| Falsification | qa.verify.verify_quotes (real verifier) | 12 |
Function live: input_text routes through real parse_pointer_claims;
the same Phase-1a shape evaluators (shape_match / pointer_set_match
/ threshold_on_metric) run on the live-parsed output. Tests
parser→shape pipeline against actual organism behavior.
Finetuning live: gated via fixture's "live": true flag. Runner
writes parent + child SelfModel + capability_claim to a fresh temp
shard via real arborist.selfmodel.store_snapshot, reads back via
claims_for, runs the same improvement check. Tests the SelfModel
persistence + claim-attach surface, not just static fixture data.
Falsification live: answer_text + context routes through real
arborist.qa.verify.verify_quotes. Live signals translate the
verifier's audit_mode + verifier_method + unverified_quotes into
flat tags (UNGROUNDED, STRICT_<method>, HYBRID_<method>,
UNVERIFIED_QUOTE) the runner matches against expected_reason. One
fixture (5f-fal-live-003) deliberately documents a known soft-
signal gap — the entity strategy treats Insulin/Penicillin claims
as equivalent because both share "Alexander Fleming". Production
catches it via title-relevance + claim-lattice verifier; the
fixture pins the soft-path limit so future verifier changes
re-trigger review.
Surface:
- bench/batteries/b_5f.py — _live_function_produced,
_live_finetuning_measure, _live_falsification_violations
helpers + two-mode dispatch in run_function / run_finetuning /
run_falsification.
- bench/fixtures/5f/{function,finetuning,falsification}-live-v1.jsonl
(12 + 10 + 12 fixtures).
- Makefile: bench-5f-{function,finetuning,falsification}-live
targets; bench-5f-live aggregate now covers all five sub-batteries.
- Tests: 9 new in tests/test_bench_batteries.py
- live + embedded paths for each sub-battery (6 tests)
- direct helper tests verifying the real arborist surfaces are
invoked, not stubs (3 tests)
Phase 1a fixture digests unchanged. _DEFAULT_FIXTURES still points
at synthetic Phase-1a fixtures so `runner --all` behavior is
identical and deterministic; live mode invoked via explicit
--fixtures path or make targets.
Full suite: 1210 passed, 36 skipped.
5F battery is now the first to bridge synthetic→live across every
sub-battery. The pattern + live fixture format are reusable for
5R Phase 1b.2 (when SelfModel-backed workspace ops want live
verification).
First sub-battery to bridge from synthetic gold output to actual
organism behavior. run_formulate now supports two fixture modes
selected per-task:
- Embedded (Phase 1a): produced_lattice in the fixture. 10 seed
fixtures continue to pass via this path.
- Live (Phase 1b.2): only input_text in the fixture; runner calls
arborist.qa.parse_claims.parse_pointer_claims(input_text) and
matches the live output against expected_lattice.
Embedded takes precedence if both fields are present. Per-task
detail.source ("embedded" | "live") surfaces in bench output so
synthetic vs live signal is distinguishable.
Surface:
- bench/batteries/b_5f.py — _live_produced_lattice helper +
two-mode dispatch in run_formulate
- bench/fixtures/5f/formulate-live-v1.jsonl — 15 live-mode fixtures
with input_text + expected_lattice (no produced_lattice)
- Makefile: bench-5f-formulate-live target
- Tests: 4 new in tests/test_bench_batteries.py
- live path routes through real parser, all 15 pass
- embedded path still works (10 Phase-1a fixtures)
- _live_produced_lattice helper directly verifies parse output
- fixture missing both fields fails cleanly with explanatory reason
Phase 1a fixture digests unchanged. _DEFAULT_FIXTURES still points
at formulate-v1.jsonl so `runner --all` behavior is identical;
live-mode fixtures invoked via explicit --fixtures path.
Full suite: 1196 passed, 36 skipped.
Pattern set. Function/Finetuning/Falsification/Feedback Loop
follow in subsequent commits.
Phase 2 of #000021. React/Rearrange/Restore/Replicate/Resonate over
the workspace surface — selfmodel_records (#000014) + memory_records
(#000017), both landed earlier today. Closes the gap that gated 5R
since the substrate work shipped.
Sub-battery semantics (per SQD whitepaper §9.3 + ticket #000021 §4.2):
- React: incorporate new fact/constraint. Workspace = (snapshot_t0,
snapshot_t1, expected_delta). Pass = added_facts present + removed_facts
absent in t+1.
- Rearrange: restructure without semantic shift. Re-canonicalize
different surface forms through a named π*; pass = bytes match
expected_equivalent flag. Tests the order-invariance contracts in
SelfModel (capability_claim_hashes sorted) and Memory (branches
sorted by branch_id).
- Restore: retrieve prior fact. Workspace = (history[], current_facts[]).
Pass = fact in current OR any historical snapshot.
- Replicate: independent canonical encodings via π*. Same input run
N times must yield byte-equal output. Tests determinism contract.
- Resonate: variance across N runs. Deterministic π*'s yield
distinct=1; expected_max_distinct=1 enforces zero-variance contract.
Surface:
- bench/batteries/b_5r.py (5 deterministic runners; no LLM-as-judge)
- bench/fixtures/5r/{react,rearrange,restore,replicate,resonate}-v1.jsonl
(30 each = 150 new fixtures)
- runner.py registers 5r in _BATTERIES + _DEFAULT_FIXTURES
- Makefile: bench-5r + bench-suite (5S+5T+5F+5R aggregate)
Final tally:
5S syntax/semantics/syllogism/synthesis/semiotics 108
5T transfer/transfer-learning/triangulation/... 154
5F function/finetuning/falsification/... 50
5R react/rearrange/restore/replicate/resonate 150
TOTAL: 462 fixtures across 21 sub-batteries — 100% pass.
Tests: 6 new in tests/test_bench_batteries.py + adjustment to
test_session_integration.py for the 312→462 count + 5R sub-battery
presence assertion. Full suite: 1192 passed, 36 skipped.
Closes#000021. Phase 3 (external-corpus expansion) remains open
under the ticket but does not gate closure — the complete
Dav1DPrometheus surface is now executable infrastructure.
The top-level module diagram still used the old aborist- (one R) prefix
from before the package rename. arborist.unturf.com served from the
symlinked docs/_source/diagrams/, so the live URL
arborist.unturf.com/en/diagrams/arborist-modules.svg 404'd because the
file on disk was aborist-modules.svg.
Other diagram base names (ingest-pipeline, mesh-*, query-pipeline,
verifier-ladder) were never aborist-prefixed; only this one needed it.
Per fox's 2026-05-08 review of fbd99a8: implement adaptation_efficiency
and feedback_efficiency in run_finetuning + run_feedback_loop with
explicit sentinels — not Python floating-point accidents.
_efficiency(gain, cost) helper:
- cost > 0: standard ratio
- cost == 0, gain > 0: EFFICIENCY_INFINITE (free improvement)
- cost == 0, gain == 0: EFFICIENCY_UNDEFINED (= 0.0; no signal)
- cost == 0, gain < 0: -EFFICIENCY_INFINITE (free regression)
Battery-level metrics report mean_finite (computed over finite
values only) + infinite_count + neg_infinite_count so the mean
stays dimensionally truthful and consumers can pivot on the special
buckets separately.
Phase 1a cost proxies:
- run_finetuning: _capital_cost_delta sums resource_budget
(max_compute_ms_delta * 1e-3 + max_storage_delta_bytes / 1e6).
Phase 1b.2 will replace with real capital_ledger reads.
- run_feedback_loop: chain length = cost. Phase 1b.2 capital_ledger
integration replaces it.
Tests added (7):
- _efficiency over four boundary cases
- 5F finetuning + feedback_loop emit the new metrics keys
- Synthetic zero-cost finetuning fixture verifies +inf path
Full suite: 1110 passed, 36 skipped.
Closes the one actionable from the fbd99a8 review.
Captures design for an action-provenance DAG layer downstream of
final_label - five new stages (action_plan -> tool_call ->
tool_output -> postcondition_check -> action_label) - and why this
stays a research doc rather than an open ticket today.
Three options analyzed: Option A in-tree action DAG (identity
drift), Option B sidecar package (recommended; preserves the
verified-answer-cache identity by chaining a separate action_root
that cross-links into run_dag_root), Option C out-of-scope.
Promotion criteria spelled out so the doc graduates to a ticket
when the first agent use case shows up. Cross-links #000001
(upstream provenance gap), #000022 (LossReport, same axiom one
stage upstream), #000012 (v8 selection could later score action
histories), and the 2026-05-07 arborist-vs-donto comparison.
TICKETS.md gains a pointer in "Distinction from other docs" so
future shifts find the doc.
Per the three review responses (~/Downloads/RESPONSE_*) folded in
2026-05-08, the 5S / 5T / 5F tickets are corrected from "text-only
with future hooks" to "carrier-aware design from day one." Phase 1
implementation stays text / claim-lattice / memory-root only, but the
fixture schema MUST accommodate future visual / world / code / audio /
sensor / hidden-channel-detection carriers without re-authoring.
Common corrections across all three tickets:
- Mandatory fixture metadata: carrier, domain, pi_star_ref,
loss_report_refs, modality_notes.
- Unsupported carriers MUST fail or skip explicitly with
reason="unsupported_carrier" — never silently accepted.
- No LLM-as-judge in any runner.
- Hidden-channel work is defensive only (detection / flagging),
never generation or concealment.
Per-ticket headlines:
#000023 — 5S
Syntax / Semantics / Semiotics defined as carrier-general operations
over sign-bearing representations. Semiotics gets the biggest
correction: visual symbols, layout, metadata, encoded sign systems
are valid carriers (Phase 1 still text-only). Synonym source
policy: concept_relations.relation_kind='synonym' only for v1
positives.
#000024 — 5T
Vocabulary alignment with Dav1DPrometheus authoritative wording
(Transfer→Transfer Learning, Truth→Truthtables, Timing→Time).
Transitivity gets a typed-relation whitelist (implies, subset_of,
ancestor_of, before, less_than) — not all edges transitive.
Truthtables capped at N=2..4 to avoid combinatorial blowup. Time
is the first sub-battery where v8 substrate (memory_root #000017
+ selfmodel #000014) becomes a measurable bench target.
#000025 — 5F
New axis. Function/Finetuning/Falsification/Formulate/Feedback
Loop. Folds in the state-space synthesis: SQD + v7 + 5S/5T/5F +
arborist together instantiate a discrete state-space/time
Ω_t = (W, I, C, L, MRoot, SMRoot, PRoot, BRoot, ARoot) with
Ω_{t+1} = T(Ω_t, Δ_t). Counting / mathematics / logic / time
emerge as auditable operations over committed state, not text from
a latent model. adaptation_efficiency and feedback_efficiency
metrics hook into the capital ledger (#000020) so v8 fork choice
has cost-aware fitness signals. Falsification fixtures tagged
with verifier_method_root so verifier shape changes warn rather
than false-fail.
All three tickets remain "open · awaiting go/no-go" — design-only.
Implementation tickets land in follow-up commits when fox approves
the corrected scope.
Source: Legally Unprecedented Dav1DPrometheus (BasementAGI host,
Where The mAGIc Happens). Honoring his framework.
Three new design-only tickets surfacing the gaps between arborist's
current bench harness (ticket #000021 Phase 1a, landed) and
Dav1DPrometheus's authoritative 5S/5F/5T evaluation framework.
- #000023 — 5S Phase 1b: real implementations + fixtures for
Syllogism, Synthesis, Semiotics (currently stubbed).
- #000024 — 5T Phase 1b: rename Transfer→Transfer Learning,
Truth→Truthtables, Timing→Time to honor Dav1DPrometheus's
vocabulary; ship real Triangulation, Truthtables, Transitivity,
Time runners (currently stubbed). Time integrates with
memory_root (#000017) for the first measurable use of v8
substrate as fitness target.
- #000025 — 5F battery: entirely new — Function, Finetuning,
Falsification, Formulate, Feedback Loop. arborist had no 5F
coverage before this ticket; the SQD whitepaper omitted the
axis. Each sub-battery integrates with surfaces already shipped
(selfmodel_records, providence_cache.falsification_state,
memory_branch_summaries).
Source attribution: Legally Unprecedented Dav1DPrometheus
(BasementAGI host). Honoring his framework as the authoritative
taxonomy for non-embodied AGI evaluation.
Next ID bumped 000023 → 000026.
Per fox's "partial punt on larger ones" — ships the bench/ skeleton +
small seed fixture sets so future v8/v7-W/SelfModel work can cite a
real fitness target. Full Phase 1 (50-200 fixtures per sub-battery)
and Phases 2-3 stay open in the ticket.
Phase 1a delivers:
- bench/batteries/{base,b_5s,b_5t,runner}.py — Battery protocol,
BatteryResult, fixture-digest helpers, CLI runner.
- Seed fixtures:
- bench/fixtures/5s/syntax-v1.jsonl — 10 tasks against
wikitext-base@v1 and claim-lattice@v1
- bench/fixtures/5s/semantics-v1.jsonl — 8 equivalence tasks
- bench/fixtures/5t/transfer-v1.jsonl — 4 paraphrase-invariance
tasks
- Runners for 5S Syntax, 5S Semantics, 5T Transfer. Other 5S/5T
sub-batteries are stubs returning zero-task results.
- Makefile targets: bench-5s, bench-5t, bench-5s5t.
- runtime_digest field captures the active π* registry fingerprint
so a registry change surfaces in bench results.
Tests: tests/test_bench_batteries.py (17 cases). Full suite:
1076 passed, 36 skipped. `make bench-5s5t` runs end-to-end and
emits JSON results.
Ticket #000021 status: in progress · Phase 1a landed; Phase 1b/2/3
remain open.
Doc-only landing. docs/spec-methodology.md codifies the discipline
arborist already practices — versioning rule, round-trip discipline,
soundness/completeness honesty, default-value greenfield rule,
sidecar separation — so new π*, V, and policy-field authors don't
re-derive it from audit-chain failures.
Three author-class sections each ship with:
- Five questions the author must answer before landing.
- Worked example drawn from arborist's existing surface.
- One-page checklist.
Worked examples cited:
- π* — wikitext-base@v1
- V — paraphrase strategy
- policy field — quantifier_guard_apply_caps
Cross-references to bench-maxing, seven-point-program, pi-star-
composition, concept-relations-design, and CLAUDE.md.
New arborist.pi_star/ namespace centralizes canonical projections
under a name@version registry. Two existing canonicalizers re-homed
as registered π*'s:
- wikitext-base@v1 wraps arborist.wikitext.to_base
- claim-lattice@v1 wraps arborist.qa.parse_claims.parse_pointer_claims
Four stubs registered for follow-up modality tickets:
code-py-ast@v1, logic-kernel@v1, time-series-quantized@v1,
tabular-pinned@v1 — each raises NotImplementedError with a pointer
to ticket #000015.
Composition algebra in compose.py: PiStarComposition exposes
outer ∘ inner as a first-class π* with its own registry key
(default "<inner-name>-then-<outer-name>@v1"). canonical_composition_id
returns a SHA-256 fingerprint suitable for governance hash inclusion.
Order-sensitive: a∘b ≠ b∘a → different fingerprints.
Documentation: docs/pi-star-composition.md covers the rule (type-
compatible, deterministic, equivalence-class preserving), lossy vs
invertible compositions, worked text→claim-lattice example,
cross-domain anchor projections (future), authoring checklist.
Re-home is non-breaking: arborist.wikitext.to_base remains importable.
Tests: tests/test_pi_star.py (19 cases). Full suite: 1059 passed,
36 skipped.
CapitalProfile (8 forms: living, material, financial, intellectual,
experiential, social, cultural, spiritual) attached per state-changing
op as a sibling-table row in capital_ledger. Sibling semantics: ledger
rows reference an audit_event_hash but do NOT enter the audit-event
preimage, so retroactive cost re-estimation cannot break the chain.
Surface:
- arborist.capital.{profile,store}
- profile_for_op dispatch with per-op estimators (ingest/qa/distill)
- record/summary/op_cost/top_by_form
- CLI: arborist capital summary|op-cost|top
Wire-up at three op sites:
- ingest.py — one row per batch (doc_count + total_bytes)
- qa/runner.py — one row per cache-miss (answer_chars + llm_seconds)
- distill/runner.py — one row per derivation (positive intellectual)
Estimator constants are heuristic v1 (ESTIMATOR_VERSION pin in the
schema). Re-estimation is supported by re-running estimators against
the recorded inputs_blob and writing a new row with a bumped version
pin; old rows stay queryable.
Tests: tests/test_capital.py (13 cases). Sibling-table invariant
verified: audit chain stays intact across capital writes.
Full suite: 1025 passed, 36 skipped.
SelfModel binds an arborist agent's identity to bytes a verifier can
recompute: model_profile_hash, verifier_method_root, governance hash,
canonicalization/chunking versions, optional patch + memory roots,
sorted capability-claim hashes. Hard-hash committed; no soft state in
preimage. State transitions live on the row, not the body, so the
selfmodel_root stays stable across live → stale → falsified.
Surface:
- arborist.selfmodel.{canonical,snapshot,store,falsify}
- CLI: arborist selfmodel snapshot|show|falsify|list
- Schema: selfmodel_records + selfmodel_capability_claims (additive)
- Audit events: selfmodel_snapshot_landed,
selfmodel_capability_claim_added, selfmodel_falsified,
selfmodel_marked_stale (all chain via existing append_audit)
Also folds in:
- CLAUDE.md operational rule: arborist stays Python-only; non-Python
toolchains live in sibling repos. Forks/clients/servers in any
language follow our schemas + canonical encodings.
- Ticket #000016 update: ZK lives in sibling repo arborist-zk-bench;
arborist gains at most a wire-format consumer, never a Rust dep.
- Schema migrations also stub capital_ledger and memory_records
tables for tickets #000020 and #000017 respectively (additive,
empty until those modules land).
Tests: tests/test_selfmodel.py (14 cases; canonical-JSON stability,
root order-invariance, snapshot determinism, store idempotency,
audit events, falsify/mark_stale semantics, audit-chain integrity).
Full suite: 1012 passed, 36 skipped.
Both items surfaced by the 2026-05-07 arborist-vs-donto comparison
report (/home/fox/Downloads/arborist_vs_donto.pdf).
Ticket #000022 — adapter LossReport (PRD I9 analogue). Today wikitext
to_base() and html_page _normalize_text drop <ref> tags, image/category
wikilinks, HTML chrome, whitespace runs without recording any of it;
only the canonicalization-version pin survives. Ticket proposes a
sidecar adapter_loss_reports table (Option A) over Merkle-bound
loss_root (B) or audit-chain entries (C), preserving arborist's
"soft signals are sidecars" discipline. ~1.6% storage tax expected,
matching concept_relations. Doc-only; no code in this commit.
Federation doc-discoverability: PDF author concluded "federation
exists in code but the public docs page returned 404" because the
mesh page lives at /api/mesh.html and the concepts orientation
never surfaces the topic. Adds a "Federation across peers" section
to concepts.rst pointing at api/mesh, a :ref:\`federation\` anchor
+ context lead on api/mesh.rst, and a footer link in concepts.rst's
"Where to go next." Sphinx build passes; api/mesh.html#federation
deep link resolves.
Per shared-things convention: prefer 'our' for community-owned things;
'the' implies fixed singular ownership. The permacomputer is collective
infrastructure, so 'our' fits better than 'the'.
Updated three places (single canonical preamble text):
- LICENSE (Permacomputer Preamble section, our text — not the AGPL)
- README.md License section
- docs/_source/conf.py rst_epilog (per-page footer on RTD)
Previous attempt to move the full project TOC to the right via
html_sidebars={'**': []} stripped the theme's left sidebar — the right
sidebar in sphinx-book-theme renders only page-local TOC, not the
project tree, so the project structure disappeared and the page
looked unthemed.
Restore the default layout:
- Left: full project toctree (themed sphinx-book-theme sidebar)
- Right: 'On this page' (current page sections), expanded to depth 3
If we want a unified right-side project TOC later, that needs
pydata-sphinx-theme + secondary_sidebar_items override, not just
hiding the left.
sphinx_rtd_theme only renders left-side navigation. Switch to
sphinx-book-theme which puts the full project TOC on the right and
leaves the reading area centered.
conf.py:
- html_theme = 'sphinx_book_theme'
- html_sidebars = {'**': []} hides the left sidebar so the right TOC
is the single navigation surface
- show_toc_level=3 expands subpages; show_navbar_depth=2 controls top nav
requirements.txt: sphinx-rtd-theme → sphinx-book-theme>=1.1
Makefile cuts (64 → 58 documented targets):
- ingest-cur-parallel, ingest-old-parallel: parallel-shared mode
superseded by attached (no WAL contention)
- distill-shards: sequential never preferred over parallel variant
- bench-qa-quick: bench-qa-smoke covers same use case (~30s vs ~10s)
- ingest-grok, ingest-grok-media: single-DB grok rare; -attached is
canonical path
All cuts land in code that the underlying CLI still exposes — operators
who need the dropped variant call '.venv/bin/aborist ingest --shard ...'
directly. No behavior loss, just shortcut removal.
Docs improvements:
- New Concepts page (docs/_source/concepts.rst): orientation on what
aborist is, three layers (surface/core/providence), Merkle commitment,
8-dim cache key, audit chain, trichotomy + four-rung ladder, layered
verifier, falsification state, sidecars. Embeds module-graph and
verifier-ladder SVG diagrams.
- New Cookbook page (docs/_source/cookbook.rst): 8 recipes — recrawl,
falsify, ingest-self-providence, mixed-corpus query, LLM endpoint
override, integrity after bulk ops, bench, retrieval tuning.
- Quickstart embeds query-pipeline SVG diagram.
- docs/_source/diagrams symlinks to docs/diagrams so Sphinx can include
the SVGs (was orphaned, only README referenced them).
Better Makefile RTD page (docs/_source/_ext/makefile_targets.py):
- Group by workflow phase (Setup → Fetch → Ingest → Distill → Query →
Verify → Operations → Tests → Docs → Clean) instead of alphabetical
prefix. Tells a new operator the order they'd actually run things.
- Phase descriptions added; targets prefixed with 'make ' for copy-paste.
- Uncategorized leftover surfaces missing entries in PHASES list.
New docs/_source/quickstart.rst — install, two end-to-end paths
(Wikipedia 2003 + crawler), after-the-answer commands, query pipeline
overview, and links to deeper reference pages. Mirrors the README's
quickstart but adapts cross-references to the Sphinx structure.
Added to index.rst as a 'Getting started' toctree section above the
API modules — RTD users land on it first.
Use the canonical Read the Docs theme. Build dropped from 33 warnings
to 3 (most furo warnings were sidebar template lookups in the dark/light
mode switcher).
requirements.txt updated to pull sphinx-rtd-theme instead of furo.
- docs/_source/license.rst — new RTD page that literalincludes the
repo's LICENSE file (single source of truth, no duplication)
- conf.py rst_epilog — appended to every RST source so every doc page
carries the Permacomputer Preamble + AGPL-3.0-only notice + link to
the full license
- index.rst — adds 'Project / License' section to the toctree
The LICENSE file already had the full Permacomputer Preamble + complete
GNU Affero GPL v3 text (matches the whitepaper version).
Adds a Sphinx extension at docs/_source/_ext/makefile_targets.py that
parses the project Makefile's '## description' annotations and writes
docs/_source/api/makefile.rst at build time. Same convention 'make help'
uses, so the reference stays in sync with the source.
Generated page is grouped by target prefix (fetch-, ingest-, distill-,
docs-, etc.) and rendered as a list-table. Shows on RTD alongside the
autodoc API modules.
Generated file is gitignored — RTD regenerates on every build.
Initial docstring pass focused on syntax/style; this pass verified each
docstring against actual function behavior. Found and corrected:
WRONG (claimed behavior didn't match):
- _cmd_verify: claimed Q&A/audit verification — actually round-trips
Merkle proofs on N random documents
- _cmd_snapshot_verify: claimed Merkle proof round-trip — actually
re-derives snapshot root and checks for drift
- _cmd_evict: claimed 'archive unused content' — actually NULLs content
and removes FTS row; cores never evict
- _cmd_stats: listed 'index size' which is not in stats() output
OVERSTATEMENT (claim stronger than contract):
- _cmd_ask: 'grounded answer' — verifier may return UNGROUNDED
- _cmd_snapshot_list: 'named' — snapshots have hash roots, not names
MISSING IMPORTANT BEHAVIOR:
- _cmd_rehydrate: didn't mention drift-detection exit code
- _cmd_mesh_status: didn't mention 'enabled' flag (most important field)
- _cmd_distill: didn't mention recursive core→core distillation
- MerkleTree.proof(): didn't mention IndexError on out-of-range
VAGUE:
- _cmd_search: 'Search the corpus with FTS5' → mention output formats
Sphinx rebuild successful (29 warnings, down from 31).
Implements Read the Docs infrastructure to generate API documentation
directly from code docstrings. Replaces static modules.md (1200+ lines).
New structure:
- docs/_source/conf.py — Sphinx configuration (furo theme)
- docs/_source/index.rst — Main TOC
- docs/_source/api/*.rst — Module groups (substrate, storage, retrieval,
qa, distill, mesh, cli)
- docs/_source/Makefile — Local build targets
- docs/_source/README.md — Documentation on building and extending
Makefile integration:
- make docs-api — generate HTML (output: docs/_source/_build/html/)
- make docs-api-clean — remove build artifacts
Build output (40 HTML files):
- API module reference with docstrings
- Source code links (:viewcode: extension)
- Full-text search index
- Module index (genindex, py-modindex)
Sphinx installed in venv as dev dependency. HTML is browseable at
docs/_source/_build/html/index.html (open in browser after build).
This justifies the deletion of modules.md: code docstrings + Sphinx
autodoc = automatically-generated, always-current API reference.
Delete redundant, superseded, or design-log docs:
- qa-modes-bench-2026-04-30: prior snapshot (rolling journal is current)
- verifier-semantic-gap-design: unimplemented future proposal
- bench-emergent-design: stress-test design (script self-documents)
- modules.md: API reference (code + docstrings are source of truth)
- self-reference-design: v1 shipped, v2 scoped to future; history in git
- concept-relations-design: live system documented in code
- mesh-deploy: runbook for off-by-default system; mesh wire future work
Reduces docs/ from 23 files to 16, keeping north-star (seven-point-program),
architecture (cti, mesh.md), and operational docs (benchmarks, qa-modes-bench,
bench-maxing). All deleted docs recoverable from git history.
Single canonical entry point that ties together the four existing
bench-related docs (qa-modes-bench.md / bench-maxing.md /
bench-emergent-design.md / qa-modes-bench-2026-04-30.md) plus the
make targets, fixtures, and bench-row schema.
Sections:
1. Two harnesses, two purposes
bench/qa_sweep.py — curated regression bench
scripts/bench_emergent.py — random-word stress test
2. Four question fixtures (75 / 28 / 9 / 1 / smoke) with cell
sizes + use cases per fixture
3. Signal floor (5pp / n=3 × 9 = 27 / vLLM c=3-4 saturation)
4. Make targets cheat sheet (bench-qa, bench-emergent,
--policy KEY=VALUE A/B pattern, --resume)
5. Bench-row schema — every field a row carries
(identity / verdict / diagnostics / preflight projection /
quantifier classifier / capacity / directive compliance / time)
6. Where headlines live (qa-modes-bench.md addenda, per-ticket
§12/§13 bench sections)
7. bench-emergent log shape + #000006 rolling-amend pattern
8. How to run a focused A/B (the four-cell pattern from #000008)
9. How to interpret results (STRICT-rate, mean-ratio,
UNGROUNDED-rate, FORMAT_COLLAPSED rate, violation kind
distribution, audit-line tails)
10. Operator commands cheat sheet (--show-preflight,
--apply-quantifier-caps, --reject-broad, --soft-preflight,
Makefile shortcuts)
CLAUDE.md docs index updated to point to benchmarks.md as the
"read first" entry for bench work, and to add bench-emergent-design.md
to the index (was missing).
Bench-emergent stress test ran another 100 cycles under the
post-#000008/9/10/11 substrate. Total accumulated: 300 cycles.
Verdict distribution shift on last 100 vs 134-cycle baseline:
STRICT 5% (7/134) → 0% (0/100) -5pp
HYBRID 22% (29/134) → 16% (16/100) -6pp
UNGROUNDED 73% (98/134) → 84% (84/100) +11pp
Zero false-positive STRICTs across 100 random-word triplets.
The 5pp drop in STRICT-rate isn't a regression — it's the
verifier ladder + new preflight contracts doing their job.
Random-word triplets are genuinely ungrounded for the most
part; the prior 5% STRICT rate included false-positives that
the post-hardening verifier now catches.
Violation profile (last 100 cycles, claim_lattice JSON):
CITATION_MISMATCH: 86 dominant gate
TOO_MANY_EVIDENCE_IDS: 24
SUBJECT_TOKENS_ABSENT: 12 Rule 9 firing on parroting
DEFLECTION_DETECTED: 12
TITLE_MISMATCH: 10
...
metaphor_deflection fires 6/100 — still rare. Item 3
(calibration) is now closer to sample-size threshold (~30
signals across 300 cycles; needs ~50-100 to calibrate).
No new tuning candidates surface. Original three remain at
their resolution states.
Big batch — closes 4 of the 5 deferred items from the prior status
report plus opens & implements a previously-deferred design ticket
(#000011) zero-shot.
#000025 — Metacog test fixture expansion:
bench/qa_questions_metacog_subset.txt grows from 6 → 28 questions
covering edge cases per detector kind: temporal (4 cases),
contradiction (4), false-premise (5), out-of-corpus (3), multi-
trigger (2), well-formed controls (5). Documents two known
detector ceilings: Q11 over-fires on past-tense factoid
("who was the first president"); Q16/Q17/Q19 (Edison/Australia/
NASA-fake) miss false premises that lack a presupposition
pattern match. Fixture now serves as long-term regression suite.
#000026 — --show-preflight full clause render:
build_run_dag() and build_reject_run_dag() gain optional
preflight_payload kwarg. When supplied, the canonical 5-clause
CTI payload (classifier / answer_contract / prompt_contract /
evidence_contract / policy_refs + question_state + node_version)
persists alongside the leaf hash in run_dag_blob.
aborist providence --show-preflight CACHE_KEY now renders the
full payload + verifies the persisted hash matches the
recomputed canonical hash (audit-replay tamper detection).
Legacy rows fall through cleanly: payload_hash_check reports
"unavailable: legacy row predates preflight_payload persistence".
#000027 — Latency profile:
Microbenched preflight: 0.46ms/question (negligible). Single
fresh call breakdown: search 2.4s, llm 2.8s, total 5.4s — the
33-35s in Addendum 3 was vLLM concurrency contention at c=4
(per qa-modes-bench.md saturation note), not substrate
overhead. Added preflight_ms + soft_preflight_ms to timings
dict for explicit confirmation in future cycles.
#000028 — Auto-quality-check sweep revival:
scripts/bench_emergent.py running with EMERGENT_N=100 in
background (PID 125680). Will accumulate cycles into
bench/emergent_log.jsonl for #000006 rolling log re-aggregation.
Async — not blocking on completion.
#000029 — #000011 SOFT_PREFLIGHT_HINT implementation:
aborist/qa/soft_preflight.py — new module. SoftPreflightHint
dataclass + soft_preflight_question() pure function. 9
canonical labels mapping to soft analogues of #000010 hard
detectors plus 2 stub states (SOFT_DISABLED, SOFT_PARSE_FAIL).
Constrained-generation prompt (max_tokens=128, temp=0.0) asks
the model to pick ONE label + one-line rationale.
Fail-closed across every parse path:
- chat_client raises → SOFT_PARSE_FAIL
- response unparseable → SOFT_PARSE_FAIL
- label outside enum → SOFT_PARSE_FAIL
Sidecar enforces SOFT_ prefix at the normalize step so a
model that drops the prefix still gets caught.
Wired into query() between preflight & retrieval. Default
OFF (`soft_preflight_enabled: False`). NOT folded into
_VERIFIER_POLICY_FIELDS — soft hints don't gate cache
identity (#000011 §4). Audit-line tail renders as
"· soft: <label>" (e.g. "· soft: time sensitive") so the
signal is visually distinct from hard tails.
--soft-preflight CLI flag opts in per-call. End-to-end
live-verified on "When did Mr. Burns become Homer's biological
father?" — produces:
EVIDENCE-WARRANTED · via claim_lattice
· false premise · soft: time sensitive
1/1 16.4s
Hard `· false premise` (from #000010 deterministic detector)
composed with soft `· soft: time sensitive` (from #000011
sidecar). The model classified a different shape than the hard
detector — by design; soft hints are independent advisory
signals, not redundant with the hard layer.
25 new tests pin: default-OFF behavior, parse-failure modes,
label normalization (SOFT_ prefix enforced), all 8 actionable
labels round-trip, fail-closed on client exceptions, dataclass
JSON round-trip, rationale-length cap.
Other:
- #000010 §13.3 documents 2/5 metacog-trigger questions return
STRICT despite hard-detector warning — direct empirical
motivation for #000011 design.
- tests/test_dag.py extends with 3 _extract_preflight_hash_*
helper tests (cleaning #000009 §7.2 unfinished state).
- bench/emergent_log.jsonl adds new cycles from background run.
#000011 status: closed. Hard rule (D1) preserved across all
1021 tests (up from 996, +25 new). Soft preflight is purely
advisory; the verifier proof path is unchanged.
Fan-out execution of the deferred-but-not-blocking pile from
prior status reports.
#000009 §7.2 — bench harness preflight_hash field:
- aborist/qa/query.py surfaces `preflight_hash` on result dict
(miss path, reject path, and cache-hit path via new helper
`_extract_preflight_hash_from_blob` that pulls the stage hash
out of persisted run_dag_blob).
- bench/qa_sweep.py adds 12-char preflight_hash prefix to bench
rows. Mirrors cache_key truncation pattern. Operators can
grep / SQL-filter bench JSONL by preflight policy state.
- 3 new tests in tests/test_dag.py for the extract helper.
#000009 §7.2 — `aborist providence --show-preflight CACHE_KEY_PREFIX`:
- New CLI flag pulls the preflight stage payload from a row's
run_dag_blob. Match by 12-char prefix. Renders preflight stage
hash + run-DAG stage list. Operator tool for inspecting which
policy state governed a cached row.
- Live verified on a real cache row (8a212fecb2a9 — current CEO
of OpenAI question, 10-stage CTI shape with preflight at idx 1).
- Legacy rows (predating #000009) report a clean fall-through
message: "run_dag has no preflight stage (predates #000009)".
#000010 §12.6 — reference-frame plumbing into QuestionState:
- Pre-retrieval preflight runs with reference_frames=()
(frame_detection needs source titles, not available yet).
Post-retrieval, query.py re-runs preflight_question() with
the detected frames so the result-dict + run-DAG QuestionState
carry frame-aware logical_statuses (specifically
`reference_frame_ambiguous` when 2+ frames match).
- Live verified on Orwell-style question; logical_statuses now
correctly includes `reference_frame_ambiguous` in the result.
Metacog-trigger bench fixture (#000010 §13.3):
- bench/qa_questions_metacog_subset.txt — 6 questions, one per
detector kind plus a well-formed control.
- Bench artifact 2026-05-04T02-18-42Z. Detector accuracy 6/6
on fixture; 2 of 5 trigger questions return STRICT on lattice
mode despite metacog warning (JSON STRICT on
George-Washington-stop-being-president-of-France false-premise
+ uploaded-contract out-of-corpus questions). Audit-line tails
correctly surface the warnings.
- qa-modes-bench.md Addendum 4 captures the per-question matrix
+ interpretation. #000010 §13.3 cross-references with bench
artifact stamp.
#000011 SOFT_PREFLIGHT_HINT design ticket opened:
- docs/tickets/ticket-000011-soft-preflight-hint-sidecar.md
captures the design proposal per #000010 §18 / source doc.
Implementation deferred — design only.
- Sidecar would add model-assisted preflight as a soft signal
(`SOFT_FALSE_PREMISE_SUSPECTED` etc.) that NEVER enters the
verifier hard path. Strict guardrail: cannot create
PREFLIGHT_OK or PREFLIGHT_BLOCKED without deterministic
support.
- Validated by §13.3 finding: deterministic detectors flag
correctly; corpus-accidental grounding produces 2/5 STRICT
on trigger questions; soft sidecar would add independent
semantic skepticism.
- TICKETS.md index row added; Next ID bumped to 000012.
996 tests passing (3 new for the extract helper).
Cross-doc consistency:
- qa-modes-bench.md Addenda 1+2+3+4 chronological
- #000010 §13.1 (broad subset) + §13.2 (full bench) + §13.3
(metacog trigger subset)
- #000011 design captured but not implemented
Cross-references the 2026-05-03T23-30-12Z full 75-question
regression bench (Addendum 3 in qa-modes-bench.md) into:
- #000010 §13.2 — full-bench validation alongside the §13.1
broad-subset validation. Same verdict: defaults stay on.
Substrate-level wins beyond noise-bounded SR deltas:
pointer FORMAT_COLLAPSED: 0/225 (eliminated globally)
pointer NO_EVIDENCE_POINTER: 13% (down from 33% broad-only)
JSON mean ratio: +3pp
- seven-point-program.md addendum — bench-validation paragraph
after the test-count line. Names the SR deltas + the
substrate-level wins; concrete evidence the post-landing
directive coverage claim survives full-corpus measurement.
No code changes; 993 tests still passing.
Addendum 3 to qa-modes-bench.md. Validates that flipping
quantifier_reminder_enabled=True for lattice modes (per #000010
§12.10 / §13.1) doesn't regress narrow-question performance.
Prior validation (Addendum 2) covered the 9-question broad subset
only. This run sweeps the full 75-question bench/qa_questions.txt
(~10% broad, ~89% narrow), 225 runs per mode, comparing against
the frozen 2026-05-02T15-07Z baseline.
Findings:
Mode | Pre-flip SR | Post-flip SR | Δ
----------+-------------+--------------+--------
quote | 0.54 | 0.52 | -2pp (within 5pp floor)
pointer | 0.20 | 0.21 | +1pp
JSON | 0.42 | 0.44 | +2pp
Mean ratio: -1/+2/+3pp — all within noise band.
Substrate-level wins beyond the headline metrics:
- pointer FORMAT_COLLAPSED: 0/225 across the full sweep.
Reminder eliminates collapse mode globally, not just on broad.
- pointer NO_EVIDENCE_POINTER: 13% (vs 33% on broad-only when
reminder was off). Citation discipline propagates beyond the
rows where the reminder text actually fires — the model's
session attention reinforces.
- JSON mean ratio +3pp consistent with broad-subset finding.
Quote-mode is essentially unchanged because it's mode-gated off
the guard by default.
Latency 33-35s/call this evening (vs 17-19s in prior runs) is
endpoint contention, not a substrate regression — preflight adds
zero LLM calls.
Verdict: no regression. The default flip ships clean across the
full corpus. Defaults stay on. The substrate is strictly more
honest (FORMAT_COLLAPSED → 0, NO_EVIDENCE_POINTER ↓) without
sacrificing throughput on non-broad questions.
Bench artifact: bench/qa_results/2026-05-03T23-30-12Z.{jsonl,md}.
Post-landing validation cell. Same 9-question broad subset as
§12.6 reminder-only baseline; this run flips both
metacognition_enabled and quantifier_reminder_enabled to False so
we can isolate the preflight contribution.
Metric | OFF | ON | Δ
--------------------------+-----------+----------+----------
pointer mean ratio | 0.483 | 0.643 | +16pp
JSON mean ratio | 0.570 | 0.735 | +17pp
JSON UNGROUNDED rate | 7/27 | 1/27 | -22pp
pointer FORMAT_COLLAPSED | 2/27 | 0/27 | -100%
any-mode STRICT-rate | within ±11pp noise (27-sample n=3 floor)
Mean-ratio + UNGROUNDED + FORMAT_COLLAPSED metrics all clear the
5pp signal floor on lattice modes. STRICT-rate moves are within
Hermes nondeterminism. The #000010 default flip is doing what
was claimed.
On this subset none of the metacognition detectors fire (no
temporal / contradiction / false-premise / out-of-corpus shapes),
so the delta effectively isolates the reminder contribution from
#000008. A metacog-trigger subset bench is deferred.
Documentation:
- docs/qa-modes-bench.md Addendum 2 captures the comparison
table + interpretation + verdict.
- docs/tickets/ticket-000010-... §13.1 cross-references with
bench artifact stamp.
Bench artifact: bench/qa_results/2026-05-03T23-06-21Z.{jsonl,md}.
Three Explore agents fanned out in parallel for a docs/ + diagrams/
+ code-comment audit against the shipped state of the three
preflight tickets. This commit lands all the alignment fixes.
Core docs updates:
CLAUDE.md
- dag.py module description: stage counts now read
"7/8 quote · 9/10 CTI · 3 reject" reflecting #000009 preflight
stage + reject-broad early-return shape.
docs/cti-architecture.md §2.2 + §2.3
- §2.3 Merkle-AGI-DAG section rewritten: documents all five DAG
shapes (legacy 7/9, post-#000009 8/10, reject-broad 3),
describes the preflight stage's 5 nested CTI clauses
(classifier / answer_contract / prompt_contract /
evidence_contract / policy_refs), pins
PREFLIGHT_NODE_VERSION = "preflight-node-v1", states the
audit-replay payoff.
- §2.2 CTI section: adds the four new modules
(quantifier, model_profiles, quantifier_reminder,
metacognition) as code anchors. Notes that pre-answer
preflight contract extends CTI upstream of retrieval.
docs/seven-point-program.md
- D3 status ½ → ¾ — pre-answer preflight contract landed via
#000008 + #000010. Code anchors + pinning tests updated.
- D4 status ½ → ¾ — preflight stage adds upstream control
commitment to the run-DAG. Code anchors include
build_reject_run_dag + preflight_node_hash.
- Status snapshot table: tickets column now references
#000008/#000009/#000010 against D1/D3/D4 directives.
- "Post-landing addendum (2026-05-03 / 2026-05-04)" subsection
summarises all three tickets + their commit shas + final
test count (993 passing, up from 734).
docs/modules.md
- Q&A pipeline table: added 4 new modules (quantifier.py,
model_profiles.py, quantifier_reminder.py, metacognition.py).
dag.py row updated to "7/8 quote · 9/10 CTI · 3 reject".
- dag.py subsection rewritten: documents all 5 DAG shapes,
describes the preflight payload's 5 clauses + question_state.
- 4 new module subsections (quantifier / model_profiles /
quantifier_reminder / metacognition) explaining each
module's purpose, signature, and how it feeds the run-DAG
preflight clause.
Diagram updates:
docs/diagrams/query-pipeline.dot + .svg
- New "PREFLIGHT (#000008 + #000010)" node inserted between
cache_check and concepts_lookup.
- New "REJECT-BROAD" node showing the 3-stage minimal DAG
escape path.
- render node label extended with the audit-line tail token
catalog.
docs/diagrams/aborist-modules.dot + .svg
- 4 new qa_* nodes in the retrieval & verifier cluster.
- 8 new edges: qa_query/qa_runner each call into all 4
preflight modules; qa_dag has dotted edges to qa_quantifier
+ qa_metacognition (preflight clause sources).
- qa_dag label updated to mention preflight_node_hash + 5 clauses.
docs/diagrams/verifier-ladder.dot + .svg
- Soft-demote violations list extended: BROAD_QUANTIFIER_RUNAWAY
/ CAP_APPLIED / SCOPE_UNBOUND, FORMAT_COLLAPSED, BARE_NAME_CLAIM.
- New "AUDIT-LINE TAILS" annotation node listing all 11 tail
tokens (#000008 broad-* + #000010 metacog + classic verifier).
- Dashed edges from each rung to tails note showing tails
compose onto labels.
Code-side stale-comment fixes (caught by 3rd Explore agent):
aborist/qa/keys.py:218
- "The four fields" → "The seven fields"; mention #000010 adds
six more for metacognition.
aborist/qa/query.py:2644
- 7-stage / 9-stage comment expanded to enumerate all four
base+preflight shapes plus the 3-stage reject path.
aborist/qa/runner.py:835
- same expansion as query.py for runner.ask() callsite.
mesh-*.dot, ingest-pipeline.dot, qa-modes-bench.md, bench-maxing.md,
bench-emergent-design.md, verifier-semantic-gap-design.md,
self-reference-design.md, concept-relations-design.md confirmed
orthogonal — no edits needed.
993 tests still passing (no behavior change). 7 files modified
across docs/ + 3 dot diagrams + 3 SVGs + 4 code-comment fixes.
Architectural feedback at ~/Downloads/RESPONSE-ticket-000009-... .txt
(2026-05-04) flagged five gaps in the c36e85c landing. Most
critical: reject-broad early-return path emitted no run_dag_blob,
so audit replay couldn't see that a rejection happened (let alone
under what policy state).
A — reject-path DAG (the critical gap):
aborist/qa/dag.py: build_reject_run_dag() — 3-stage minimal DAG
question → preflight → final_label. final_label payload carries
rejection_reason + answer_text_hash so two rejections under
different policy state produce different roots.
query.py reject path now wires it: returns run_dag_root +
run_dag_blob on the rejection result dict. Live-verified end-
to-end on `make query Q="winners of all major sports?"
REJECT_BROAD=1 BURN=1`.
Audit replay rule: 3 stages always means reject path. Operators
can read the stage list and tell instantly without parsing the
payload.
B — nested CTI clauses:
preflight_node_hash() payload restructured from flat 3-key to
nested 5-clause:
classifier — quantifier classifier output (#000008)
answer_contract — guard / cap / reject / metacog state (per-run)
prompt_contract — reminder enabled / injected / template_id
evidence_contract — exposure budget, line discipline
policy_refs — governance_policy_hash, model_profile_hash,
answer_mode (reference, not raw policy)
Plus question_state (metacog) as its own clause and top-level
stage + node_version. Single DAG stage; nested clauses inside
for diff legibility (feedback §3).
C — node_version field:
PREFLIGHT_NODE_VERSION = "preflight-node-v1" pinned in the
payload so legacy runs without the node can be unambiguously
labeled `unavailable_legacy_run` by audit tools (feedback §9).
D — reference hashes only:
policy_refs uses governance_policy_hash + model_profile_hash
rather than bundling raw policy booleans. Avoids
double-committing already-hashed state (feedback §4).
E — reminder_template_id:
prompt_contract.reminder_template_id = "broad-quantifier-bounded-v1"
or "broad-quantifier-unbounded-v1" depending on scope_bound_hint,
populated only when reminder actually fires.
F — stage name kept as `preflight` (not `quantifier_preflight`):
Node carries both #000008 quantifier AND #000010 metacognition
payloads. node_version disambiguates schema for audit tools.
G — docs/cti-architecture.md update deferred to a small follow-up.
Bug fixes:
- free-variable shadowing on verifier_policy_hash /
model_profile_hash / question_hash — local re-imports inside
the reject branch shadowed module-top imports used elsewhere
in query() / runner(); now use the module-top names.
- reject path question_hash signature: takes `mode=` not
`dedup_mode=` — fixed in the reject DAG builder caller.
Hash compatibility:
Rows written between c36e85c and this commit have hash payloads
matching the OLD flat 3-key shape. The persisted run_dag_blob
captures the actual payload that was hashed, so those rows
still verify via verify_run_dag(). New rows use the nested
5-clause shape.
7 new tests in tests/test_dag.py:
- hash sensitivity to answer_contract / prompt_contract /
policy_refs flips (audit-replay payoff demonstrations)
- PREFLIGHT_NODE_VERSION pinning
- reject DAG: 3-stage shape, root changes with preflight hash,
round-trips through verify_run_dag
993 tests passing (6 net new); 36 skipped.
Live verification:
make query Q="winners of all major sports?" REJECT_BROAD=1 BURN=1
→ status=broad_quantifier_rejected, run_dag_root populated,
blob carries 3-stage shape.
make query Q="winners of all major sports?" BURN=1
→ 10-stage shape preserved (question → preflight → retrieval
→ ... → final_label).
Ticket #000009 status: closed · re-landed 2026-05-04 with §8
corrections.
Closes ticket #000009 zero-shot. Scope expanded to cover BOTH
ticket #000008 (broad-quantifier) AND ticket #000010
(meta-cognition) preflight contracts in a single combined node —
both share the same audit-replay gap and inserting two separate
nodes between question and retrieval was operationally awkward.
aborist/qa/dag.py:
+ preflight_node_hash() — combines QuestionState +
quantifier classifier output + behavioral policy_state into
one canonical SHA-256 hex.
+ build_run_dag() gains optional preflight_hash parameter.
When supplied, inserts {"stage": "preflight", "hash": ...}
at position 1 (between question and retrieval).
Backward-compat: None → original 7/9-stage shapes preserved
for legacy run_dag_root re-validation.
Quote-mode: 7 → 8 stages with preflight.
Pointer-mode CTI: 9 → 10 stages with preflight.
aborist/qa/query.py + runner.py:
Both build the preflight payload from question_state +
quantifier dict + 10-field policy_state (guard_enabled,
guard_apply_caps, guard_apply_caps_mode_gated,
claim_cap_resolved, claim_cap_actually_applied,
reminder_enabled, reminder_eligible, reject_broad_active,
metacognition_enabled, block_on_contradiction).
This means two cache rows that share the same question + same
model output + same verifier verdict but DIFFERENT preflight
policy state now produce different run_dag_root values. Audit
replay can pin the policy decision per row.
9 new tests in tests/test_dag.py:
- preflight_node_hash determinism
- hash bumps on question_state change
- hash bumps on policy_state change (the audit-replay payoff)
- all-None defensive shape
- 7→8 stage transition (quote mode)
- 9→10 stage transition (pointer mode), preflight at index 1
- run_dag_root bumps when preflight_hash bumps
- verify_run_dag round-trips through preflight stage
Live verification: latest providence_cache row carries
['question', 'preflight', 'retrieval', 'evidence_map', 'prompt',
'raw_answer', 'parsed_claim_lattice', 'verify', 'render',
'final_label'] — preflight stage living in the persisted DAG.
987 tests passing (9 new); 36 skipped.
Tickets:
#000009 status: closed · landed 2026-05-03 (zero-shot)
#000010 cross-ref updated: "DAG binding shipped via #000009"
What's NOT in this ticket (logged in §7.2):
- CLI flag for inspecting preflight node from cache_key
- Bench harness preflight_hash field for cross-row comparison
- SOFT_PREFLIGHT_HINT (model-assisted preflight sidecar)
Closes ticket #000010 (Meta-Cognition Preflight Guard). Mechanism
complete; defaults preserve the dry-run discipline pattern from
#000008.
Phase 2 — wire preflight into query() and runner.ask():
- preflight_question() runs after policy resolution + quantifier
classification, before retrieval.
- QuestionState surfaces on miss path, cache-hit path, AND
reject-broad early-return path of query() — schema column-
aligned across all four returns.
- runner.ask() carries the same fields for `aborist ask` parity.
Phase 3 — policy fields + governance hash + CLI flags:
- 6 new policy fields, all default-on except
metacognition_block_on_contradiction (default False — label-
only by default; opt-in via --block-on-contradiction).
- All 6 folded into _VERIFIER_POLICY_FIELDS so flipping any
invalidates prior cache records on lookup.
- 2 new CLI flags on `aborist query`:
--no-preflight Level 2 master kill
--block-on-contradiction strict mode (hard-block on
lexical contradictions)
Phase 4 — audit-line labels + bench fields + tests:
- _render_warrant_tail extended with 5 metacog tail tokens:
· false premise
· contradictory
· stale risk
· out of corpus
· frame ambiguous
- Bench rows in qa_sweep.py gain 7 new bounded-size projection
fields (logical_statuses, question_shape, preflight_result,
temporal_sensitivity, has_false_premise, has_contradiction,
corpus_requirement). Full QuestionState stays on result dict
for CLI render only.
- tests/test_metacognition.py grew from 42 → 68 tests
(16 new: 6 governance + 6 audit-line tail + 4 default-policy
pinning).
Live verified end-to-end:
$ make query-dry Q="Who is the current CEO of OpenAI?" BURN=1
UNGROUNDED · via claim_lattice · stale risk
$ make query-dry Q="When did Mr. Burns become Homer's biological
father?" BURN=1
UNGROUNDED · via claim_lattice · false premise
978 tests passing; 36 skipped.
What's NOT shipped (deferred):
- Run-DAG node binding for metacognition_preflight stage —
joins ticket #000009 Phase 5 (same audit-replay gap; both
nodes can land together).
- Reference-frame plumbing — frame_detection runs post-retrieval,
preflight here is pre-retrieval; deferred until two-pass
or post-classification update lands.
- SOFT_PREFLIGHT_HINT (model-assisted sidecar) — source doc §18
reserves this label; hard rule preserved (no LLM in preflight
hard path).
- Bench A/B measuring preflight on vs off — quick to run once
stack settles.
Ticket #000010 status: closed · landed 2026-05-03.
