russell@unturf.com
7f29ee91e6
Surfaced while sizing #000061's federation story. Today shard count conflates two roles: - producer ingest parallelism (wants = vCPU count) - consumer ATTACH fan-out (capped at SQLITE_MAX_ATTACHED=10) On default Python sqlite3, the ATTACH ceiling is 10 and can't be raised without a custom sqlite3 build (which violates CLAUDE.md's "python3 + venv + sqlite3 is enough" property). Producer with 16 shards → consumer fails to attach the 11th, federation silently breaks. Design: introduce M = canonical shard count (corpus-wide constant, default 8) decoupled from N = ingest workers. Document → shard assignment becomes content-deterministic: shard_idx = int(document_root[:8], 16) % M Same input → same output across every peer. Today's "spray by ingest order" is non-deterministic across peers — two peers re-ingesting the same corpus put the same document_root in different shards. That weakens federation more than it should. Migration hard-constraint (fox: "this implies we will need to reprocess all our data into shards"): re-ingest required. Current layout is sprayed by ingest order; post-ticket is sprayed by content hash. Two layouts are incompatible by construction. Captured in ticket §Migration as the load-bearing operational note. Phases laid out (0-4: design lock → read path → ingest path → pack-restore → corpus migration tool). Open audit-chain re-numbering question (per-shard event_hash chains break when rows rebalance across shards). Kept as one ticket — do-not-proliferate. Scaffold only. No code yet — design lock first. Next ID bumped to 000066.
182 lines
59 KiB
Markdown
182 lines
59 KiB
Markdown
# Arborist tickets — index
|
||
|
||
Arborist uses a flat `ticket-NNNNNN-<slug>.md` convention for actionable
|
||
proposals: defects with a known root cause, design proposals awaiting
|
||
go/no-go, or scoped enhancements queued for future implementation.
|
||
|
||
## Convention
|
||
|
||
- **Filename:** `docs/tickets/ticket-NNNNNN-<slug>.md`. The ID is a six-digit
|
||
zero-padded integer; the slug is hyphen-lowercase and short enough
|
||
to stay readable on a 100-column terminal.
|
||
- **Numbering:** monotonic, never reused. Pick the next free integer
|
||
greater than every existing ticket. The current next ID is on the
|
||
`Next ID` line below; bump it when you open a ticket and commit
|
||
the bump in the same commit as the new ticket file.
|
||
- **Header:** every ticket starts with
|
||
```
|
||
# Ticket #NNNNNN — <short title>
|
||
|
||
**Status:** open · awaiting go/no-go (or "in progress", "closed")
|
||
**Opened:** YYYY-MM-DD
|
||
**Scope:** <one-liner — what this ticket is for>
|
||
**Audience:** <who reads this>
|
||
**Hard constraint:** <invariants this ticket commits to NOT break>
|
||
```
|
||
- **Body:** problem statement, design choices with trade-offs,
|
||
recommendation, implementation sketch, scope boundaries, status.
|
||
See `ticket-000001-retrieval-keywords-audit-gap.md` for a worked
|
||
template.
|
||
- **Closure:** when a ticket lands, flip Status to `closed · landed in
|
||
commit <sha>` and leave the file in place. Do not delete tickets;
|
||
they are the design log.
|
||
|
||
## Distinction from other docs
|
||
|
||
`docs/` also contains architecture references and bench journey notes
|
||
that are NOT tickets — they describe state of the world rather than
|
||
proposing change. Examples that stay un-numbered:
|
||
|
||
- `seven-point-program.md` (north-star architectural directive
|
||
distilled 2026-05-01; every new ticket / feature / prompt edit
|
||
walks past this; bench harness reports per-mode directive
|
||
coverage)
|
||
- `cti-architecture.md` (architecture reference for the Clause
|
||
Tree Intelligence claim-lattice-pointer mode)
|
||
- `mesh.md` (mesh wire format + group-key state machine)
|
||
- `benchmarks.md` (bench harness orientation — fixtures, signal
|
||
floor, make targets, bench-row schema, addenda index; read
|
||
first when running a bench)
|
||
- `bench-maxing.md` (bench discipline — 5pp signal floor, n=3
|
||
default, avoid negation in prompts, honest verdicts over
|
||
optimistic ones)
|
||
- `qa-modes-bench.md` (rolling bench-journal addenda with
|
||
cross-references to per-ticket bench data)
|
||
- `soft-hash-channel-analysis.md` (analysis doc closing #000018
|
||
open questions §9.1 / §9.2 / §9.3; companion to
|
||
`soft-hash-channel-t3-bound.md`)
|
||
- `soft-hash-channel-t3-bound.md` (#000036 formal derivation of
|
||
the per-window covert-channel budget; companion to the analysis
|
||
doc, both feed the t3_bound_calculator)
|
||
- `tool-action-dag-design.md` (research path for an action-
|
||
provenance layer downstream of `final_label`. Captured pre-ticket
|
||
per the 2026-05-07 arborist-vs-donto comparison; promotes to a
|
||
ticket when an actual agent use case surfaces)
|
||
- `warrant-substrate-cookbook.md` (architecture reference for the
|
||
18 open-licensed textbook substrates backing every claim-pack
|
||
record, written 2026-05-10 after the 18/92 → 92/92 push under
|
||
`#000031`. Per-pillar substrate map, ingest patterns, alias
|
||
discipline, cascade tuning, honest tier breakdown.)
|
||
- `calculator-test-patterns.md` (architecture reference for
|
||
testing calculator / probe-style code. Twelve patterns split
|
||
into two tiers: the original 9-item core (KAT regression,
|
||
hand-computed formula assertions, monotonicity, closure /
|
||
sum-of-parts invariants, parametrized invalid-input cones, CLI
|
||
subprocess, doc parity, module-export shape) plus three
|
||
domain-specific contract pins (projective-contract for one-way
|
||
canonicalizers, dispatch-order pin for Python type-hierarchy
|
||
gotchas, tie-breaking-rule pin for banker's rounding).
|
||
Codified 2026-05-10 from the original three-module pattern
|
||
bench across `t3_bound_calculator` / `phi_alignment_probe` /
|
||
`anchor_prg`; extended same day with the four π* kernel
|
||
exemplars (`pi_star_arithmetic` / `_logic` / `_code` /
|
||
`_time_series`). Explicit "doesn't apply to verifier-style
|
||
code" caveat.)
|
||
- `lexical-first-rationale.md` (positioning/architecture reference
|
||
for why lexical-first FTS5 + content-addressed Merkle proofs is
|
||
arborist's *default* retrieval and dense-vector semantic search is
|
||
an opt-in additive layer: arborist's per-document ingest is
|
||
~10-100× cheaper than building a vector representation — same
|
||
SQLite substrate, different philosophy — which is the difference
|
||
between "ingest + search runs on a phone" and an NPU hand-warmer.
|
||
Generalizes the "soft hash vs hard hash" discipline; the numbers
|
||
come from #000039 §14.6. Written 2026-05-12.)
|
||
- `relevance-and-veto-synthesis-for-dav1d.md` (2026-05-13 dav1d
|
||
handoff: synthesis of #000049 [NLI recombination veto] + #000052
|
||
§3.1 [coherence sidecar] + §3.2 [relevance reranker] into one
|
||
decision-ready brief. Three orthogonal sidecars / vetoes
|
||
covering non-overlapping failure shapes; recommended runtime
|
||
operating points; per-step bench-maxing receipts; the
|
||
eight-instance meta-lesson "clean candidate-bench mis-predicts in
|
||
both directions, real-data on both axes is the only load-bearing
|
||
measurement". Awaiting fox+dav1d sign-off on the three runtime-
|
||
promotion decisions.)
|
||
|
||
If a doc proposes change AND awaits a decision AND has scoped
|
||
implementation cost, it's a ticket. Otherwise it's reference.
|
||
|
||
## Index
|
||
|
||
Newest first. Update on every open/close.
|
||
|
||
| ID | Title | Status | Opened | Directive |
|
||
|----------|------------------------------------------------|-----------------------|------------|-----------|
|
||
| #000065 | Canonical shard count `M` + content-hash routing (decouple ingest parallelism from ATTACH ceiling) | **open · scaffold + design · awaiting go/no-go** (2026-05-26; surfaced while sizing #000061's federation story). Today shard count conflates two roles: producer ingest parallelism (wants vCPU count) + consumer ATTACH fan-out (capped at SQLITE_MAX_ATTACHED=10 on stock python3 sqlite3). Producer with 16 vCPU → 16 shards → consumers fail to attach the 11th. Producer with 4 shards → 16-vCPU box runs 75% idle on ingest. Fix: pin a corpus-wide canonical M (default 8, ≤ ATTACH ceiling), introduce N (ingest workers) decoupled from M. Document → shard assignment becomes content-deterministic: `shard_idx = int(document_root[:8], 16) % M`. Same input → same output across every peer (today's "spray by ingest order" is non-deterministic across peers, a real federation weakness). Migration hard-constraint per fox: **requires re-ingest of every shard** — current layout is sprayed by ingest order, post-ticket is sprayed by content hash, the two are incompatible by construction. Phases: 0 design lock + pin M in meta table → 1 read path (connect_query honors M) → 2 ingest path (multi-shard write per worker) → 3 cold-pack restore re-routes on pull → 4 corpus migration tool. Open audit-chain re-numbering question (every shard has its own seq + event_hash; rebalancing splits a producer's chain across M consumer shards). Don't proliferate sub-tickets; the audit handling is part of this design lock. Out of scope: custom-built sqlite3 with higher MAX_ATTACHED (rejected: violates "python3 + venv + sqlite3 only" property from CLAUDE.md); topic-clustering shards (would break ingest determinism). | 2026-05-26 | — |
|
||
| #000064 | Cold-object operations toolkit (verify/diff/doctor/repair-fts/gc-plan + audit taxonomy) | **scaffold-only · awaiting go/no-go** (2026-05-26; from Dav1d #000061 review §11/§12/§14). Operator-facing observability + repair tools on top of #000061: `cold verify` (sample/full integrity check), `cold diff` (local vs remote manifest), `cold doctor` (one-shot health: connectivity / credentials / manifest age / missing-object count / tamper sample / audit-chain integrity), `cold repair-fts` (rebuild FTS5 from chunks.content), `cold gc-plan` (orphan bucket objects, read-only by default — destructive only with `--apply` + confirm). Plus expanded audit-event taxonomy: per-PUT/HEAD/GET success/failure events, manifest-pointer events, verify/doctor/gc events. All read-mostly; destructive ops require `--apply`. Bundled so the audit-taxonomy gets one design pass instead of five-way drift. Sequence: doctor → verify → diff → repair-fts → gc-plan. No code until #000061 closes. | 2026-05-26 | — |
|
||
| #000063 | Cold-object private-ciphertext mode (mesh-keyed object keys) | **scaffold-only · awaiting go/no-go** (2026-05-26; from Dav1d #000061 review §9 / response A §13.3). Adds private mode to #000061 cold-object format so chunk bodies + manifest can be uploaded to public-read bucket without leaking corpus membership. Two strategies: (A) deterministic `object_key = HMAC(group_key, leaf_hash)` + AEAD-encrypted body — supports lookup-by-leaf-hash given the key; (B) random-key ciphertext + encrypted private manifest — stronger membership hiding, needs manifest fetch first. Strategy A default; B opt-in. Group key from existing `arborist/mesh/crypto.py`; pack manifest carries `epoch_id` for rotation. Verifier path unchanged: consumer decrypts, then `hash_leaf(plaintext) == leaf_hash` as in public mode. No code until (1) a real non-public corpus needs cold-object shipping, (2) mesh group-key ABI is stable enough to reference, (3) threat-model split between A vs B is settled by real adversary. | 2026-05-26 | — |
|
||
| #000062 | Mechanistic Witness: governed diagnostic sidecar (CNA/SAE/Neuronpedia) | **scaffold-only · awaiting go/no-go** (2026-05-26; Dav1d de-novo review §4.7 / §9.1.F). Specification of a mechanistic-interpretability sidecar that produces a content-addressed `MechanisticWitnessRoot` over (model, prompts, capture policy, neurons/features, intervention deltas), used as a **diagnostic input** to SelfModel (#000014/#000017) + benchmark-fixture generation. **Hard constraint:** soft signals never enter the hard proof path — `audit_mode` does NOT move based on witness output, `providence_cache` is untouched, `governance_policy_hash` only moves via explicit ForkScore ACCEPT with M+C+X axes passing (#000060 §7). Four guardrails (diagnostic-only by default · sandbox intervention only · no production steering without governance · feature labels never semantic proof). Witness root TLV-encodes `model_config_root | activation_capture_policy_root | contrastive_prompt_set_root | feature_or_neuron_set_root | intervention_result_root | behavioral_delta_root | safety_policy_root`. Scaffold only — no code until a real falsifier-in-hand use case exists + the four guardrails are restated in CLAUDE.md as rules + #000060 H-ABCDEFG-M+C+X harness exists to gate promotion. Captured to keep mechanistic-interp tooling out of the substrate unless and until it earns its place; the dual-use risk (Pan et al. 2025 CNA: 0.1% MLP ablation breaks refusal in 72B models) makes the governance-first framing load-bearing. | 2026-05-26 | — |
|
||
| #000061 | Cold-pack distribution tier (boto3 S3-compat, DO Spaces + DVD-R targets) | **in progress** — opened 2026-05-25 (fox: "implement it now … target digital ocean first as a test"; later "I wanted a way to hydrate using tarballs (the core and important data) for bringing new machines up"). Tarball-only distribution mechanism — bucket holds `tar.zst` packs keyed by `hash_leaf(manifest)`, no individual-chunk blobs. New peers hydrate by downloading packs from the bucket's CDN edge (~4 HTTPS GETs for the current ~14.1M-chunk corpus, packs filled to 4.4 GB compressed each via streaming zstd, vs ~14M for individual blobs). Same artifact ≤4.4 GB safe-fit (~6.5 % buffer below DVD-R's 4.7 GB marketing capacity, accommodating ISO9660 overhead + media variance + drive-edge refusal) burns directly to physical media via `--local-dir` + `growisofs`. Packs are *delayed* snapshots: each pack pins the corpus `snapshot_root` it covers in audit + body, so falsifications between repacks produce new pack_hashes and stale packs stay in the bucket until explicit GC (future ticket). Packs include cores AND surfaces (full-corpus hydration). Default selection covers every hot chunk with local content in the shard. Multi-pack splitting via `stream_packs` (streaming zstd, FLUSH_BLOCK peek of compressed buffer after each chunk, cut at cap) fills each disc to ~4.4 GB compressed instead of leaving ~50% empty. One backend class (`S3CompatibleBackend` via boto3 + `endpoint_url`) covers AWS S3, DO Spaces, GCS S3-interop, R2, B2, MinIO. CDN public-read makes packs accessible to anyone; hash binding via in-tar `leaf_hash` member names makes hostile-bucket scenarios safe. Optional dep `[object-store]` = boto3>=1.34. Voyeur: credentials via standard `AWS_ACCESS_KEY_ID`/`_SECRET_ACCESS_KEY` (env or `~/.aws/credentials`), never printed; only endpoint URL + bucket name surface in logs. Initial individual-blob path (per-chunk S3 objects) was scoped+landed then **deleted same day** (fox: "what ever was blobs? I wanted a way to hydrate using tarballs"); the five-step deletion record lives inline in the doc — we'd added 14M-object storage and ~$70/hydrate request cost for a workflow that needed neither. Sizing math for current shards: ~4 packs total (17.2 GB compressed ÷ 4.4 GB compressed per pack), ~17 GB bucket storage, ~$0.34/mo DO Spaces. | 2026-05-25 | — |
|
||
| #000060 | H-ABCDEFG same-model substrate-delta harness (+ jaggedness tensor + curvature) | open · awaiting go/no-go (2026-05-20; from Dav1dPrometheus *Protocol-Layer AGI* working report §26/§50/§82-84). The report's "decisive proof": run the SAME base model substrate-OFF vs substrate-ON over long-horizon/adversarial/non-jagged batteries, report the delta. Two new metrics: jaggedness tensor `J_norm` (§73 — variance across nearby variants, normalized by difficulty) + discrete performance curvature `κ_t` (§5.2, with the honest no-global-convexity bound, Erratum 5). A-vs-C spine (B optional, D=mesh OUT → #000012/#000016). Curvature-aware ForkScore extension folds into **#000012** (NOT a new ticket — reserved `iota`/`kappa` weight slots already exist). Budget: control arms = Hermes/Qwen, never Opus without go; heavy passes on GPU box. Held-out/mechanism-agnostic variants required so ABCDEFG doesn't self-validate. **2026-05-26 scope refinement (Dav1d review §4 / §7):** split the harness output into three axes — **H-ABCDEFG-M** (mechanism tests: does the substrate work?), **H-ABCDEFG-C** (capability tests: does the substrate improve task performance?), **H-ABCDEFG-X** (external adversarial: does it generalize outside author-designed fixtures?). The split prevents "self-validating benchmark theology" — a harness that reports only M+C with no X can pass while still failing on held-out adversarial generalization. Fold into the harness design before any code lands; doesn't change the bench-row schema (`carrier`/`domain`/`pi_star_ref`) but does change what "ACCEPT" requires (must clear all three axes). | 2026-05-20 | — |
|
||
| #000059 | Admission discipline: claim-graveyard query + self-providence quarantine | open · awaiting go/no-go (2026-05-20; Dav1dPrometheus report §11 Priority 2 + §57 "admissible state transition" thesis). Two coupled write-path mechanisms. **(A) GraveyardCheck:** storage already exists (`falsification_state` failed/stale/quarantined records ARE the graveyard); the gap is burden-shifting — a re-asked claim family with a known `failed` history should require stronger evidence (§42.9/§62-63). **(B) Self-providence quarantine:** `make ingest-self-providence` (Makefile:769) deliberately promotes STRICT records into the corpus — the exact self-confirmation loop §70 warns of — and ships with NO guard; detect/lineage-tag self-providence-descended evidence + quarantine for high-impact claims. Both advisory-sidecar-first (run-DAG only, never `providence_cache`/`audit_events`); demote-hooks bench-gated + unwired pending net win (would fold `governance_policy_hash`). Soft signals never enter the hard proof path. **Bounded-ingestion hard constraint (fox 2026-05-20, §7):** the graveyard MUST reach a steady-state size ∝ the *recurring*-error surface, never queries-ever — earn-to-enter (recurrence-gated), fingerprints not transcripts (UTXO-set analogy), decay/compact (evicts like a surface), off the hot path. BTC's lesson is bounded self-regulating ingestion, not "store everything." Gossip-group falsifier admission inherits difficulty-adjusted stable-rate + per-window budget (#000036) → enforced in #000012/`mesh/`. If it can't be bounded, it isn't built. | 2026-05-20 | — |
|
||
| #000058 | `cache_key_9` verifier-policy: mandatory-vs-legible decision + doc reconciliation | open · awaiting go/no-go (2026-05-20; Dav1dPrometheus report §2 Erratum 1 / §11 Priority 1 "mandatory cache_key_9"). **Five-step #1 correction:** the report's *correctness* premise is already false in arborist — verifier fields are a subset of the policy dict and so already fold into `governance_policy_hash` (`keys.py:269-275`); a verifier-rule change ALREADY changes the cache_key today. The explicit 9th `verifier_policy_hash` buys **audit legibility**, not correctness — so "mandatory" would stale every prior record for zero correctness gain. Decision: A leave-as-is (8-dim default, 9th optional) + doc reconcile [recommended] · B default-write 9-dim · C flag-staged bench-gated default-write — never a hard mandatory flip. Doc reconcile (CLAUDE.md/concepts "8-dim" → "8 + optional legible 9th") is the do-regardless. | 2026-05-20 | — |
|
||
| #000057 | Witness-preserving vs ordinary recursive loop: minimal deterministic drift A/B | **open · awaiting go/no-go · doc-only scaffold** (2026-05-19; fox relaying a Hamming-framed GPT-5.5 de-novo review). The review's one ticket-worthy nugget: prove the whitepaper's headline claim — *untracked evidence loss → unbounded recursive drift; witness-preserving state bounds it (detectable+reversible)*. Everything else in the review (rename, corpus hierarchy, IQ, ToE/Riemann/identity) **deliberately NOT ticketed** — narrative, not arborist engineering; don't-proliferate. Hard discipline encoded from the 2026-05-18 precedent: **instrument before experiment** (deterministic, no-LLM-judge, ground-truth-carrying, noise-resolvable — the `recall_at_k` discipline), **one task not eight** (recursive stale-source-invalidation *or* contradiction-repair — the falsification-state-exercising ones), minimal A/B (witness-binding ON vs OFF, N iterations, deterministic surviving-unsupported-claim count), non-claims pinned (necessary substrate, NOT AGI). **Sharpened by fox 2026-05-19 (§4b/§5):** the review's strongest point — every measurement so far is *within* Arborist, no control arm. v1 (smallest proof) = single-shot **Hermes-solo vs Arborist** on a fabrication-bait fixture; headline = fabrication-vs-honest-abstention delta (NOT raw accuracy — common-knowledge would falsely show null via parametric memory). **Judge correction (fox):** "no LLM-as-judge" is a *proof-path* rule (LLM judge inside Arborist's verifier = the circularity it kills); the experiment's judge is **external science** outside *both* arms → use a **strong SOTA model**, blinded + reference-grounded + transcript-logged. That lifts the atomic-only limit (judge handles essay grounding too — #000049's exclusion was right for the proof path, wrong for external science); deterministic gold-key demoted to a cheap atomic cross-check. Recursive-drift loop = v2, gated on v1. Still open (fox calls): fixture population, **which SOTA judge** (API cost + Voyeur egress note), spend. Scaffold; no build past gate-1. One ticket, not ten. | 2026-05-19 | — |
|
||
| #000056 | Operation Sandwich — cross-language grounding via query+display MT | **implemented & landed 2026-05-17 · default-OFF** (fox: "call it operation sandwich, create a new ticket and finish it"). Mechanism + bright line **verified live end-to-end** (real Hermes + real opus-mt: es query → English answer+verifier → es display; `answer_text` English, `display_answer` Spanish additive, `question_hash`/`verifier_policy_hash` invariant; 6 tests + full suite 2477 passed, 0 regressions). **Fan-out measured (§9, n=1, 75 q):** EN baseline 85% → es+sandwich 71% = **−14pp cost**; transitions PRESERVED 31 / DOWNGRADE 16 / LOST 17 / N/A 11. A deterministic round-trip predictor was tried and **refuted** (12/17 LOST round-tripped CLEAN; another instance of the codified CLAUDE.md bench-maxing lesson — not re-added). LOST taxonomy from the artifact: ≈7 entity-translate (`Boltzmann`→"perntzmann", `Tarsus`→"Tarso"), ≈3 broad-enum, ≈several n=1 noise. **Lever built+validated:** `arborist/qa/mt/entity_mask.py` mask/restore (real opus-mt: `who is Paul of Tarsus?` "Pablo de Tarso"→**"Paul of Tarsus"**); default-ON within the default-OFF sandwich. Caveat: bench is lowercased so cap-detector lift is a **lower bound** (corpus-title anchor = v2). Also: fan-out caught + fixed an 88%-engine-error concurrency defect (per-call model load → memoised singleton + lazy per-pair); French + Russian breads added (manifest, `crosslang_source_lang`). **Lift measured 2026-05-17 (comparator corrected, fox):** the true baseline is the pre-ticket ≈0% (raw es query → song-title noise, UNGROUNDED, 10.4s) — NOT native English. Against that: **the sandwich is a large net win (≈0% → 71% es grounded, proof core never corrupted); the −14pp vs English is the cost of a new capability, not a regression — calling it a "fail" was a comparator error.** Firmed 2026-05-18 (n=1, 0 engine-err): EN 85% · es-nomask 71% · fr-nomask **61%** (NOT 47% — that was the mask artifact; honest fr correction). The genuine negative is the **entity-mask lever**: net-negative in *both* languages (es 71→65 borderline, fr 61→47 = −14pp unambiguous; isolated Paul-of-Tarsus win didn't replicate — 3rd bench-maxing-lesson instance), now **default-OFF** (`crosslang_entity_mask=False`); no-mask sandwich is the keeper. Recommendation flipped: **worth continuing (minus the mask)**, not park. Remaining: n=3, fr no-mask, corpus-title anchoring (only untried lowercase-capable detector). CLAUDE.md updated with the durable cross-lang *convention* only. Tasks #14–#17. Phase 1 of the #000001 §7 family; new ticket clears don't-proliferate (fox-directed + distinct Dav1d audience + architectural inflection: a model dependency `[mt]` + a presentation-translation layer — anticipated by #000001 §7's "split the `[mt]` model-distribution work like `[nli]`/vecpack"). **Sandwich:** translate query es→en (retrieval-side, == `--retrieval-keywords`, binds into `retrieval_plan_hash`, NOT `question_hash`) → English answer through the **byte-for-byte untouched verifier** → translate the verified English `answer_text` en→es into a NEW `display_answer` field, banner-labelled, zero grounding (the `_render_audit_label` render-projection pattern). Engine: local `[mt]` extra, Helsinki-NLP `opus-mt-es-en`/`-en-es`, Apache-2.0, hash-pinned, off-repo `~/.arborist/models/mt/`, optional dep, graceful-degrade — mirrors `[nli]`/`ShadowNLI` (#000049) + vecpack (#000051) verbatim; never Hermes-3-8B; not an external API (reproducibility + zero egress + es↔en is the best-resourced pair). Default OFF (`crosslang_translate_enabled`, gated under Phase-0 `crosslang_guard_enabled`); `--crosslang-translate` / `XLANG_MT=1`. Hash invariants (corrected 2026-05-17 — `governance_policy_hash` is sha256 of the *whole* policy, keys.py:182): `question_hash` + `verifier_policy_hash` untouched (user question preserved, verifier byte-identical); `governance_policy_hash` moves like every policy flag → correct cache partitioning by config (not a leak); MT engine identity binds into `RetrievalPlan.mt_*` (run-DAG), not a policy hash. | 2026-05-17 | — |
|
||
| #000055 | Windows quickstart without `make` (`tasks.py` + `make.bat`) | **in progress** — opened 2026-05-16 (fox: "bat files or some shit … avoid needing makefile for windows … we will test the quickstart on windows"). Pure-stdlib `tasks.py` runner mirroring the **quickstart subset** of the Makefile (bootstrap / fetch-cur / ingest-cur-attached / distill ×2 / query / inspect / falsify / burn / bootstrap-crawler / crawl-ingest / stats / verify / search / clean) + a ~10-line `make.bat` shim so `make <target>` works in Windows cmd and `.\make.bat <target>` in PowerShell. Audited the artifact (not the docs): the 2003 dump is opened via stdlib `bz2` (no external `bzip2`); only `fetch-cur` used `curl` (→ stdlib `urllib`); bash `for…&wait` → `subprocess.Popen` fan-out; `arborist` console-script lands at `.venv\Scripts\arborist.exe`. Net: quickstart needs only **Python 3.10+ + sqlite3** — the repo's existing ethos, now true on native Windows. Same `KEY=VALUE` make-style args so documented commands translate 1:1 (one doc form). Makefile untouched, still canonical on POSIX ("keep it as an option"). Found + fixed a README/Makefile discrepancy: README claimed `[dev,html]` bootstrap extras, Makefile installs `.[dev]` — artifact wins. Drift-pinned by `tests/test_tasks_runner.py`. | 2026-05-16 | — |
|
||
| #000054 | Acronym-parens concept extractor (closes the abbreviation→expansion retrieval gap) | **in progress** — Phase 1 (extractor + 481K edges) landed `58027e9`; Phase 2 (consumer-side surfacing — `synonym_expand` rank-and-truncate over the per-token cap, FTS5-`bm25` ordering in `_search_titles`, expanded `accept_tokens` in title-search + core-keyword + title-rerank, `synonym_expand_strict()` for the multiplicative title-purity rerank to exclude noisy `link_reciprocity` edges, tightened extractor regex to `[A-Z]{3,6}` purging 2-letter homonym edges) landed `ce855db`. **End-to-end verified:** `what is a CPU?` → Central processing unit at #1; `what is a GPU?` → Graphics processing unit at #1 EVIDENCE-WARRANTED 1/1; Mount Kilimanjaro / Soviet Union queries unchanged. **bench-qa n=3 limit=5** (2026-05-13T14:24Z): 30/45 STRICT (67%), zero regressions on basics (mona lisa / capital of france / new london bridge each 9/9 STRICT). 2026-05-13 — `arborist/concepts/extract.py:acronym_parens_synonym` lands as a new corpus-agnostic extractor in `EXTRACTORS` (`evidence_kind="acronym_parens"`). Scans each doc's lead chunk for `<Multi-Word Phrase> (ACRO)` where the all-caps acronym's letters match the content-word initials of the phrase in order; emits bidirectional synonym edges between the lowercased acronym and each ≥3-char content token of the phrase. Conservative (strict 1:1 initials, function words filtered, repeated definitions deduped per doc). Closes the *retrieval-side* abbreviation gap (`CPU↔central processing unit`, `GPU↔graphics processing unit`, `RAM↔random access memory`, `FBI↔federal bureau of investigation`, `WHO↔world health organization`, …) that `link_reciprocity_synonym` can't reach because the relation lives in body text, not the wiki link graph (Wikipedia represents abbreviation→expansion as a *redirect* — not an edge). Per-shard like all `concept_relations` data; corpus-agnostic so HTML/blogs/textbooks benefit equally. Retrieval-side only — never proof-path. 8 new tests; full suite green. Closes #000050 §2a's CPU/GPU fixture rows *upstream* of vec; the Orwell-shape conceptual-allusion row remains the genuine #000050 justification. Operational follow-up (not code): `arborist concepts derive --extractor acronym_parens` on each shard. | 2026-05-13 | — |
|
||
| #000053 | Acronym-aware verifier content tokens | **closed · 2026-05-13** — `arborist.qa.evidence._content_tokens` now keeps all-caps 2-3-char acronyms (CPU/GPU/DNA/FBI/USB…) as content tokens instead of dropping every <4-char token; fixes the field case where "what is a CPU?" cited to "CPU design" tripped `TITLE_MISMATCH` spuriously (claim & title share "CPU" but neither registered) — also affects `SUBJECT_TOKENS_ABSENT` (Rule 9), `BARE_NAME_CLAIM`, spotlight-excerpt token pick. Versioned: `content_token_rules: "v2-acronym-aware"` in both default policies + `_VERIFIER_POLICY_FIELDS` → folds into `verifier_policy_hash`, prior cache records orphan on lookup (by design, same discipline as `base_version` / `hyphen_fold_v1`). Monotone toward *fewer* spurious demotes (only relaxes overlap checks, never tightens). 8 new tests; full suite green; `bench-qa-smoke` clean. Does NOT fix the *retrieval* abbreviation→expansion gap (`CPU`→`Central processing unit` = #000050 vec hybrid / `concepts/` synonym edges — the root cause of the satellite-article retrieval). | 2026-05-13 | — |
|
||
| #000052 | Relevance + coherence meta-cognition (answer-*shape* signals) | in progress — **§3.1 `diagnose_coherence` landed** (lexical, no model: `circular` / `phrase_component_reuse` / `vacuous`; in `arborist/qa/inspect.py`, surfaced via `inspect_cache_key` + `arborist inspect` `· incoherent: <kind>`; 9 tests; demote-policy hook deliberately not wired — advisory only). Joins the `diagnose_deflection` / `diagnose_metaphor_deflection` / `diagnose_title_relevance` / soft-preflight family of read-only, demote-only, never-in-proof-path sidecars; `phrase_component_reuse` catches the motivating field case (a subject quoting a phrase, a predicate reusing one of that phrase's own tokens as a bare `the <token>` referent). **Still open: (2) `diagnose_relevance`** — semantic (not just lexical) "aboutness": does the answer address the question; is each claim about its cited source? Today's checks (subject-anchor token overlap, stemmed title-stem overlap) are *lexical* and a token collision defeats them — a small *aboutness/reranker* model (NOT NLI — entailment ≠ topicality) under #000049 §7's discipline cage verbatim (demotion-only, hash-pinned, `relevance_model_version`→`governance_policy_hash` iff it touches `audit_mode`, shadow-first, `[…]` extra, the §7 #20 haystack lesson — never over the whole context); gated on evidence, travels with #000049's model question. Motivating field case (2026-05-12, fox): the `claim_lattice` query that returned *"the phrase 'Zionist entity' is sometimes used as the entity, referring to the State of Israel"* at `EVIDENCE-WARRANTED-PARTIAL 2/3` — incoherent + token-collision recombination that NLI can't catch (returns *neutral*, not *contradiction*) and both lexical relevance checks waved through. Flags an upstream retrieval ticket (polysemy / title-token-soup) as the root-cause fix, not scoped here. #000049 sibling | 2026-05-12 | — |
|
||
| #000051 | Federated vecpack distribution (gossip the embedding backfill) | open · awaiting go/no-go · doc-only scaffold. Makes `chunk_vecs` a distributable artifact: backfill once on any CPU box (cloud / Prometheus-Σ sweep — #000037 §3.1), publish a **vecpack** `(shard_root, vec_backend_version, [(leaf_hash, embedding_blob)…])` over the mesh wire layer, every peer pulls + bulk-loads (sub-ms/chunk on the receiver — the laptop never runs the transformer). Keyed on `leaf_hash` (portable) not `chunk_id` (shard-local). Vecpacks are **soft data** — embeddings are `UNGROUNDED`, never proof path — so a cheap structural sanity gate (chunk exists locally w/ matching leaf_hash, right blob length for (dim,quant), finite norm, backend_version matches) suffices, no Merkle-proof-grade verification needed. Supplies #000050's prereq #1 ("a vecpack exists & is imported on the bench box", not "fox embedded the corpus locally"). GPU producer (the fast path): bge-small-en-v1.5 batched on a CUDA box (4090) ≈ 10³–10⁴ chunks/s → full 6.24M-chunk corpus in *minutes*, not days — drop a CUDA `Embedder` into `default_embedder()`; CUDA stack lives only on the producer box, never in arborist's `python+sqlite3` core. The mechanism behind whitepaper §1's "the embedding pass runs off the device". #000039 / #000050 sibling | 2026-05-12 | — |
|
||
| #000050 | Vec RRF hybrid fusion (#000039 Phase 2) | open · awaiting go/no-go · doc-only scaffold; design in #000039 §4.2 (RRF) + §8 (the gate). Wire `VecBackend` as a 5th retrieval route in `query.py`, RRF-merged (route provenance carried) with the 4 FTS5 routes; UNGROUNDED hits, additive not replacement. Phase-2 sub-items now explicit: **accept-path-5** in `_filter_by_title_relevance` (low-title-overlap vec hits survive only via a stronger span-level warrant, never similarity-score alone — else the title gate drops exactly the semantic candidates vec exists for & the bench shows no lift); **six** vec config fields fold into `governance_policy_hash` (recipe-named quant `int8sym`) **+ a cache-write guard** blocking `providence_cache` persistence for vec/hybrid runs until that's wired; **run-DAG records the vec stage** (backend version, six fields, top_k, query-embedding hash, candidate chunk_ids+distances). **Gated** on (a) a corpus backfill **distributed via #000051** AND (b) a **four-condition** recall bench (A FTS5-only / B vec-only / C RRF hybrid / D candidate-union-no-RRF) clearing the 5pp floor incl. C-beats-D, on the **§2a semantic-allusion fixture set** (running list of query-words-share-zero-tokens-with-target-title cases: Orwell→Eastasia, "what is a CPU?"→Central processing unit, "what is a GPU?"→Graphics processing unit — abbreviation ones also fixable upstream via `concepts/` synonym edges; the Orwell one is the genuine-conceptual-allusion case that justifies the vec layer) + curated + **adversarial-semantic-neighbor** fixtures (else park, vec stays opt-in `--backend vec`; if C≈D ship the union, drop RRF). #000039 follow-up | 2026-05-12 | — |
|
||
| #000049 | Attribution-aware grounding check (the recombination boundary) | open · boundary accepted · production no-go · shadow-path approved (de novo review 2026-05-13 — ticket §7) · doc-only; the home for #000048's deferred §2.3 — closing the 2 recombination over-grounds in `falsification-hard` (hard-003 Mercury / hard-005 Einstein) needs an attribution / dependency-parse or mini-NLI check, which is *not lexical* (#000048 §5). Discipline question answered: a small fixed purpose-built NLI/entailment *model* may influence `audit_mode` only as an opt-in, hash-pinned, governance-hashed, **demotion-only contradiction veto** after shadow-mode evidence (never promotes — `MODEL_ASSISTED_DEMOTION`, never `MODEL_ASSISTED_PROMOTION`). Production verifier unchanged; `falsification-hard` stays 10/12 as an honest boundary marker. Roadmap: Phase 0 (this amendment) → Phase 1 (shadow design: NLI manifest, fetch/verify, `nli_pair@v1` canonicalization, recombination-risk trigger) → Phase 2 (bench-only shadow impl, `[nli]` extra, `make fetch-nli`) → Phase 3 (demotion-only runtime, gated) → Phase 4 (mesh blob sync); §7 #12 six-condition bench gate required before Phases 2–4; if NLI ever affects `audit_mode`, `nli_policy_hash` folds into `governance_policy_hash`. **Phase-2 candidate bench done 2026-05-12** (`~/git/arborist-nli-bench/`, commits `829f9a4` + `a1cb28d`; ticket §7 #18): checkpoint-agnostic harness runs the §7 #5 clause-level algorithm over 28 synth recombination cases (incl. the 2 fixtures + harder shapes) + 26 legit cases (true summaries + near-miss decoys). 4 working candidates; `nli-MiniLM2-L6-H768` (82M, 45ms p50 CPU), `deberta-v3-base-mnli-fever-anli` (184M, 223ms), `bart-large-mnli` (407M, 259ms) all 28/28 catch · 0/26 FP with the standard θe=0.9 entailment guard; `cross-encoder/nli-deberta-v3-base` 27/28; deberta-large repo-id TODO. **Key finding: the §7 #5 two-threshold rule is load-bearing** — 3 of 4 candidates argmax-contradict 1/26 legit cases on the *wrong* source clause (competing-superlative confusion, e.g. "largest hot desert" vs "largest desert overall"); the entailment guard filters every one because another clause restates the claim → 0% guarded FP vs ~4% single-threshold. Picture: recombination is *easy* for any modern NLI checkpoint — differentiator is cost/robustness, MiniLM is the cost-pick, bart-large the threshold-robust pick. **Phase-2 shadow scaffold landed in arborist 2026-05-12** (ticket §7 #19): `arborist/qa/nli/` (manifest pins MiniLM @ a fixed HF revision + θc 0.5/θe 0.9 + 2 alternates; `ShadowNLI`/`shadow_check` lazy-imports `transformers`+`torch` behind a new `[nli]` extra, degrades to `available=False` when absent — SHADOW ONLY, never an `audit_mode` input, manifest not yet in `governance_policy_hash` per §7 #2) + `bench/scripts/nli_shadow_sweep.py` + `make bootstrap-nli` / `make bench-nli-shadow` + 16 tests. Synthetic sweep (116 records): 28/28 recombination demoted, 0/26 FP on legit summaries, 0/9 fires on already-`STRICT_SPAN`. **First bench-qa-traffic sweep** (§7 #20 — `ARBORIST_NLI_SHADOW=1 make bench-qa-smoke`, 15 cells; `query.py` surfaces verifier-input text gated off-by-default, `qa_sweep.py` carries it, `nli_shadow_sweep.py` reads it): the *naive* "NLI on every context clause" scaffold has a **~30% would-demote rate on STRICT answers** — a haystack/multiple-comparisons artifact (real Wikipedia contexts → 100–336 clauses; `max`-over-all almost always hits a tangential "contradiction"). Candidate-clause restriction (§7 #21 — NLI runs on the top-6 source clauses by content-token overlap, `max_candidate_clauses=6`) + speedup (§7 #22 — batched forwards, `ARBORIST_NLI_DEVICE` cuda auto-detect, ONNX-int8 export via `make export-nli-onnx`: torch-cpu-batch1 ~120ms/pair → onnx-int8-cpu-batched ~32ms/pair, seconds on a 4090; `optimum[onnxruntime]` added to `[nli]`; 24 tests) landed. **Verdict moved six times** (§7 #22→#23→#24→#25→#26→#27; §7 #26's "settled" walked back at §7 #27's recall-side realism check — see "REVISED" note at end of this row). The journey ( `bench/scripts/nli_shadow_grid.py` — `{model × k × agg × guard × θc × θe}` sweep, ~4min for 7 models on the 4090, run against bench-qa STRICT samples of growing size: n=1=89 → n=3=275 → n=5=444 → **pooled n=1+3+5 = 808**, plus a mining pass over the 808 real STRICT answers): §7 #22 "fails" = default-config artifact (k=6/θc=0.5); §7 #23 "deberta-base passes" = small-FP-sample artifact (11/28 at n=3); §7 #24 found `microsoft/deberta-large-mnli` passes at n=3 with `margin`; §7 #25 said "no, at n=5 plain `max`@0.96 works"; **§7 #26 (pooled 808, the biggest real FP denominator) showed *that* was the small-sample read** — on 808 cells θc must go back to ~0.995 (the n=5 444-cell set lacked the high-confidence token-collision spurious hits the pooled set has — "Mount Kenya is in Kenya" pulled into a Kilimanjaro answer, Dalí into da Vinci, Donovan into Superman), and at θc=0.995 only `margin` still catches 28/28. **Settled: `microsoft/deberta-large-mnli` / k=3 / agg=`margin` (= max over top-3 candidate clauses of `p_contra − p_entail`) / θc=0.995 → catches all 28 synthetic recombinations (incl. both `5f-fal-hard` fixtures) · 0/808 pooled real STRICT FP · 0/26 synthetic legit FP** — the *only* config (across the 7-model × full-grid sweep) that hits 1.0/0.0 on 808 cells, held at n=3 too; the simpler two-threshold variant (k=2/`max`/θc=0.995) gets 27/28 · 0/808. Findings: `margin` is the right score-shape (discriminates a real recombination — source genuinely contradicts, that clause's entail≈0, margin≈0.997 — from a spurious token-collision FP — source also has an entailing clause, margin dragged to ~0.95–0.99); the specific *large* checkpoint matters (deberta-large-mnli wins; bart-large/deberta-base/MiniLM collapse at proper-pooled-n — §7 #18 "MiniLM cost-pick" overturned); int8-ONNX costs ≈1 catch. **Mining the 808 real STRICT answers found ZERO natural recombination errors** — the boundary is theoretical-in-practice; the 37 would-fires at lowered θc are all token-collision FPs (the candidate selector pulls different-entity same-subject-area clauses — `margin@0.995` survives it, a Phase-3 verifier-matched-clause hook would eliminate it). Remaining: the 28/28 recall is a 28-case denominator and (no natural recombinations to mine) the realistic next check is ~20–30 hand-built synthetic-recombination-vs-real-bench-qa-context fixtures (real haystack, deliberate re-attribution); still SHADOW — runtime promotion is fox+dav1d-decides (then `nli_policy_hash` = model+k+agg+guard+θc folds into `governance_policy_hash` per §7 #2); deberta-large ~400M (GPU-fast). Manifest active defaults stay k=6/θc=0.5; `recommended_operating_point` = the `margin`@0.995 config. Standing lesson, sharpened five times (codified in CLAUDE.md bench-maxing): clean eval ≠ bench-qa precision (§7 #18→#20); default config ≠ best config (§7 #22→#23); small FP sample ≠ large-sample FP rate — *and a bigger sample can vindicate a config a smaller one made look unnecessary* (§7 #23→#24→#25→#26): don't trust any denominator until it's the biggest you can afford, and re-confirm the *config choice* (not just the threshold) each time it grows. Production verifier unchanged; `falsification-hard` stays 10/12. **REVISED at §7 #27 (2026-05-13):** built 25 hand-crafted recombination claims against real bench-qa 30 KB Wikipedia haystacks (deliberate re-attributions inside the same contexts the pipeline produces; `bench/fixtures/5f/recombination-realcontext-v1.jsonl`); ran the full 7-model × full-grid sweep against them vs the 808 pooled STRICT. **The §7 #26 deberta-large-mnli/k=3/margin/θc=0.995 config catches only 6/25 = 0.24 of real-haystack recombinations** (vs 28/28 synthetic — the clean eval inflated recall 4×). **The new fp=0 leader on real haystacks is `facebook/bart-large-mnli` / k=12 / agg=max / θc=0.999 → 12/25 = 0.48 catch · 0/808 STRICT FP** — strictly better than the current 0% but not the §7 #26 "full closure." Diagnosis: the candidate-clause selector (top-k by token overlap) misses the contradicting clause when it shares few subject-area tokens with the answer (the Mount Kenya clause only shares "Kenya" with a "Kilimanjaro is in central Kenya" claim → ranked low → never reaches NLI). The remaining gap is **architectural** — a semantic candidate selector (vec-driven, sibling of #000050/#000051) is the lever that lifts the recall ceiling, not threshold tuning. Permissive operating points are on the menu (bart-large at fp=0.057 catches 0.84; deberta-large at one STRICT FP buys catch 0.56). `recommended_operating_point` revised to `bart-large-mnli / k=12 / max / θc=0.999`. Sixth meta-lesson instance: synthetic eval doesn't predict bench-qa precision *or* recall — neither contrived axis is load-bearing, only the real pipeline shape is. **§7 #28 (2026-05-19):** tinygrad added as an opt-in third NLI inference backend (`ARBORIST_NLI_BACKEND=tinygrad`, parallel to torch/onnx-int8, behind the cage, never auto-preempts the proven path) + a deterministic engine-agreement A/B instrument (`bench/scripts/nli_backend_ab.py`, `make bench-nli-backends`) — gate is *numerical agreement with the torch reference* first, latency second (a divergent engine = a different shadow signal = a different `nli_policy_hash`). First instrument run (CPU, tinygrad absent) already quantified that the deployed int8 export diverges Δmax≈0.42 from torch — the immunity property (`docs/onnx-vendor-capture-immunity.md`) made measurable, not a defect. Real tinygrad numbers pending a producer-box run (tinygrad ∉ arborist deps; frontend op-coverage for the large MNLI checkpoints unverified by design). #000048 follow-up | 2026-05-12 | — |
|
||
| #000048 | Verifier upgrade — recombination-aware grounding + clause segmentation | **closed · 2026-05-12** — steps 2.1 + 2.4 landed 2026-05-11 (12 of 16 residual items: 4 HYBRID_ENTITY over-grounds + 8 Formulate mis-segments → `formulate-hard` 12/12, `falsification-hard` 10/12; each bench-gated, no STRICT-rate regression — 2.1's gate fired on 0 QA answers, 2.4's segmenter touched 7 of 450 lattice cells both verdict changes correct). Step 2.2 (single-clause-containment paraphrase check) attempted + reverted — catches the 2 recombination fixtures but also rejects legit cross-sentence summaries with no threshold separating the two; recombination-vs-summary isn't lexical (§5 "What we learned"). The attribution-aware path moved to **#000049** (fox 2026-05-12). 2 live-pack `expected_reason` updated HYBRID_ENTITY→UNGROUNDED; 12+ tests; `make bench-5f-falsification-hard` / `bench-5f-formulate-hard` / `bench-fork-baseline-hard`. #000046 follow-up; #000047 closed | 2026-05-11 | — |
|
||
| #000047 | ForkScore `_delta_*` aggregator (mean vs max vs sum) | **closed · 2026-05-11** — Option D: `WeightSet.delta_aggregator` ∈ {`mean`,`max`,`sum`} (default `mean` unchanged → no `ESTIMATOR_VERSION` bump), `fork_score._delta_5{s,t,f}` dispatch via `_aggregate`, recorded in `ScoredFork.weights`, per-sub `HARD_REGRESSION_FLOOR` flags aggregator-independent; bench data behind keeping `mean` in `5f-threshold-calibration-2026-05-11.md` §5; 8+1 tests. #000012-revision / #000025 §10.14 follow-up | 2026-05-11 | — |
|
||
| #000046 | Harder 5S/5T/5F fixture tier (below-ceiling baselines) | **closed · 2026-05-11** — Phase 1 `falsification-hard-v1.jsonl` (12 near-misses) + Phase 2 `formulate-hard-v1.jsonl` (12 mis-segments, rate 4/12) + Phase 3 `verify_quotes` paraphrase numeric-agreement gate (`_numeric_signature`; demotes a token-covering span asserting a digit-number the source lacks modulo thousands-comma) → falsification-hard rate 4/12 → 6/12 on a real change; bench-gated (`make bench-qa` n=3×75×3 before/after — no STRICT-rate regression on legit answers; only gate-caused QA shift was correctly demoting a fictional-year claim STRICT→HYBRID); `fork_score` γ·Δ5f went positive on it. Headroom now down to 2 falsification-hard over-grounds (#000048 step 2.1 closed the 4 entity over-grounds; step 2.4 closed the 8 Formulate mis-segments → that pack 12/12; step 2.2 attempted + reverted — the last 2 recombination fixtures need an attribution-aware verifier, now tracked as **#000049**, and stand as documented residue). `make bench-5f-falsification-hard` / `bench-5f-formulate-hard` / `bench-fork-baseline-hard`; 7+ tests. #000025 §10.14 follow-up; #000047 closed; #000048 closed | 2026-05-11 | — |
|
||
| #000045 | Prometheus-Σ Phase 3 sleep-sweep scheduler (gating ticket) | open · doc-only scaffold 2026-05-10; pins 8 governance parameters + 4 retrigger gates; opens implementation only after one retrigger fires | 2026-05-10 | — |
|
||
| #000044 | AUTOCOUNT doc-drift discipline | closed · landed across `fc5ba50` / `03c0f6a` / `6c6defb` / `f5dbfab` / `3b30126` 2026-05-10 (mechanism + 4 metrics + 54 tags across 7 doc files; harness catches drift at test time, refresh is 60-second turnaround) | 2026-05-10 | — |
|
||
| #000043 | Empirical tightening of T3 bound constants (#000036 Phase 2) | open · awaiting go/no-go (parks until v7 deployment surfaces measurement data) | 2026-05-10 | — |
|
||
| #000042 | Term-aliases table (vocabulary-mismatch bridge) | closed · 13 rows live across geometry + classical-physics + arithmetic domains by 2026-05-10 | 2026-05-09 | — |
|
||
| #000041 | Citation-aliases table (PD substitutes for proprietary cites) | closed · 74 rows live as of 2026-05-10 (count grew 40 → 54 → 74; Goldstein/Newton, Mendelson/Enderton/Jech/Landau/Gödel→{Russell IMP, Russell PoM, De Morgan, Boole, Cantor, Peano, Dedekind, SF-LF}, Stanley/Brualdi/Knuth → Bogart+Levin+Keller-Trotter, Dummit-Foote/Barendregt/Böhm-Jacopini → Judson/PLFA/SF, Kolmogorov → Grinstead-Snell+Laplace) | 2026-05-09 | — |
|
||
| #000040 | Phase 5 resolver fix — phrase + content-token cascade (Hilbert terminology mismatch surfaced) | closed · cascade landed 2026-05-09; lift blocked by 1902-vs-modern vocab; follow-up #000042 | 2026-05-09 | — |
|
||
| #000039 | Optional `sqlite-vec` retrieval backend (A/B vs FTS5, hybrid not replacement) | **closed · 2026-05-12** — Phase 0 (doc) + Phase 1 landed 2026-05-11: `arborist/search/vec.py` (`VecBackend`, `chunk_vecs` vec0 + `vec_meta` sibling tables, `embed_documents` incremental/`--rebuild`, pluggable `Embedder` w/ fastembed `bge-small-en-v1.5` default), CLI `arborist embed [--limit/--batch-size/--quant/--rebuild]` + `search --backend vec` + `ingest --embed` (eager opt-in), `[vec]` extra; `--quant {float32,int8}` with int8 head-to-head (3.8-4× smaller, recall ≈ float32 — int8 is the production config); 16 vec tests; ingest integration + idempotency (§14); embed-throughput measured (§14.6 — ~4/s contended, ~2.4 GB int8 full-corpus, full backfill abandoned as a days-long batch job, non-vec ingest unchanged). UNGROUNDED hits, never proof path; vec config folds into `governance_policy_hash` (noted, wired in Phase 2). **Phase 2** (RRF hybrid fusion in `query.py`) → **#000050** (gated on a corpus backfill + a ≥5pp recall bench). | 2026-05-09 | — |
|
||
| #000038 | Phase 4 content acquisition — proprietary textbook license decisions for warrant coverage | closed · obviated 2026-05-10 by alias-substitution sprint under #000031 (74 rows in #000041 + 13 rows in #000042); 92/92 records now resolve. Residue (multilingual PD, Hilbert-Ackermann OCR, Knuth permission, personal-copy path B) preserved as design log §8 | 2026-05-09 | — |
|
||
| #000037 | Prometheus-Σ recursive falsification controller (bicameral substrate) | in progress · Phases 0 + 1 + 1.b + 1.c + 2 landed 2026-05-10; **§12 Trigger 2 fired** (divergence variance 0.575 / N=37); §22 Findings 2 + 3 RESOLVED (kernel/llm cost split + sweep_weights §15.4 + per-mode τ_qa); `controller_events` carries 4 event kinds (decision · difficulty · budget_allocation · falsification_proposal) feeding `arborist controller-events` inspector + live-harvest third bucket in `bench/scripts/harvest_falsification_proposals.py`; §12 Trigger 1 probe wired 2026-05-11 (`trigger_1_branch_density` reads `fork_score_branches` — measurable, not yet fired); Phase 3 sleep-sweep scheduler tracked under #000045 (gating ticket) | 2026-05-09 | — |
|
||
| #000036 | T3 per-window covert-channel budget bound | **closed · 2026-05-11** · Phase 1 + dav1d review → Tier-1 + Tier-2 (Option B = `b1_model=max_envelope` default, in v1, no v2 fork) + KAT-regen tooling (`scripts/generate_t3_bound_kat.py`) all landed 2026-05-11; baseline 625.87 → 6183.02 (max_envelope), `NOT_CERTIFIED_BY_BOUND` at W=10000; 53 → 83 tests; 12-entry active KAT; both dav1d closure blockers cleared, all §5 acceptance criteria met. Continuation: empirical C_B* tightening under #000043 (parks on v7) | 2026-05-09 | — |
|
||
| #000035 | PRG choice for φ_PRG (HMAC-SHA-512 expansion) | **closed · 2026-05-11** — Phase 1 (`arborist/substrate/anchor_prg.py` HMAC-SHA-512 counter-mode KDF, pure stdlib, `PHI_PRG_VERSION = phi-prg-v1-hmac-sha512-le`, `dim_h ≤ 16·2^32` guard + bool-reject; 31 tests; 10 KATs) + Phase 2 (§ 9.10 + § 9.10.1 folded into the **in-repo** v7 substrate spec `docs/_source/merkle-agi-dag-v7.rst` — the v7 formal substrate was brought into the repo for this; supersedes the prior "edit upstream `~/Downloads/merkle-agi-dag_v7.txt`" plan, and is now version-controlled with `docs/_source/index.rst` toctree entry). dav1d-reviewed-final 2026-05-11: little-endian per v7 §A1 (counter + uint32-word reads flipped, KATs regenerated), manifest field `anchor_prg_seed`, float-map prose "negligible mean −2^−32" not "unbiased", seed-independence/single-purpose, M1-enablement-policy separation; HMAC-SHA-512 / 32-byte seed / SHALL-replace all LOCKED. `#000018 §9.2` ("which PRG?") resolved → HMAC-SHA-512 + 32-byte committed seed. (Full upstream v7 spec revision stays exogenous; this lands the amendment into the tracked in-repo copy.) | 2026-05-09 | — |
|
||
| #000034 | Hessian alignment under φ_linear | in progress · Phase 1a landed 2026-05-10 (synthetic-ablation probe + KAT fixture); Phase 1b parks for v7 ramp-up | 2026-05-09 | — |
|
||
| #000033 | Claim-pack pillar VII (combinatorics) | closed · landed 2026-05-09 (live in shard 000.db; lift verified) | 2026-05-09 | — |
|
||
| #000032 | combinatorics@v1 π* (integer counting kernel) | closed · landed 2026-05-09 | 2026-05-09 | — |
|
||
| #000031 | Surface-ingest cited textbooks for claim-pack warrant promotion | closed · Phase 1 + 2 + 2.5 (92/92) + 3 (verifier wiring) + B-1 (via_citation_alias attribution: 18 direct + 74 +alias) + B-2 (source-side title-from-author backfill) all landed 2026-05-09 / 2026-05-10 | 2026-05-09 | — |
|
||
| #000030 | Math π* expansion: SymPy substrate (algebra · calculus · linalg) | closed · all 7 phases + 1b landed 2026-05-09 (`abe5988`) | 2026-05-09 | — |
|
||
| #000029 | Claim-pack source (axiom/theorem JSON bundles) | closed · landed 2026-05-09 | 2026-05-09 | — |
|
||
| #000028 | Multi-modality witness for canonical shapes | closed · landed 2026-05-09 + follow-ups (capital ledger · sample rate) | 2026-05-08 | — |
|
||
| #000027 | Canonical projections persist to providence_cache | closed · landed 2026-05-09 | 2026-05-08 | — |
|
||
| #000026 | Real-shard workload baseline + search latency | closed · Phase 1 + 2 + 3 landed 2026-05-09 (Phase 3 in `60b5748`) | 2026-05-08 | — |
|
||
| #000025 | 5F battery (Function · Finetuning · Falsification · Formulate · Feedback Loop) | closed · 2026-05-11 — Phase 1a–1f landed 2026-05-09/10; Phase 1g (§10.13 feedback latency + persisted-footprint efficiency), 1h (§10.14 ForkScore threshold-calibration handoff → #000012 §8), 1i (§10.11 persistent SelfModel-chain lineage: `bench-5f-selfmodel-snapshot` grows the chain, `run_finetuning` shard-chain mode reads the 2 latest snapshots) all landed 2026-05-11. Every §10 closure criterion met | 2026-05-07 | — |
|
||
| #000024 | 5T Phase 1b + Dav1DPrometheus vocabulary alignment | closed · landed 2026-05-08 | 2026-05-07 | — |
|
||
| #000023 | 5S Phase 1b: Syllogism · Synthesis · Semiotics | closed · landed 2026-05-08 | 2026-05-07 | — |
|
||
| #000022 | Adapter LossReport (PRD I9 analogue) | closed · landed 2026-05-07 | 2026-05-07 | — |
|
||
| #000021 | 5S/5T/5R benchmark fixtures + harness | closed · Phase 1a + 2 landed 2026-05-08 | 2026-05-07 | — |
|
||
| #000020 | Capital-cost ledger (8-capital queues) | closed · landed 2026-05-07 | 2026-05-07 | — |
|
||
| #000019 | Specification methodology for π* and V | closed · landed 2026-05-07 | 2026-05-07 | — |
|
||
| #000018 | Adversarial soft-hash covert-channel analysis | closed · landed 2026-05-09 (analysis doc) | 2026-05-07 | — |
|
||
| #000017 | Memory-root: lifelong learning audit chain | closed · landed 2026-05-07 | 2026-05-07 | — |
|
||
| #000016 | ZK Phase-2 frontier proof (concretize) | parked · bench-plan + wire-protocol landed 2026-05-09 (sibling-repo measurement pending) | 2026-05-07 | — |
|
||
| #000015 | π* domain library + cross-domain composition | closed · landed 2026-05-07 | 2026-05-07 | — |
|
||
| #000014 | SelfModel: schema, falsification, integration | closed · landed 2026-05-07 | 2026-05-07 | — |
|
||
| #000013 | Spatial-temporal substrate (Merkle-AGI v7-W) | closed · landed 2026-05-09 (substrate paper + frontier catalog + namespace stub) | 2026-05-07 | — |
|
||
| #000012 | Selection & consensus protocol (Merkle-AGI v8) | in progress · Phase 1a (ForkScore) landed 2026-05-08; Phase 1b (consensus paper, `docs/_source/merkle-agi-v8-consensus.rst` 834 lines) landed 2026-05-10; Phase 1c (branch-set persistence — `fork_score_branches` sibling table, `persist_branch_score` + `branch_set_density`, 6 new CLI flags on `arborist substrate score`, default-off) landed 2026-05-10 — feeds #000037 §12 Trigger 1; Trigger 1 probe wired 2026-05-11 (`trigger_1_branch_density` reads `fork_score_branches` via `branch_set_density()` — measurable, not yet fired: no branch sets persisted) | 2026-05-07 | — |
|
||
| #000011 | SOFT_PREFLIGHT_HINT model-assisted sidecar | closed · landed 2026-05-04 (zero-shot full impl) | 2026-05-04 | D1 (preserves) |
|
||
| #000010 | Meta-Cognition Preflight Guard (M0 / MCTL) | closed · landed 2026-05-03 (Phases 1–4); DAG binding shipped via #000009 | 2026-05-03 | D1, D3 |
|
||
| #000009 | Preflight run-DAG node binding (#000008+#000010) | closed · re-landed 2026-05-04 (§8 corrections: reject-path DAG, nested CTI clauses) | 2026-05-03 | D3, D4 |
|
||
| #000008 | Broad-quantifier preflight guard | closed · landed in `4f2b5a6`; Phase 5 DAG binding split into #000009 | 2026-05-02 | — |
|
||
| #000007 | Query-layer hyphen folding | closed · 2026-05-02 | 2026-05-02 | — |
|
||
| #000006 | Bench-emergent findings (rolling research log) | open · rolling research log (verifier-ladder healthy at 134 cycles; emergent gems still being mined) | 2026-05-02 | — |
|
||
| #000005 | Label ladder migration (POINTER-LINKED → …) | closed · 2026-05-02 | 2026-05-01 | D7 |
|
||
| #000004 | Directive coverage in bench summary | closed · `acd1f9c` | 2026-05-01 | D8 |
|
||
| #000003 | Anchor-class warrant generalization (Module H+)| closed · 2026-05-02 | 2026-05-01 | D6 |
|
||
| #000002 | Reference-Frame Polarity Contract (Module L) | closed · 2026-05-02 | 2026-05-01 | D3 |
|
||
| #000001 | Retrieval-keywords audit gap (+ §7 cross-language transforms) | **reopened 2026-05-17 (in progress)** — keyword scope stays closed/landed 2026-05-02 (run-DAG `RetrievalPlan` binding, §5). §7 extends the *same* retrieval-transform-provenance substrate to a sibling: cross-language query bridging. Strict bright line — MT/aliases/stopword-guards *propose* candidates, only source-language evidence *warrants*; English answer verified English-vs-English by the **untouched binary verifier**, Spanish is display-only (zero grounding, banner-labelled). Held strictly this needs **no new `audit_mode` token, no `EVIDENCE-WARRANTED-CANONICAL`, no `surface_language` field, no `providence_cache` column, no new run-DAG stage** (the >10%-back deletion; CLAUDE.md schema-column-unchanged + verifier-stays-binary). **Phase 0 landed 2026-05-17** — `arborist/qa/crosslang.py` (deterministic, no model: `¿`/`¡`/non-ASCII signal, es-v1 stoppack); pre-preflight fail-closed mirroring `quantifier_should_reject` (Merkle-auditable reject DAG, `CROSS_LANGUAGE_UNSUPPORTED`, no LLM) + es-stoppack strip on `retrieval_query` only. Measured: `¿Qué es el anarcocapitalismo?` **10.4 s → 1.6 s** (~6.4×, still honest UNGROUNDED); English control byte-identical (by construction — `guard()` returns None for pure-ASCII). 19 tests; full suite 2470 passed, 0 regressions (incl. #000053/#000054); `bench-qa-smoke` stable anchor 3/3 STRICT. No schema change, no new governance/verifier *fields*, no `RetrievalPlan` change; `question_hash` + `verifier_policy_hash` untouched (the enabling flag moves `governance_policy_hash` like every policy flag — whole-policy hash, correct cache partition; corrected 2026-05-17, authoritative in #000056 §2 #6). `RetrievalPlan` MT extension landed in #000056 (Phase 1). **Feature-flagged default-OFF** (fox request): `policy["crosslang_guard_enabled"]` gates both seams via one point → flag-off reverts byte-for-byte to legacy (clean A/B); surface `arborist query --crosslang-guard` / `make query XLANG=1` / `tasks.py XLANG=1`. Default-flip is a separate bench-gated fox decision (rollout discipline matches #000008/#000011/#000049). 20 tests; full suite 2471 passed. **Collision constraint:** the guard is a specific source-language function-word stoppack, NOT a len≤2 heuristic — a blanket short-token drop regresses #000053/#000054 (`AI`/`ML`/`CPU`/`GPU` are load-bearing); `_FTS5_STOPWORDS`/`_TITLE_STOPWORDS` stay in sync; n=3 English bench, no >5pp regression. **Phase 1 split out → #000056 "Operation Sandwich"** (fox-directed; MT provider decided = local `[mt]` opus-mt, hash-pinned, not Hermes/not-API). Deleted by five-step (recorded in §7.6): raw-Spanish route (measured noise vs `en`), alias-map substrate (hand-curated worse-MT), cross-modal/multilingual-vec/SQD-language/ABCDEFG-5S (no named defect — the retired `arborist/v7/` anti-pattern). | 2026-05-01 | D4 |
|
||
|
||
## Next ID
|
||
|
||
`000066`
|