arborist/docs
russell@unturf.com 514dcd8342
#000065 closed: production reshard landed; record in corpus-history
2026-05-26 19:47 UTC. ~94 min wall. 3,468,226 globally-unique docs +
6,235,588 chunks + 90,592,990 edges + 3,468,403 audit events re-routed
from non-deterministic spray-by-ingest-order layout to canonical
content-hash M=4 layout (shard_idx = int(document_root[:8], 16) % 4).

Final state:
  per-shard doc uniformity within ±0.04% (theoretical max ±0.05% for
    first-32-bit SHA-256 prefix)
  audit chain consolidated to shard 000 via Option A (re-sorted by ts,
    re-chained; bodies preserved unchanged; tail event type=reshard
    carries plan+result body, hash 8da3aa19…)
  on-disk sizes: 000=11.0 / 001=8.8 / 002=8.8 / 003=8.8 GB
  validation: chunks delta 176 (0.003%) + edges delta 547 (0.0006%)
    are cross-shard dupes from re-ingest history, collapsed by
    INSERT OR IGNORE; within the 1% tolerance gate
  smoke queries: Barack Obama / YouTube / Albert Einstein all
    returned proper evidence from correct (hash-routed) shards
  chain-check-shards: 0 breaks on every shard

Two mid-flight defect fixes (also committed):
  04edff7: derivations.src_root FK guard fired on legitimately
    cross-shard refs → writer connection PRAGMA foreign_keys = OFF
  c86d5ac: WAL accumulated ~37 GB across passes (SQLite auto-checkpoint
    blocked by open reader cursors) → _checkpoint_truncate between
    phases. Production migration was rescued mid-flight by manual
    sibling-connection wal_checkpoint(TRUNCATE) freeing 27.7 GB.

Closes #000065 in both the index and the per-ticket file. The
per-ticket design doc stays open as a design-log artifact (its
content is still the right reference for the next reshard / for
graft mode #000066).

Follow-on tracked as tasks #44–#47:
  #44 re-pack post-reshard shards into DO Spaces (current bucket
      packs are stale, still in pre-reshard topology)
  #45 verify bucket pack hydration is deterministic against the new
      content-hash layout
  #46 genesis a fresh peer on 3090-ai.foxhop.net from cloud — first
      real SPV-wallet end-to-end test
  #47 retire stale pre-reshard bucket packs after #46 confirms
2026-05-26 15:51:20 -04:00
..
_source docs: 3 concepts diagrams + Python-library cookbook recipes 2026-05-24 14:12:26 -04:00
diagrams docs: 3 concepts diagrams + Python-library cookbook recipes 2026-05-24 14:12:26 -04:00
posts docs: draft COGS tweet — cost of a grounded answer (Hermes ~9c, Qwen ~16c per 1k) 2026-05-21 13:52:20 -04:00
tickets #000065 closed: production reshard landed; record in corpus-history 2026-05-26 15:51:20 -04:00
bench-maxing.md aborist/arborist 2026-05-07 09:31:49 -04:00
benchmark-matrix.md docs(#000057): capture known-good serving invocations for the salt buildout 2026-05-20 15:20:34 -04:00
benchmarks.md docs+code: ground §12 judge pipeline in the actual judge_code.py 2026-05-22 19:17:28 -04:00
calculator-test-patterns.md ticket #000036 Tier-2: dav1d Option B (conservative B1 envelope) applied in v1 2026-05-11 07:06:50 -04:00
cold-object-store.md #000061: pack format v2 — self-sufficient new-peer hydration 2026-05-25 22:21:45 -04:00
corpus-history.md #000065 closed: production reshard landed; record in corpus-history 2026-05-26 15:51:20 -04:00
crawler.md docs: web crawler guide — discovery, fast mode, dedupe, orphans 2026-05-22 07:38:37 -04:00
cti-architecture.md aborist/arborist 2026-05-07 09:31:49 -04:00
embedding.md feat: arborist.embed — supported library-embedding surface 2026-05-22 13:03:15 -04:00
energy-cogs-benchmark.md docs(#000057): correct cost claim — <$0.10/1k-q is hermes-8B only, not qwen 2026-05-21 13:50:01 -04:00
lexical-first-rationale.md docs: lexical-first-rationale.md — why the cheap retrieval path is the default 2026-05-12 09:23:50 -04:00
mesh.md aborist/arborist 2026-05-07 09:31:49 -04:00
onnx-vendor-capture-immunity.md feat(#000049 §7 #28): tinygrad NLI backend + deterministic engine-agreement A/B; ONNX-immunity rationale 2026-05-19 12:34:04 -04:00
pager.style docs: arborist-one-pager + arborist-two-pager — Dav1d/fox-signoff summaries with letterhead, license, and 2 strategic appendix diagrams 2026-05-14 09:48:50 -04:00
pi-star-composition.md pi_star: land ticket #000015 (π* domain library + composition algebra) 2026-05-07 16:51:33 -04:00
qa-modes-bench.md docs(#000057): Addendum 8 — control sweep retrieval × model × framing × reasoning 2026-05-20 06:52:42 -04:00
relevance-and-veto-synthesis-for-dav1d.md docs: relevance-and-veto-synthesis-for-dav1d.md — single decision brief synthesizing #000049 + #000052 §3.1 + §3.2 for forward review 2026-05-13 15:34:21 -04:00
seven-point-program.md tests/doc_counts: regression test for numeric claims in docs/ (4x drift fix) 2026-05-10 16:15:52 -04:00
soft-hash-channel-analysis.md docs/#000018 §9.2: mark resolved — φ_PRG = HMAC-SHA-512 (#000035 closed) 2026-05-11 17:20:48 -04:00
soft-hash-channel-t3-bound.md ticket #000036: add KAT-regen tooling + close 2026-05-11 08:02:25 -04:00
spec-methodology.md docs: land ticket #000019 (spec methodology for π*, V, policy fields) 2026-05-07 16:53:28 -04:00
stock-v1-config.md fix(#000057): GPU COGS = generation only — retrieval/verify don't touch the card 2026-05-21 11:07:37 -04:00
TICKETS.md #000065 closed: production reshard landed; record in corpus-history 2026-05-26 15:51:20 -04:00
tool-action-dag-design.md docs: add tool-action-dag-design.md research path (pre-ticket) 2026-05-07 19:47:50 -04:00
v7w-frontier-catalog.md #000013 closed: v7-W spatial-temporal substrate paper + namespace 2026-05-09 15:00:05 -04:00
v8-fork-score.md CLI: arborist v8 score → arborist substrate score 2026-05-10 09:12:34 -04:00
warrant-substrate-cookbook.md docs: bump warrant-substrate-cookbook AUTOCOUNT 20 -> 28 for #000054 tests 2026-05-13 07:01:02 -04:00
zk-frontier-bench.md #000016 parked: ZK frontier-proof bench plan + wire protocol 2026-05-09 15:05:08 -04:00
zk-wire-protocol.md #000016 parked: ZK frontier-proof bench plan + wire protocol 2026-05-09 15:05:08 -04:00