Commit graph

1 commit

Author SHA1 Message Date
3aae8119a6
#000065 step 1: routing helper + meta field + pre-migration snapshot
Three pieces, all read-only or additive — no shard mutation, no
schema-version bump:

1. shard_for_document(document_root, M) in arborist/document.py.
   Pure function: int(document_root[:8], 16) % M. 22 tests cover
   determinism, range-bounds, near-uniform distribution (±5pp at
   N=20k), and seven lock-in fixtures so peers will disagree
   loudly if anyone changes the formula.

2. corpus_shard_count meta field + get/set helpers in store.py.
   Lives in the existing key/value meta table; SCHEMA_VERSION
   stays at v9.8.0 (the DDL doesn't change and source_root is
   layout-independent, so cache records survive a reshard).
   Legacy shards (without the field) return None; reshard tool
   populates it on every target shard at migration time.

3. Pre-migration snapshot captured to
   bench/results/pre-migration-snapshot.json:
     docs        3,468,392  (3,468,226 globally unique)
     chunks      6,235,764
     edges      90,593,537
     audit       3,468,403
   This is the reference set post-reshard row counts must match.

4. Audit-event extraction script writes all 3.47M events from
   all 4 shards to /tmp/audit-events.ndjson (2.0 GB) for the
   Option-A canonical-chain consolidation step. Verifies chain
   integrity on extract — all 4 source chains report 0 breaks.

5. Fixed a wrong chunk count in docs/corpus-history.md
   (had ~3.54M/shard; actual is ~1.56M/shard) and added the
   edge-count column (~22.6M/shard, 90.6M total). 6.24M chunks
   total, not 14.12M.

Tests: 29 new pass (22 routing + 7 meta). No existing tests
touched.
2026-05-26 13:13:44 -04:00