Open ticket. Today's hydrate_from_metadata_pack takes one conn and
writes every incoming row into one shard — fine when the corpus
was a single shard, broken now that #000065 put the producer in
M=4 hash-routed topology. A fresh peer pulling packs must land each
doc on `shard_for_document(document_root, M)` — same routing
function as the producer — or the consumer's M=4 ATTACH-and-route
assumption is just decoration over a single-shard reality.
Plan:
1. Add corpus_shard_count to pack manifest (read from source meta
during dump_shard_metadata) — pack carries the M it was built
against.
2. restore_shard_metadata_routed(targets, M, table_dir) in
cold_pack_metadata.py — mirrors _route_per_doc_table from
migrate.py (per-document tables route by document_root /
src_root / core_root; consolidated tables all go to target 0).
3. hydrate_from_metadata_pack gains a targets / shards_dir param.
4. arborist cold unpack --shards-dir DIR initialises M target
shards from the manifest's corpus_shard_count and routes.
5. Regression test: pack 2 shards → hydrate into fresh 4 shards
→ assert every doc on its hash-routed target.
Refactor question (raised, not decided): the routing rules
(ROUTED_BY_DOCUMENT_ROOT, CONSOLIDATED_TABLES) currently live in
migrate.py. Either duplicate them in cold_pack_metadata.py (fast)
or factor into arborist/multi_shard.py (cleaner, also serves
#000066 graft mode). Shared module is more honest.
Prerequisite for #46 (genesis on 3090 from cloud). Without this,
genesis is a 2-step α-kludge (hydrate-then-reshard) that wastes
~30 min and treats packed shards as if from an arbitrary topology.
Index entry bumped; next-id 67→68. Per-ticket spec doc to follow
when the implementation gates open.
russell@unturf.com
9cfb9c8d01