cold_pack: filter _content_size in single-shard restore — align with pull_metadata_pack NULL contract
_restore_generic_table forwarded every JSONL column to the INSERT statement, including the producer's synthetic ``_content_size`` column (#53 — emitted on chunks to carry the size hint while the actual content BLOB is dropped from the metadata pack to keep it small). The target chunks table has no such column, so the restore hard-errored on ``no column named _content_size`` before any row landed; four cold_object tests had been failing on the branch (push_pack_then_pull_full / just_enough_hydrate / splits_to_fit_dvdr / v2_hydrates_fresh_empty_db). Drop ``_content_size`` from the column list. chunks rows land with ``content IS NULL`` per ``pull_metadata_pack``'s documented contract (``evict.py:825``); phase 2 (``pull_chunk_pack``) fills bytes via UPDATE WHERE leaf_hash=?. The size hint is unused in the single-shard path. The multi-shard ``_restore_routed_table`` still pre-allocates a zero-byte BLOB of that size for in-place phase 2 UPDATE (avoids page splits at corpus scale); that's a latent inconsistency with the doc contract on the just-enough routed path, but no test exercises it and the performance argument is real at scale. Not touching it here. Suite: 2995 passed (up from 2991), no regressions.
This commit is contained in:
parent
f550a46627
commit
ab7ae4c0fb
1 changed files with 14 additions and 2 deletions
|
|
@ -470,7 +470,19 @@ def _restore_generic_table(
|
|||
table: str,
|
||||
in_path: Path,
|
||||
) -> int:
|
||||
"""Read array-per-line JSONL → INSERT OR IGNORE batches into `table`."""
|
||||
"""Read array-per-line JSONL → INSERT OR IGNORE batches into `table`.
|
||||
|
||||
Chunks special case (#53): the producer dumps a synthetic
|
||||
``_content_size`` column carrying each chunk's content byte-length
|
||||
(the actual ``content`` BLOB is dropped from the metadata pack to
|
||||
keep it small). Consumer drops the synthetic column from the INSERT
|
||||
column list — chunks rows land with ``content IS NULL`` per
|
||||
``pull_metadata_pack``'s documented contract; phase 2
|
||||
(``pull_chunk_pack``) fills the bytes via UPDATE WHERE leaf_hash=?.
|
||||
The size hint is currently unused in the single-shard path; a
|
||||
future zeroblob pre-allocation could read it back if page-split
|
||||
cost becomes measurable.
|
||||
"""
|
||||
BATCH = 5000
|
||||
batch: list[tuple] = []
|
||||
cols: list[str] | None = None
|
||||
|
|
@ -478,7 +490,7 @@ def _restore_generic_table(
|
|||
insert_sql: str | None = None
|
||||
for row in read_columnar_jsonl(in_path):
|
||||
if cols is None:
|
||||
cols = list(row.keys())
|
||||
cols = [c for c in row.keys() if c != "_content_size"]
|
||||
placeholders = ", ".join("?" for _ in cols)
|
||||
col_list = ", ".join(f'"{c}"' for c in cols)
|
||||
insert_sql = (
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue