cold_clone: handle SQLite BLOB-as-str affinity in just-enough strip

Some rows in shards-genesis-v2 came back from the chunks.content BLOB
column as  instead of bytes — SQLite's type-affinity rule lets a
BLOB-affinity column hold any storage class. The strip-and-upload path
crashed on bytes(some_str) without an encoding. latin-1 preserves
arbitrary byte values 1:1, so the leaf_hash check still matches.
This commit is contained in:
russell@unturf.com 2026-05-29 21:05:59 -04:00
parent 3fcaa2ae9e
commit 7bda7450c8
No known key found for this signature in database

View file

@ -185,8 +185,19 @@ def _strip_chunk_content_and_dump_blobs(
f"WHERE chunk_id IN ({placeholders})",
batch_ids,
).fetchall()
# SQLite type affinity: a `BLOB` column can come back as
# bytes / bytearray / memoryview, OR as `str` if the value
# was originally INSERTed as text. latin-1 preserves
# arbitrary byte values 1:1 (codepoints 0..255) so the
# hash check below still matches.
def _as_bytes(v):
if isinstance(v, (bytes, bytearray)):
return bytes(v)
if isinstance(v, memoryview):
return v.tobytes()
return v.encode("latin-1")
triples = [
(cid, lh, bytes(c)) for cid, lh, c in rows if c is not None
(cid, lh, _as_bytes(c)) for cid, lh, c in rows if c is not None
]
with ThreadPoolExecutor(max_workers=upload_workers) as ex:
for cid in ex.map(_upload, triples):