#000061: bound memory in edges fan-in dump (index + batched groupby)

Live v2 corpus run showed ~5 GB RSS per worker — RssAnon dominant, so
process heap, not mmap. Traced to two unfixed memory pits in the edges
fan-in dump path:

1. SQLite ORDER BY on edges (22M rows, no covering index for the v2
   sort order dst_uri+edge_type+anchor) allocates a multi-GB in-memory
   sort area before spilling. Adding:

       CREATE INDEX IF NOT EXISTS idx_edges_dst_uri_type_anchor
       ON edges(dst_uri, edge_type, anchor, dst_root, src_root)

   means the ORDER BY walks the index in order — no in-memory sort.
   First create takes ~30-60 s on a 22M-row shard; idempotent on
   subsequent dumps. Disk cost ~1 GB per shard (4 shards × 1 GB ≈
   2-3 % corpus footprint increase). Worth it.

2. Python groupby accumulator: src_roots = [row[4] for row in group]
   materializes the entire src_root list per destination. For
   en.wikipedia.org/wiki/* destinations with millions of inbound links,
   this list is itself ~GB-sized. Switch to bounded batches:

       _FAN_IN_BATCH = 10_000     # max src_roots per fan-in JSON row

   A destination with N inbound links splits into ceil(N / batch) rows.
   Restore path (INSERT OR IGNORE) handles multi-row destinations
   correctly because PK includes src_root — accidental duplicates
   collapse cleanly.

New regression test test_edges_fan_in_batches_huge_destinations builds
an edges table with FAN_IN_BATCH+137 rows pointing at one dst_uri,
verifies the dump produces the expected number of split rows and the
restore reconstructs all N edges with no loss or duplication.

25 cold-object + evict tests pass.

tests/test_cold_object.py

@@ -299,6 +299,75 @@ def test_push_pack_local_dir_writes_files(tmp_path):
         conn.close()
 
 
+def test_edges_fan_in_batches_huge_destinations(tmp_path):
+    """Regression test for the memory-bounded edges dump path.
+
+    A destination with > _FAN_IN_BATCH inbound links should be split
+    into multiple JSON rows on dump, and restored to the live schema
+    with no missing or duplicated edges. Without this batching the
+    groupby accumulator held the whole src_roots list in Python heap
+    (~5 GB per worker observed on the v2 corpus run).
+    """
+    import sqlite3
+    from arborist.cold_pack_metadata import (
+        _dump_edges_fan_in,
+        _restore_edges_fan_out,
+        _FAN_IN_BATCH,
+    )
+    from arborist.store import connect as store_connect
+
+    # Build a minimal schema-bearing DB and stuff edges directly.
+    src_db = tmp_path / "edges.db"
+    conn = store_connect(src_db)
+    try:
+        # Insert N_LINKS edges all pointing at the same dst_uri so the
+        # groupby for this destination spans multiple batches.
+        N_LINKS = _FAN_IN_BATCH + 137  # one full batch + a partial
+        dst_uri = "https://example.com/popular"
+        edge_type = "wikilink"
+        anchor = ""
+        dst_root = ""
+        with conn:
+            conn.executemany(
+                "INSERT OR IGNORE INTO edges "
+                "(src_root, dst_root, dst_uri, edge_type, anchor) "
+                "VALUES (?, ?, ?, ?, ?)",
+                [
+                    (f"src{i:08d}" + "0" * 24, dst_root, dst_uri, edge_type, anchor)
+                    for i in range(N_LINKS)
+                ],
+            )
+
+        # Dump
+        dump_path = tmp_path / "edges.jsonl"
+        n_rows_written = _dump_edges_fan_in(conn, dump_path)
+        # One destination split into ceil(N_LINKS / FAN_IN_BATCH) rows.
+        expected_rows = (N_LINKS + _FAN_IN_BATCH - 1) // _FAN_IN_BATCH
+        assert n_rows_written == expected_rows, (
+            f"expected {expected_rows} fan-in rows for {N_LINKS} edges, "
+            f"got {n_rows_written}"
+        )
+
+        # Wipe edges and restore. INSERT OR IGNORE handles any duplicate
+        # PK rows from chunk boundaries cleanly.
+        with conn:
+            conn.execute("DELETE FROM edges")
+        n_restored = _restore_edges_fan_out(conn, dump_path)
+        conn.commit()
+        assert n_restored == N_LINKS, (
+            f"expected {N_LINKS} edges restored, got {n_restored}"
+        )
+
+        # Verify the live table has exactly N_LINKS rows for that destination.
+        final_count = conn.execute(
+            "SELECT COUNT(*) FROM edges WHERE dst_uri = ?",
+            (dst_uri,),
+        ).fetchone()[0]
+        assert final_count == N_LINKS
+    finally:
+        conn.close()
+
+
 def test_push_pack_v2_hydrates_fresh_empty_db(tmp_path):
     """Pack format v2 self-sufficiency test: build a pack from a populated
     DB, then unpack it into a FRESH empty DB and verify every table is