phase 0 explore: aborist core + sources + distill + evict

A content-addressed, Merkle-committed document store implementing the
runtime spec from Merkle Providence Reverse RAG (April 2026 whitepaper)
and Merkle-AGI v9.8 admissibility ledger. Ports proxy.unturf.com Go
merkle conventions to Python: non-commutative HashCombine with 0x03
prefix, explicit IsLeft per sibling, self-duplicate odd elements.

What's in:

- merkle.py — proof generation/verification, JSON serialization
- store.py — v9.8 SQLite schema: 8-dim providence_cache key,
  falsification_state, append-only audit chain, surface/core kind,
  hot/warm/cold tier, derivations, edges
- ingest.py — Source -> normalize -> chunk -> merkle -> upsert,
  idempotent on document_root collision
- search/ — SearchBackend ABC with explicit AuditMode (STRICT/HYBRID/
  VISUAL), FTS5 backend returning VISUAL hits
- sources/ — wikipedia.py (streaming bz2/MySQL extended-INSERT parser
  for 2003-era cur dumps); html_page.py (selectolax + httpx, robots.txt
  honored automatically)
- distill/ — Distiller ABC + first-sentence-v1 stub. Runner generates
  per-contributing-chunk Merkle proofs binding cores back to source
  document_root.
- evict.py — hot->cold demote (NULLs content, drops FTS row, retains
  leaf_hash). rehydrate() refetches via source pipeline; matching root
  restores content, mismatching root marks providence stale and writes
  rehydrate_drift event. Cores never evict.
- cli.py — ingest / search / verify / stats / distill / evict /
  rehydrate
- 31 tests covering merkle round-trip, ingest+audit, chunker version
  binding, html parse, distillation proof verification, evict+
  rehydrate including drift detection.

Smoke: 503 Wikipedia 2003-05-16 + 3 fox-owned HTML pages ingested,
478 cores produced (24 surface->core merkle dedups), 7 chunks evicted
to cold and round-tripped via rehydrate, 987 audit events chained 0
breaks.

Makefile

@@ -0,0 +1,70 @@
+# aborist — Makefile entry points
+# Every workflow lives behind a `make` target. Bare python commands are not
+# the user interface.
+
+# Tools and config
+PYTHON       ?= python3
+VENV         ?= .venv
+PIP          := $(VENV)/bin/pip
+PY           := $(VENV)/bin/python
+ABORIST      := $(VENV)/bin/aborist
+
+# Data + DB
+DATA_DIR     ?= data
+WP_DUMP_URL  ?= https://dumps.wikimedia.org/archive/2003/2003-05-16/en/20030516_cur_tablesql.bz2
+WP_DUMP      := $(DATA_DIR)/20030516_cur_tablesql.bz2
+DB           ?= $(HOME)/.aborist/aborist.db
+
+# Smoke-test caps so make all stays fast
+INGEST_LIMIT ?= 500
+VERIFY_N     ?= 10
+SEARCH_Q     ?= computer
+
+.PHONY: all bootstrap fetch ingest verify search stats test clean clean-db clean-data help
+
+all: bootstrap fetch ingest verify stats ## bootstrap → fetch → ingest → verify → stats
+
+help: ## show this help
+	@awk 'BEGIN{FS=":.*##"} /^[a-zA-Z0-9_-]+:.*##/{printf "  %-14s %s\n", $$1, $$2}' $(MAKEFILE_LIST)
+
+$(VENV)/bin/activate: pyproject.toml
+	$(PYTHON) -m venv $(VENV)
+	$(PIP) install --upgrade pip wheel
+	$(PIP) install -e '.[dev]'
+	@touch $(VENV)/bin/activate
+
+bootstrap: $(VENV)/bin/activate ## create venv and install editable package
+
+$(WP_DUMP): | $(DATA_DIR)
+	@echo ">> fetching $(WP_DUMP_URL)"
+	curl -fL --retry 3 -o $@ "$(WP_DUMP_URL)"
+
+$(DATA_DIR):
+	mkdir -p $(DATA_DIR)
+
+fetch: $(WP_DUMP) ## download Wikipedia 2003-05-16 cur dump (cached)
+
+ingest: bootstrap fetch ## ingest INGEST_LIMIT articles into $(DB)
+	$(ABORIST) --db $(DB) ingest --source wikipedia_cur --path $(WP_DUMP) --limit $(INGEST_LIMIT)
+
+verify: bootstrap ## round-trip Merkle proofs for VERIFY_N random documents
+	$(ABORIST) --db $(DB) verify -n $(VERIFY_N)
+
+search: bootstrap ## keyword search; override SEARCH_Q (or pass Q=...)
+	$(ABORIST) --db $(DB) search '$(if $(Q),$(Q),$(SEARCH_Q))'
+
+stats: bootstrap ## counts: documents, chunks, edges, audit chain
+	$(ABORIST) --db $(DB) stats
+
+test: bootstrap ## run pytest suite
+	$(VENV)/bin/pytest -q
+
+clean: ## remove venv + caches (keeps fetched data and db)
+	rm -rf $(VENV) .pytest_cache **/__pycache__ aborist.egg-info
+	find . -type d -name __pycache__ -prune -exec rm -rf {} +
+
+clean-db: ## drop the aborist db (keeps fetched data and venv)
+	rm -f $(DB) $(DB)-journal $(DB)-wal $(DB)-shm
+
+clean-data: ## remove fetched dumps
+	rm -rf $(DATA_DIR)