arborist/docs/tickets/ticket-000039-sqlite-vec-optional-backend.md

Ticket #000039 — Optional sqlite-vec retrieval backend (A/B vs FTS5, hybrid not replacement)

Status: open · awaiting go/no-go (doc-only Phase 0) Opened: 2026-05-09 Scope: Spec an optional sqlite-vec backend that runs alongside the existing FTS5 retrieval pipeline (never replacing it), with phased gates on storage cost, embedder choice, and measured recall lift. Phase 0 is this doc — no code, no schema bump. Phase 1 (ingest-side embedding + VecBackend class) opens only when the storage projection in §3 lands within budget on a real shard. Phase 2 (hybrid fusion in query.py) opens only when Phase 1 measures ≥5pp recall lift on the bench fixtures with no STRICT-rate regression. Audience: fox + future blackops shifts + downstream Arborist clients deciding whether to ship a vec layer. Hard constraint: vec retrieval never enters proof path — hits land audit_mode=UNGROUNDED, same as FTS5 today (per the soft-hash / hard-hash separation in CLAUDE.md). No new cache_key dimension; embedder identity + dimensionality + quantization fold into governance_policy_hash like every other versioned default. Vec storage is additional to FTS5, not a replacement (see §4 for the architectural reasoning). Optional dep — pip install '.[vec]' — graceful fallback when extension is not loadable. Pre-v1 dep discipline: pin sqlite-vec==X.Y.Z and document the upgrade rule.

---

1. Problem statement

Arborist's retrieval today is four parallel FTS5 routes per shard merged then reranked (CLAUDE.md "Retrieval pipeline" §1):

body BM25  +  title LIKE  +  core-keyword (TF-IDF)  +  phrase-pattern

Each route closes a specific gap. Phrase-pattern (route 4) closes the lexical allusion gap — "always been at war" verbatim-matches the 1984 article whose title shares zero tokens with the query. There is no symmetric route that closes the semantic allusion gap — the case where the query and target chunk share zero stems but mean the same thing. Examples that today fall through every route:

query:  "the document store that proves what it answered"
target: an Arborist documentation chunk that uses "content-addressed",
        "Merkle-committed", "audit chain", "providence cache" — zero
        stem overlap with the query, no verbatim 5-gram phrase, no
        title-LIKE candidate, no TF-IDF core overlap.

query:  "what did Orwell call the country at war with Oceania?"
target: 1984 article using "Eastasia" / "Eurasia" — phrase route
        catches "always been at war"-style verbatim quotes but not
        synonymic re-phrasings of the same fact.

A vector embedding backend would close that gap by mapping query and chunk into a shared semantic space and returning top-K nearest. The question is whether the storage and compute cost pencils out, and whether vec retrieval adds to FTS5 or replaces it.

This ticket answers both questions with measured numbers, then specifies the integration surface so a Phase 1 implementation can proceed under a known budget.

---

2. Why sqlite-vec specifically (vs alternatives)

Arborist is SQLite-native by design. CLAUDE.md "Python only" rule forbids Rust / C / non-Python sibling indexes inside the repo; extension modules loaded by SQLite itself are fine because they appear to Python as ordinary sqlite3 rows.

Candidate	Verdict	Why
sqlite-vec	chosen	Pure C extension dlopen-able into stock sqlite3. Apache-2.0 / MIT dual-license. Same DB file as FTS5 — no second backing store. Matches the selectolax optional-dep precedent.
FAISS (sibling file)	rejected	Adds a second on-disk format outside the audit-chain. Two-file consistency is a new failure mode (cache_key talks about a single shard, not a shard + a vector blob).
pgvector (Postgres)	rejected	New runtime dependency. CLAUDE.md "fresh checkout needs only python3.12 + venv + sqlite3" rule.
Hand-rolled NumPy + flat scan	rejected	Works at small scale; doesn't survive 6.24M chunks (§3). No ANN escape hatch when corpus grows.
LanceDB / Chroma / Weaviate	rejected	Server runtimes. Fresh-checkout rule again.

Caveats on sqlite-vec:

• Pre-v1 (latest published v0.1.9). Repo explicitly warns "expect breaking changes." Pin tightly; treat upgrades as schema-bump events that re-embed every chunk.
• No formal max-dimension or transactional-write guarantees in upstream docs. Phase 0 deliverable: a 1k-chunk smoke test that measures (a) insert throughput, (b) WAL behavior, (c) crash recovery semantics. If any of those misbehave under WAL + synchronous=NORMAL (the existing arborist convention), Phase 1 is gated on a fix or a different config.

---

3. Storage hypothesis — the real 6 GB → 38 GB number

Fox's hypothesis: 6 GB compressed wiki blows up to ~35 GB after Merkle tree + FTS5 index. Measured today on ~/.arborist/shards/:

shard 000.db   9.6 GB    867 K docs   1.56 M chunks
shard 001.db   9.5 GB    868 K docs   1.56 M chunks
shard 002.db   9.6 GB    867 K docs   1.56 M chunks
shard 003.db   9.6 GB    867 K docs   1.56 M chunks
TOTAL         38.3 GB    3.47 M docs  6.24 M chunks

Within one shard the breakdown (SELECT name, SUM(pgsize) FROM dbstat GROUP BY name) is:

Component	MB	% of shard	Note
edges	3 670	38 %	Lossless supersedes / derives_from / related-doc links. 90.6 M total rows across the corpus — ~26 edges per document. Biggest single cost, bigger than chunks themselves.
chunks (compressed content)	2 649	28 %	Already zstd-compressed at rest.
chunks_fts_data (FTS5 inverted index)	1 501	16 %	The full lexical index.
idx_edges_dst_root	589	6 %	Edges secondary index.
audit_events	377	4 %	Hard-hash chain, one row per state-change.
documents	170	2 %	Document metadata.
chunks autoindex + leaf index	251	3 %
merkle_nodes	114	1 %	Internal nodes only — leaves are folded into chunks.leaf_hash.
concept_relations	10	0.1 %	Per-shard rivalry/synonym graph (#000018-area).
Everything else	~ 270	3 %

So the 6 GB → 38 GB blowup is not "FTS5 + Merkle tree." It is:

edges          ~38 %        relationship graph
chunks         ~28 %        compressed content (decompressed at query)
fts5           ~16 %        the actual inverted index
indexes        ~10 %        secondary indexes on the above
audit chain    ~4 %         hard-hash provenance receipts
merkle interior ~1 %        log-N internal nodes

The FTS5 index itself is only ~16 % of the shard. Most of the blowup is lossless provenance (edges + chunks + audit). Anyone proposing a vec backend should measure against that baseline, not against the FTS5 fraction.

3.1 Vec storage projection at 6.24 M chunks

sqlite-vec supports float32 / int8 / binary across configurable dimension. ANN index choice multiplies that. Using 6.24 M chunks (today's full corpus) and the bytes-per-chunk math:

Quantization	Dim	Bytes / chunk	Vec data total	+ Flat index	+ IVF (~1.1 ×)	+ DiskANN/HNSW (~1.7 ×)
float32	384	1 536	9.6 GB	9.6 GB	10.6 GB	16.3 GB
float32	768	3 072	19.2 GB	19.2 GB	21.1 GB	32.6 GB
int8	384	384	2.4 GB	2.4 GB	2.6 GB	4.1 GB
int8	768	768	4.8 GB	4.8 GB	5.3 GB	8.2 GB
binary	384	48	300 MB	300 MB	330 MB	510 MB
binary	768	96	600 MB	600 MB	660 MB	1.0 GB
binary	1024	128	800 MB	800 MB	880 MB	1.4 GB

Mapped onto the existing 38 GB shard footprint:

Config	% tax over 38 GB	Comparable to
float32 × 384 + flat	+ 25 %	Bigger than the entire FTS5 index. Reject.
float32 × 768 + DiskANN	+ 85 %	Doubles disk. Reject outright.
int8 × 384 + flat	+ 6 %	Comparable to a single secondary index. Acceptable.
int8 × 768 + flat	+ 13 %	Comparable to FTS5. Acceptable if recall justifies it.
binary × 384 + flat	+ 0.8 %	Same scale as concept_relations (1.6 % tax tolerated).
binary × 768 + flat	+ 1.6 %	Tied with concept_relations. Acceptable.
binary × 1024 + flat	+ 2.1 %	Acceptable.

Phase 0 storage budget: ≤ 15 % tax over today's 38 GB total (i.e. ≤ 5.7 GB of vec data across all shards). That admits everything from binary × 1024 up through int8 × 768 + flat. It excludes anything float32 and anything with DiskANN graph overhead above ~int8 × 384.

3.2 What this looks like per-shard

At the recommended Phase-1 config (int8 × 384 + flat as the default, configurable):

per shard:    1.56 M chunks × 384 bytes = 600 MB
all 4 shards: 2.4 GB total
% of shard:   ~6 %  (sits between idx_edges_dst_root and audit_events)

At the storage-conservative config (binary × 768 + flat):

per shard:    1.56 M chunks × 96 bytes = 150 MB
all 4 shards: 600 MB total
% of shard:   ~1.6 %  (same scale as concept_relations)

The Phase 0 deliverable picks one of these two as the default and makes the other reachable via flag.

---

4. Architectural answer — additional, not same

Q: would vector search be the same as FTS, or additional?

A: Additional. Always. Vec is a fifth retrieval route, not a replacement for any of the four FTS5 routes.

4.1 Why never replacement

The four FTS5 routes each close a gap that vec cannot:

Route	What it catches	Why vec misses it
Body BM25	Term-frequency-weighted topical relevance	Embeddings smooth over rare terms; specialized vocab gets averaged into the centroid
Title LIKE	Exact title-prefix matches	Embeddings of short titles are dominated by stopword centroids — "the X of Y" embeds nearly identically for many X, Y
Core-keyword (TF-IDF)	Distilled-doc anchors	Cores are a different artifact (distillation output), not a chunk-level signal
Phrase-pattern (n=5/n=6)	Verbatim quote allusion ("always been at war")	Embeddings are bag-of-information; phrase order is lost. Verbatim recall is FTS5's job.

Vec adds:

New route	What it catches
Vec ANN top-K	Semantic paraphrase ("country at war with Oceania" → "Eastasia"); cross-vocabulary synonymy when the corpus uses unfamiliar terminology; concept-level matching when the user's words and the article's words share no stems

So the architectural choice is fixed: vec is a fifth parallel route, merged into the existing rerank pipeline.

4.2 Hybrid fusion design

Two standard merge strategies, both feasible with the existing pipeline:

Option A — Reciprocal Rank Fusion (RRF). Each route emits ranked hits; final score is Σ_routes 1 / (k + rank_route) with k=60 per the original RRF paper. Robust to score-scale mismatch (BM25 scores are not comparable to cosine similarity). Recommended.

Option B — Score-level merge with normalization. Convert each route's score to a [0,1] percentile within its own distribution, then weighted-sum. More tunable but introduces a weights config that becomes a governance_policy_hash input. More moving parts than RRF.

Phase 0 picks Option A (RRF) as the default. Option B is a follow-up if RRF leaves measurable lift on the table.

4.3 Rivalry / title-relevance / phrase routes still apply

The post-merge filter chain (rivalry exclusion, four-accept-paths title-relevance, stem-aware token matching, per-source context cap) runs after the merge. Vec hits go through the same filter as FTS5 hits. A vec hit that passes ANN top-K but fails the title-relevance filter is dropped exactly like an FTS5 BM25 hit that fails the same filter — so the existing retrieval-driven hallucination guards (e.g. the spin-glass / QCD case from CLAUDE.md "Title-relevance hard check (Rule 8)") still hold.

---

5. The embedder is the gating item

Arborist's only LLM endpoint today is Hermes-3 Llama-3.1-8B at hermes.ai.unturf.com. Hermes is a chat model, not an embedding model. Without an embedder there is nothing to put into the vec table.

Phase 0 must pick one of these paths, in declining order of preference:

1. Local embedder bundled as optional dep. A small sentence-transformer (e.g. bge-small-en-v1.5, 384-dim; all-MiniLM-L6-v2, 384-dim) shipped via pip install '.[vec]'. Runs on CPU in arborist's process; no network call; no concurrency interaction with Hermes. ~100 MB model file. Strong default candidate because it preserves the "fresh checkout needs only python3.12 + venv + sqlite3" rule (the embed model is an optional extra, not a runtime dep).
2. Dedicated embedding endpoint. Stand up a sibling endpoint alongside Hermes (hermes-embed.ai.unturf.com?) running bge-large or similar. Frees arborist from CPU embedding work. Costs an extra service to operate.
3. Hermes itself for embeddings. Hijack a generative model for embeddings via prompt + last-hidden-state pooling. Possible but research-grade — bench quality unknown, breaks the ~4-concurrent bottleneck (#000037 §11) at every ingest. Not recommended.

Phase 0 deliverable: pick path 1 or 2 with a named model. Phase 1 implementation depends on which.

5.1 Why embed at ingest, never at query

Embedding cost is amortized:

ingest-time:  embed each chunk once,  store in vec table
query-time:   embed query string once (~10 ms on CPU),  ANN top-K

If the embedder lives in arborist's process (path 1), the only network cost at query time is the ANN scan inside SQLite — no LLM call. This sidesteps the Hermes ~4-concurrent bottleneck for queries entirely. Ingest pays the embedding cost up front, once per chunk, on the chunker-version-pinned content.

---

6. Versioning and governance_policy_hash integration

Embedder choice is a versioned default. Per CLAUDE.md "Versioned defaults" rule (tok-512-v1, norm-v1, wikitext-base-v1, v9.8.0), the vec layer adds:

embedder-name      e.g. "bge-small-en-v1.5"
embedder-version   e.g. "@hf-rev-deadbeef"
quantization       e.g. "int8" / "binary" / "float32"
dimension          e.g. 384 / 768 / 1024
ann-index          e.g. "flat" / "ivf" / "diskann"
distance-metric    e.g. "cosine" / "l2_normalized" / "dot" / "hamming"

These six fields fold into governance_policy_hash (NOT into a new cache_key dimension — keep the 8-dim invariant). Bumping any one stales every prior cached answer that consulted the vec backend, exactly as bumping the chunker stales every prior FTS5 hit. The vec table itself stays in place but is treated as "cold": query-time lookups filter on embedder_version = current and ignore older rows; a make rebuild-vec target re-embeds.

6.1 Distance metric — the silent-correctness field

Switching distance metric without re-embedding silently changes ranking. A query that returned chunk A as top-1 under cosine may return chunk B under L2 if the embeddings are not unit-normalized. Because nothing else in the pipeline observes the change (the vec backend just emits ranked Hits; the verifier and cache_key are ranking-blind), a metric-only flip would be invisible drift — exactly the failure mode CLAUDE.md "Versioned defaults" exists to prevent. Hence its inclusion in the policy hash.

The recommended canonical config:

1. embedder produces unit-normalized vectors    (||v|| = 1)
2. ingest stores vectors as-is                  (no further normalize)
3. query stores  metric = "l2_normalized"        (in policy hash)
4. SQL uses      vec_distance_l2()              (== cosine ranking,
                                                 no per-query norm,
                                                 SIMD-friendly)

metric = "cosine" (i.e. vec_distance_cosine() at query time) is the alternative when the embedder output is not guaranteed unit-normalized; pays a per-row norm computation in exchange for correctness regardless of input vector magnitude. Slower, never needed if §1 holds.

metric = "dot" is for pre-scaled int8 / float16 where unit normalization is impossible — the embedder writer is expected to have rescaled the vectors so dot-product ranks identically to cosine on the original float32 normalized vectors. Documented in the embedder version note; bump embedder-version if the rescale recipe changes.

metric = "hamming" is the only correct option for binary quantization — cosine on bit-vectors is undefined. Switching from binary + Hamming to int8 + L2 is a two-field policy bump (quantization AND distance-metric), and re-embeds nothing because the underlying float vectors weren't stored — make rebuild-vec must re-embed from source.

6.2 Quantization × metric compatibility matrix

Quantization	Valid metrics	Default
float32 (normalized)	l2_normalized, cosine	l2_normalized
float32 (unnormalized)	cosine	cosine
int8 (pre-scaled)	dot, l2_normalized	dot
binary	hamming	hamming

Phase 1 enforces the matrix at ingest: a config that pairs `binary

• cosine` errors out before any chunk is embedded. The error message names the matrix entry that would resolve it.

---

7. SearchBackend integration — where the code lands

Existing surface (arborist/search/base.py):

class SearchBackend(ABC):
    name: str
    audit_mode: AuditMode

    def search(self, query: str, limit: int = 20) -> list[Hit]: ...

Phase 1 adds arborist/search/vec.py mirroring fts5.py:

class VecBackend(SearchBackend):
    name = "vec"
    audit_mode = AuditMode.UNGROUNDED   # same as FTS5 — soft signal

    def __init__(self, conn, embedder):
        super().__init__(conn)
        self.embedder = embedder    # callable: str -> np.ndarray

    def search(self, query, limit=20):
        qv = self.embedder(query)
        rows = self.conn.execute(
            "SELECT chunk_id, distance FROM chunk_vecs "
            "WHERE embedding MATCH ? AND k = ? "
            "ORDER BY distance",
            (qv.tobytes(), limit),
        ).fetchall()
        # JOIN back to chunks + documents, build Hits with the same
        # _build_snippet helper FTS5 already uses.
        ...

The merge with FTS5 happens in arborist/qa/query.py where the four FTS5 routes already merge today. RRF is a six-line addition once both rankings exist.

Optional-dep guard at import time (mirrors [html]):

try:
    import sqlite_vec
    _VEC_AVAILABLE = True
except ImportError:
    _VEC_AVAILABLE = False

CLI surface only exposes --retrieval=vec or --retrieval=hybrid when _VEC_AVAILABLE, otherwise the flag errors with an install hint. Same pattern as selectolax for the HTML source.

---

8. Phased plan with measurement gates

Phase 0 — this ticket (doc)

• Decide embedder (path 1 or path 2, named model).
• Decide default quantization + dimension (recommendation: int8 × 384 + flat as the speed/quality default; binary × 768 + flat as the storage-conservative alternative).
• Run the 1k-chunk smoke test under WAL + synchronous=NORMAL.
• Write the bench protocol: which fixtures, what signal floor. Lift target: ≥ 5pp recall@20 on bench-emergent random-word fixtures (bench-maxing.md signal floor).

Phase 1 — code (gated on Phase 0 sign-off)

• arborist/search/vec.py with optional-dep guard.
• chunk_vecs virtual table created lazily on first ingest with --embed.
• arborist ingest --embed ingest-side flag (default off).
• make rebuild-vec Makefile target.
• Rebuilds populate chunk_vecs for all current chunks rows.
• Bench: vec-only retrieval vs FTS5-only on a single shard. Phase 1 succeeds if vec-only matches FTS5-only within ±5pp on STRICT-rate AND beats FTS5 by ≥ 5pp on at least one bench-emergent semantic-allusion fixture.

Phase 2 — hybrid fusion (gated on Phase 1 success)

• RRF merge in arborist/qa/query.py between FTS5 routes and vec route.
• Bench: hybrid vs FTS5-only.
• Phase 2 succeeds if hybrid lifts STRICT-rate by ≥ 5pp over FTS5-only with no UNGROUNDED-rate regression.
• If Phase 2 fails by < 5pp lift: park the vec layer as opt-in per-call (--retrieval=vec) but do not make hybrid the default. The 2.4 GB tax is not paid for a sub-signal-floor lift.

Phase 3 — governance_policy_hash wiring (mechanical)

• Fold the five vec-version fields (§6) into the policy hash.
• Document in CLAUDE.md "Versioned defaults" and "Schema invariants" sections.
• Update make chain-check-shards semantics if needed (it should not need changes — vec data is sibling, like concept_relations).

---

9. Decision tree on quantization

Phase 0 budget: ≤ 15 % storage tax (≤ 5.7 GB across all shards).

If Phase 1 bench shows int8 × 384 hits the recall target
   → ship int8 × 384 + flat as default.
   → 6 % storage tax. Best speed/quality for the budget.

If Phase 1 bench shows int8 × 384 misses the target
   AND int8 × 768 hits it
   → ship int8 × 768 + flat.
   → 13 % storage tax. Borderline acceptable.

If Phase 1 bench shows int8 × 768 also misses
   → DO NOT ship float32 anywhere — too expensive.
   → Park the ticket. Reopen when a smaller, sharper embedder lands
     (research-bench problem, not an arborist problem).

If storage budget tightens later
   → fall back to binary × 768 + flat. < 2 % tax,
     ~10–20 % recall hit, still wins on the semantic-allusion
     fixtures the lexical pipeline misses entirely.

---

10. Risks and open questions

1. sqlite-vec pre-v1 instability. Mitigation: pin tightly, treat upgrades as schema-bump events, run the 1k-chunk smoke test after every dep bump. Phase 0 deliverable.
2. Embedder model drift. A new HuggingFace revision of bge-small invalidates every prior embedding. Mitigation: pin model+revision via embedder_version in §6, track upgrades under make rebuild-vec.
3. Re-embed cost on chunker bump. tok-512-v1 → v2 already stales every FTS5 chunk. Adds re-embed work proportional to chunk count. At 6.24 M chunks × ~10 ms/chunk on a CPU bge-small ≈ 17 hours single-threaded; parallelizes trivially. Already a known-large cost; vec adds ~10–20 % to it.
4. Verifier semantics unchanged. Vec lifts retrieval recall; the lexical verifier (quote / span / entity / paraphrase) stays binary. A vec-retrieved chunk that the verifier cannot ground still lands UNGROUNDED. Worth confirming on the bench: does vec recall translate to STRICT-rate lift, or just to more UNGROUNDED hits being dropped at verification time?
5. Title-relevance hard check (Rule 8) interaction. Vec is precisely the retrieval mode most likely to surface structurally-unrelated cited chunks (the spin-glass / QCD case). Phase 1 bench must include the existing title-mismatch fixtures and confirm vec hits get caught by Rule 8 like any other retrieval-driven hallucination candidate. If vec hits bypass Rule 8 somehow, that is a Phase 1 blocker.
6. edges is the dominant cost (38 % per §3), not FTS5. Confirmed with the corpus-wide row count: 90.6 M edges across 3.47 M documents = ~26 edges/document. If storage matters more than retrieval quality, compacting edges is a higher-leverage optimization than anything vec-related (a single int8 × 384 vec layer adds 6 % tax; trimming edges by 10 % saves 3.8 % at zero quality cost). Out of scope for this ticket but worth a reminder when we benchmark.
7. Concurrent-Hermes bottleneck (#000037 §11) is irrelevant here as long as we use a local embedder (path 1 in §5). If we go path 2 or 3, vec ingest competes with inference traffic and the Prometheus-Σ controller (#000037) would need to throttle ingest under load.

---

11. What this ticket does NOT do

• Does not commit to shipping a vec backend. Phase 0 is doc-only; every later phase has a measurable gate.
• Does not bump cache_key dimensions. Vec config folds into governance_policy_hash, same pattern as every prior versioned default.
• Does not change the verifier or the audit chain. Vec is a soft signal at the retrieval layer only; proof-path stays identical.
• Does not replace any FTS5 route. §4 settles this: vec is a fifth route, never a substitute.
• Does not implement the unconscious falsification sweep (#000037 Phase 1). That is a separate ticket; vec just makes semantic neighbors easier to surface, which is one capability the sweep can use later.

---

12. References

• This ticket: doc-only Phase 0.
• arborist/search/base.py — SearchBackend ABC, the integration surface.
• arborist/search/fts5.py — the contract the vec backend mirrors.
• arborist/qa/query.py — where RRF merge would land.
• CLAUDE.md "Retrieval pipeline" — the four-route baseline.
• CLAUDE.md "Soft hash vs hard hash" — the rule that licenses vec retrieval as long as it never enters the proof path.
• CLAUDE.md "Versioned defaults" — the pattern §6 follows.
• #000018 (concept_relations) — the 1.6 % storage-tax precedent.
• #000037 §11 — the Hermes ~4-concurrent constraint that drives the "embed at ingest, not at query" rule.
• ~/.arborist/shards/ — 4 × 9.6 GB shards measured 2026-05-09; the per-table breakdown in §3 is dbstat-derived from 000.db.
• Upstream: https://github.com/asg017/sqlite-vec (v0.1.9 at time of writing; expect breaking changes pre-v1).

---

13. Phase 1 implementation plan (proposal, 2026-05-10)

Status: awaiting fox go/no-go on the four §13.1 decisions below. Phase 1 code does not start until they're settled.

13.1 Phase-0 deliverables — proposed picks

Four explicit decisions Phase 0 left open. Recommendations:

1. Embedder path → Path 1 (local sentence-transformer bundled as optional dep). Preserves CLAUDE.md's "fresh checkout needs only python3.12 + venv + sqlite3" rule; sidesteps the Hermes 4-concurrent bottleneck (#000037 §11) entirely; no new service to operate. Model: BAAI/bge-small-en-v1.5 — 384-dim, ~33 MB, MIT-licensed, top of MTEB-en/retrieval among sub-100MB models, unit-normalized output (matches §6.1 canonical config).
2. Default quantization → int8 × 384 + flat per the §9 decision tree's primary leg. 6 % storage tax (~2.4 GB across the 4-shard cluster), well within the 15 % budget. Speed/quality default. Storage-conservative alternative binary × 768 + flat (1.6 % tax) reachable via --vec-quantization=binary --vec-dim=768.
3. 1k-chunk smoke test → §13.2 below.
4. Bench protocol → §13.3 below.

13.2 Pre-flight smoke (gate before any Phase 1 code lands)

make smoke-vec     # new make target, ~2 min wall

What it does:

• Pull 1k random chunks.content rows from ~/.arborist/shards/000.db into /tmp/smoke-vec.db (read-only on the source shard).
• Embed with bge-small-en-v1.5 on CPU (single-threaded; ~10 ms/chunk → ~10 s wall).
• INSERT into chunk_vecs virtual table under WAL + synchronous=NORMAL (existing arborist convention).
• Kill -9 mid-insert (process group); reopen; check WAL recovery + journal mode + row count consistency.
• Run 100 query embeddings × top-20 ANN. Measure p50/p95 query latency.

Gate criteria: insert throughput ≥ 100 chunk/s, p95 query latency ≤ 50 ms, zero data loss across the kill -9 boundary. Fail on any of those → Phase 1 blocked, surface to fox + upstream issue.

13.3 Bench protocol

Fixtures (all already in bench/):

• bench/qa_questions_smoke.txt — 5-question fast loop
• bench/qa_questions_progressive_and.txt — progressive-AND fallback fixture
• Existing bench-emergent random-word stress fixture
• Existing qa-modes-bench fixture for STRICT-rate guard

Three conditions:

condition	retrieval routes	claim_lattice mode
Baseline	FTS5 only (current 4 routes)	claim_lattice (JSON)
Vec-only	Vec ANN top-20 only	claim_lattice (JSON)
Hybrid	FTS5 + Vec, RRF merge (k=60)	claim_lattice (JSON)

Phase 1 success criteria (from §8):

• Vec-only matches FTS5-only within ±5pp STRICT-rate (sanity)
• AND beats FTS5-only by ≥5pp on at least one semantic-allusion bench-emergent fixture
• AND no UNGROUNDED-rate regression vs baseline

Phase 2 success criteria (gates on §8):

• Hybrid lifts STRICT-rate by ≥5pp over baseline
• AND no UNGROUNDED-rate regression
• AND title-relevance hard check (Rule 8) catches mismatched vec hits at the same rate it catches mismatched FTS5 hits

13.4 Code structure

arborist/embed.py                 [NEW]
    class Embedder:
        - lazy-loads BAAI/bge-small-en-v1.5 once per process
        - normalize=True (unit vectors, §6.1 canonical config)
        - encode(texts: list[str]) -> np.ndarray[N, 384] dtype=int8
        - sentencetransformer dependency under [vec] extras

arborist/search/vec.py            [NEW, mirrors fts5.py]
    class VecBackend(SearchBackend):
        - audit_mode = AuditMode.UNGROUNDED  # §1 hard constraint
        - chunk_vecs virtual table creation (lazy; first ingest --embed)
        - search(query, limit=20) -> list[Hit] via vec_distance_l2

arborist/store.py                 [PATCH]
    - SCHEMA_SQL: NO new always-on tables (vec table is virtual,
      created only when --embed runs)
    - Add embedder_version, vec_quantization, vec_dim, vec_ann_index,
      vec_distance_metric to governance_policy_hash inputs (NOT
      cache_key — §6 hard constraint)

arborist/qa/query.py              [PATCH]
    - if vec backend present: run RRF merge over (FTS5_routes_merged,
      vec_route) with k=60. New code is ~6 lines per §7.

arborist/cli.py                   [PATCH]
    - ingest --embed flag (gated on _VEC_AVAILABLE import-check)
    - query --retrieval={fts5|vec|hybrid} flag
    - new subcommand: arborist vec rebuild [--db ... --batch N]

Makefile                          [PATCH]
    smoke-vec:        # §13.2 pre-flight; gate-before-Phase-1
    rebuild-vec:      # populate chunk_vecs for an existing shard
    bench-vec:        # run §13.3 three-condition bench

pyproject.toml                    [PATCH]
    [project.optional-dependencies]
    vec = [
        "sqlite-vec>=0.1.9,<0.2",
        "sentence-transformers>=2.7,<3",
        "numpy",
    ]

13.5 Test plan

Unit tests:

• tests/test_embed.py — deterministic output for fixed input (regression guard against silent model drift between revs); unit-norm verification; int8 round-trip.
• tests/test_search_vec.py — VecBackend.search against a 3-chunk in-memory shard with hand-crafted embeddings; verify ranking matches expected; verify audit_mode=UNGROUNDED; verify _VEC_AVAILABLE=False graceful degradation.
• tests/test_governance_policy.py — verify the 5 new vec-config fields fold into governance_policy_hash; flipping any one invalidates prior cached records.

CLI surface tests (per docs/calculator-test-patterns.md §6 — subprocess invocation catches argparse + main() drift the import-only tests miss; this hazard surfaced three times during the 2026-05-10 substrate rename refactor in yesterday's commit chain, so we pin the pattern):

• tests/test_cli_vec_rebuild.py — subprocess invocation of arborist vec rebuild --db <tmp_path/test.db> --batch 4 against a 12-chunk synthetic shard; assert exit 0, JSON status output if any, row-count parity in chunk_vecs. Skipped via pytest.importorskip when [vec] extras absent.
• tests/test_cli_ingest_embed_flag.py — subprocess invocation of arborist ingest --source html --embed --author X against a tmp shard; assert --embed flag wires through to the embedder; row count after ingest matches; vec rows match chunk rows 1:1.
• tests/test_cli_query_retrieval_flag.py — subprocess invocation of arborist query --retrieval=vec and --retrieval=hybrid and --retrieval=fts5 against a pre-embedded shard fixture; assert all three exit 0 and produce structurally-distinct retrieval candidate sets (vec ≠ fts5 on at least one query).

Integration tests (gated on [vec] extras installed; skipped otherwise):

• tests/test_vec_smoke_integration.py — runs the §13.2 smoke on a 100-chunk synthetic shard (smaller than 1k for CI speed); asserts WAL recovery + row count parity.

13.6 Out of scope for Phase 1

• ANN index variants (IVF, DiskANN, HNSW) — Phase 1 ships flat only. ANN is opt-in via vec_ann_index policy field; flat works at ≤10 M chunks per the upstream sqlite-vec docs.
• Re-embed cost on chunker bump — already a known-large cost (§10.3); covered by make rebuild-vec not Phase 1.
• Edges compaction (§10.6) — different ticket entirely.

13.7 Estimated size

• New files: 2 (embed.py, search/vec.py) ~ 200 LOC.
• Patches: 4 (store.py, query.py, cli.py, Makefile) ~ 100 LOC.
• Tests: 7 files ~ 400 LOC + fixtures (4 unit + 3 CLI subprocess per the §13.5 amendment 2026-05-10 + 1 integration). The CLI subprocess tests are ~30-40 LOC each — boilerplate + tmp_path
  • subprocess.run + JSON parse.
• Phase-1 doc append in docs/: this §13 + a journey-note bench result file.
• pyproject.toml: 1 stanza.

Total: ~550 LOC + ~400 test LOC. Single substantial commit if all tests pass + smoke succeeds; otherwise broken into the natural gates (smoke → embed.py → vec.py → CLI subprocess tests → integration tests → bench).

13.8 Decisions fox needs to make to unblock Phase 1

#	Decision	Recommendation
1	Embedder path 1 vs 2 vs 3	Path 1 (local bundle)
2	Default model name	BAAI/bge-small-en-v1.5 (MIT, 33 MB)
3	Default quantization × dim	int8 × 384 + flat
4	Approve sentence-transformers PyPI dep under [vec]	yes (~120 MB install footprint when extras pulled; default not pulled — pip install '.[vec]' only)

If fox approves all four → Phase 1 starts with §13.2 smoke as the first gate.

If fox rejects path 1 → Phase 1 blocked on standing up hermes-embed.ai.unturf.com (path 2; outside this ticket).

If fox rejects int8 × 384 default → propose binary × 768 as default (1.6 % tax, ~10–20 % recall hit per §9). Either is defensible.

If fox rejects the dep footprint → fall back to the lighter onnxruntime path with a bge-small ONNX model (still ~33 MB weights but smaller runtime). Adds complexity but keeps the no-server constraint.