engineering/arborist
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

ticket #000039: sqlite-vec optional backend (parallel-shift orphan landed)

Browse source 
Parallel-shift session drafted #000039 earlier today and updated
TICKETS.md with the index row, but the ticket file itself sat
untracked in working tree (same situation #000037 had until that
ticket landed in commit 178cc42).

Committing the file as-drafted by the original author so the
design log entry is intact. No content changes from this
session; the file is exactly what the parallel shift produced.

Per the design log convention in TICKETS.md ("Do not delete
tickets; they are the design log") — every opened ticket file
ships with its index row.
This commit is contained in:
russell@unturf.com 2026-05-09 19:19:44 -04:00
parent 178cc422e2
commit 565f763967
No known key found for this signature in database
1 changed files with 596 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

596
docs/tickets/ticket-000039-sqlite-vec-optional-backend.md Normal file
View file

@ -0,0 +1,596 @@
# Ticket #000039 — Optional `sqlite-vec` retrieval backend (A/B vs FTS5, hybrid not replacement)
**Status:** open · awaiting go/no-go (doc-only Phase 0)
**Opened:** 2026-05-09
**Scope:** Spec an optional `sqlite-vec` backend that runs **alongside** the
existing FTS5 retrieval pipeline (never replacing it), with phased gates
on storage cost, embedder choice, and measured recall lift. Phase 0 is
this doc — no code, no schema bump. Phase 1 (ingest-side embedding +
`VecBackend` class) opens only when the storage projection in §3 lands
within budget on a real shard. Phase 2 (hybrid fusion in `query.py`)
opens only when Phase 1 measures ≥5pp recall lift on the bench fixtures
with no STRICT-rate regression.
**Audience:** fox + future blackops shifts + downstream Arborist
clients deciding whether to ship a vec layer.
**Hard constraint:** vec retrieval **never enters proof path** — hits
land `audit_mode=UNGROUNDED`, same as FTS5 today (per the soft-hash /
hard-hash separation in CLAUDE.md). No new `cache_key` dimension;
embedder identity + dimensionality + quantization fold into
`governance_policy_hash` like every other versioned default. Vec
storage is **additional** to FTS5, not a replacement (see §4 for the
architectural reasoning). Optional dep — `pip install '.[vec]'` —
graceful fallback when extension is not loadable. Pre-v1 dep
discipline: pin `sqlite-vec==X.Y.Z` and document the upgrade rule.
---
## 1. Problem statement
Arborist's retrieval today is four parallel FTS5 routes per shard
merged then reranked (CLAUDE.md "Retrieval pipeline" §1):
```text
body BM25 + title LIKE + core-keyword (TF-IDF) + phrase-pattern
```
Each route closes a specific gap. Phrase-pattern (route 4) closes the
**lexical** allusion gap — "always been at war" verbatim-matches the
1984 article whose title shares zero tokens with the query. There is
no symmetric route that closes the **semantic** allusion gap — the
case where the query and target chunk share zero stems but mean the
same thing. Examples that today fall through every route:
```text
query: "the document store that proves what it answered"
target: an Arborist documentation chunk that uses "content-addressed",
"Merkle-committed", "audit chain", "providence cache" — zero
stem overlap with the query, no verbatim 5-gram phrase, no
title-LIKE candidate, no TF-IDF core overlap.
query: "what did Orwell call the country at war with Oceania?"
target: 1984 article using "Eastasia" / "Eurasia" — phrase route
catches "always been at war"-style verbatim quotes but not
synonymic re-phrasings of the same fact.
```
A vector embedding backend would close that gap by mapping query and
chunk into a shared semantic space and returning top-K nearest. The
question is whether the storage and compute cost pencils out, and
whether vec retrieval **adds** to FTS5 or **replaces** it.
This ticket answers both questions with measured numbers, then
specifies the integration surface so a Phase 1 implementation can
proceed under a known budget.
---
## 2. Why `sqlite-vec` specifically (vs alternatives)
Arborist is SQLite-native by design. CLAUDE.md "Python only" rule
forbids Rust / C / non-Python sibling indexes inside the repo;
extension modules loaded by SQLite itself are fine because they
appear to Python as ordinary `sqlite3` rows.
| Candidate | Verdict | Why |
|---|---|---|
| `sqlite-vec` | **chosen** | Pure C extension dlopen-able into stock `sqlite3`. Apache-2.0 / MIT dual-license. Same DB file as FTS5 — no second backing store. Matches the `selectolax` optional-dep precedent. |
| FAISS (sibling file) | rejected | Adds a second on-disk format outside the audit-chain. Two-file consistency is a new failure mode (cache_key talks about a single shard, not a shard + a vector blob). |
| `pgvector` (Postgres) | rejected | New runtime dependency. CLAUDE.md "fresh checkout needs only python3.12 + venv + sqlite3" rule. |
| Hand-rolled NumPy + flat scan | rejected | Works at small scale; doesn't survive 6.24M chunks (§3). No ANN escape hatch when corpus grows. |
| LanceDB / Chroma / Weaviate | rejected | Server runtimes. Fresh-checkout rule again. |
Caveats on `sqlite-vec`:
- Pre-v1 (latest published v0.1.9). Repo explicitly warns "expect
breaking changes." Pin tightly; treat upgrades as schema-bump
events that re-embed every chunk.
- No formal max-dimension or transactional-write guarantees in
upstream docs. Phase 0 deliverable: a 1k-chunk smoke test that
measures (a) insert throughput, (b) WAL behavior, (c) crash
recovery semantics. If any of those misbehave under WAL +
`synchronous=NORMAL` (the existing arborist convention), Phase 1
is gated on a fix or a different config.
---
## 3. Storage hypothesis — the real 6 GB → 38 GB number
Fox's hypothesis: **6 GB compressed wiki blows up to ~35 GB after
Merkle tree + FTS5 index.** Measured today on `~/.arborist/shards/`:
```text
shard 000.db 9.6 GB 867 K docs 1.56 M chunks
shard 001.db 9.5 GB 868 K docs 1.56 M chunks
shard 002.db 9.6 GB 867 K docs 1.56 M chunks
shard 003.db 9.6 GB 867 K docs 1.56 M chunks
TOTAL 38.3 GB 3.47 M docs 6.24 M chunks
```
Within one shard the breakdown (`SELECT name, SUM(pgsize) FROM dbstat
GROUP BY name`) is:
| Component | MB | % of shard | Note |
|---|---|---|---|
| `edges` | 3 670 | 38 % | Lossless `supersedes` / `derives_from` / related-doc links. **90.6 M total rows across the corpus — ~26 edges per document.** Biggest single cost, bigger than chunks themselves. |
| `chunks` (compressed content) | 2 649 | 28 % | Already zstd-compressed at rest. |
| `chunks_fts_data` (FTS5 inverted index) | 1 501 | 16 % | The full lexical index. |
| `idx_edges_dst_root` | 589 | 6 % | Edges secondary index. |
| `audit_events` | 377 | 4 % | Hard-hash chain, one row per state-change. |
| `documents` | 170 | 2 % | Document metadata. |
| `chunks` autoindex + leaf index | 251 | 3 % | |
| `merkle_nodes` | 114 | 1 % | Internal nodes only — leaves are folded into `chunks.leaf_hash`. |
| `concept_relations` | 10 | 0.1 % | Per-shard rivalry/synonym graph (#000018-area). |
| Everything else | ~ 270 | 3 % | |
So the 6 GB → 38 GB blowup is **not** "FTS5 + Merkle tree." It is:
```text
edges ~38 % relationship graph
chunks ~28 % compressed content (decompressed at query)
fts5 ~16 % the actual inverted index
indexes ~10 % secondary indexes on the above
audit chain ~4 % hard-hash provenance receipts
merkle interior ~1 % log-N internal nodes
```
The FTS5 index itself is **only ~16 %** of the shard. Most of the
blowup is **lossless provenance** (edges + chunks + audit). Anyone
proposing a vec backend should measure against that baseline, not
against the FTS5 fraction.
### 3.1 Vec storage projection at 6.24 M chunks
`sqlite-vec` supports float32 / int8 / binary across configurable
dimension. ANN index choice multiplies that. Using 6.24 M chunks
(today's full corpus) and the bytes-per-chunk math:
| Quantization | Dim | Bytes / chunk | Vec data total | + Flat index | + IVF (~1.1 ×) | + DiskANN/HNSW (~1.7 ×) |
|---|---|---|---|---|---|---|
| float32 | 384 | 1 536 | 9.6 GB | 9.6 GB | 10.6 GB | 16.3 GB |
| float32 | 768 | 3 072 | 19.2 GB | 19.2 GB | 21.1 GB | 32.6 GB |
| int8 | 384 | 384 | 2.4 GB | 2.4 GB | 2.6 GB | 4.1 GB |
| int8 | 768 | 768 | 4.8 GB | 4.8 GB | 5.3 GB | 8.2 GB |
| binary | 384 | 48 | 300 MB | 300 MB | 330 MB | 510 MB |
| binary | 768 | 96 | 600 MB | 600 MB | 660 MB | 1.0 GB |
| binary | 1024 | 128 | 800 MB | 800 MB | 880 MB | 1.4 GB |
Mapped onto the existing 38 GB shard footprint:
| Config | % tax over 38 GB | Comparable to |
|---|---|---|
| float32 × 384 + flat | + 25 % | Bigger than the entire FTS5 index. Reject. |
| float32 × 768 + DiskANN | + 85 % | Doubles disk. Reject outright. |
| **int8 × 384 + flat** | **+ 6 %** | **Comparable to a single secondary index. Acceptable.** |
| int8 × 768 + flat | + 13 % | Comparable to FTS5. Acceptable if recall justifies it. |
| **binary × 384 + flat** | **+ 0.8 %** | **Same scale as `concept_relations` (1.6 % tax tolerated).** |
| binary × 768 + flat | + 1.6 % | Tied with `concept_relations`. Acceptable. |
| binary × 1024 + flat | + 2.1 % | Acceptable. |
**Phase 0 storage budget:** ≤ 15 % tax over today's 38 GB total
(i.e. ≤ 5.7 GB of vec data across all shards). That admits
everything from binary × 1024 up through int8 × 768 + flat. It
excludes anything float32 and anything with DiskANN graph overhead
above ~int8 × 384.
### 3.2 What this looks like per-shard
At the recommended Phase-1 config (**int8 × 384 + flat** as the
default, configurable):
```text
per shard: 1.56 M chunks × 384 bytes = 600 MB
all 4 shards: 2.4 GB total
% of shard: ~6 % (sits between idx_edges_dst_root and audit_events)
```
At the storage-conservative config (**binary × 768 + flat**):
```text
per shard: 1.56 M chunks × 96 bytes = 150 MB
all 4 shards: 600 MB total
% of shard: ~1.6 % (same scale as concept_relations)
```
The Phase 0 deliverable picks one of these two as the default and
makes the other reachable via flag.
---
## 4. Architectural answer — additional, not same
> **Q: would vector search be the same as FTS, or additional?**
>
> A: **Additional. Always.** Vec is a fifth retrieval route,
> not a replacement for any of the four FTS5 routes.
### 4.1 Why never replacement
The four FTS5 routes each close a gap that vec cannot:
| Route | What it catches | Why vec misses it |
|---|---|---|
| Body BM25 | Term-frequency-weighted topical relevance | Embeddings smooth over rare terms; specialized vocab gets averaged into the centroid |
| Title LIKE | Exact title-prefix matches | Embeddings of short titles are dominated by stopword centroids — "the X of Y" embeds nearly identically for many X, Y |
| Core-keyword (TF-IDF) | Distilled-doc anchors | Cores are a different artifact (distillation output), not a chunk-level signal |
| Phrase-pattern (n=5/n=6) | Verbatim quote allusion ("always been at war") | Embeddings are bag-of-information; phrase order is lost. Verbatim recall is FTS5's job. |
Vec adds:
| New route | What it catches |
|---|---|
| Vec ANN top-K | Semantic paraphrase ("country at war with Oceania" → "Eastasia"); cross-vocabulary synonymy when the corpus uses unfamiliar terminology; concept-level matching when the user's words and the article's words share no stems |
So the architectural choice is fixed: **vec is a fifth parallel
route**, merged into the existing rerank pipeline.
### 4.2 Hybrid fusion design
Two standard merge strategies, both feasible with the existing
pipeline:
**Option A — Reciprocal Rank Fusion (RRF).** Each route emits
ranked hits; final score is `Σ_routes 1 / (k + rank_route)` with
`k=60` per the original RRF paper. Robust to score-scale
mismatch (BM25 scores are not comparable to cosine similarity).
Recommended.
**Option B — Score-level merge with normalization.** Convert each
route's score to a `[0,1]` percentile within its own distribution,
then weighted-sum. More tunable but introduces a `weights` config
that becomes a `governance_policy_hash` input. More moving parts
than RRF.
Phase 0 picks **Option A (RRF)** as the default. Option B is a
follow-up if RRF leaves measurable lift on the table.
### 4.3 Rivalry / title-relevance / phrase routes still apply
The post-merge filter chain (rivalry exclusion, four-accept-paths
title-relevance, stem-aware token matching, per-source context cap)
runs **after** the merge. Vec hits go through the same filter as
FTS5 hits. A vec hit that passes ANN top-K but fails the
title-relevance filter is dropped exactly like an FTS5 BM25 hit
that fails the same filter — so the existing retrieval-driven
hallucination guards (e.g. the spin-glass / QCD case from
CLAUDE.md "Title-relevance hard check (Rule 8)") still hold.
---
## 5. The embedder is the gating item
Arborist's only LLM endpoint today is Hermes-3 Llama-3.1-8B at
`hermes.ai.unturf.com`. Hermes is a chat model, **not** an
embedding model. Without an embedder there is nothing to put into
the vec table.
Phase 0 must pick one of these paths, in declining order of
preference:
1. **Local embedder bundled as optional dep.** A small
sentence-transformer (e.g. `bge-small-en-v1.5`, 384-dim;
`all-MiniLM-L6-v2`, 384-dim) shipped via `pip install '.[vec]'`.
Runs on CPU in arborist's process; no network call; no
concurrency interaction with Hermes. ~100 MB model file. **Strong
default candidate** because it preserves the "fresh checkout
needs only python3.12 + venv + sqlite3" rule (the embed model
is an optional extra, not a runtime dep).
2. **Dedicated embedding endpoint.** Stand up a sibling endpoint
alongside Hermes (`hermes-embed.ai.unturf.com`?) running
`bge-large` or similar. Frees arborist from CPU embedding
work. Costs an extra service to operate.
3. **Hermes itself for embeddings.** Hijack a generative model for
embeddings via prompt + last-hidden-state pooling. Possible but
research-grade — bench quality unknown, breaks the ~4-concurrent
bottleneck (#000037 §11) at every ingest. **Not recommended.**
Phase 0 deliverable: pick path 1 or 2 with a named model. Phase 1
implementation depends on which.
### 5.1 Why embed at ingest, never at query
Embedding cost is amortized:
```text
ingest-time: embed each chunk once, store in vec table
query-time: embed query string once (~10 ms on CPU), ANN top-K
```
If the embedder lives in arborist's process (path 1), the only
network cost at query time is the ANN scan inside SQLite — no LLM
call. This sidesteps the Hermes ~4-concurrent bottleneck for
queries entirely. Ingest pays the embedding cost up front, once
per chunk, on the chunker-version-pinned content.
---
## 6. Versioning and `governance_policy_hash` integration
Embedder choice is a versioned default. Per CLAUDE.md "Versioned
defaults" rule (`tok-512-v1`, `norm-v1`, `wikitext-base-v1`,
`v9.8.0`), the vec layer adds:
```text
embedder-name e.g. "bge-small-en-v1.5"
embedder-version e.g. "@hf-rev-deadbeef"
quantization e.g. "int8" / "binary" / "float32"
dimension e.g. 384 / 768 / 1024
ann-index e.g. "flat" / "ivf" / "diskann"
distance-metric e.g. "cosine" / "l2_normalized" / "dot" / "hamming"
```
These **six** fields fold into `governance_policy_hash` (NOT into
a new cache_key dimension — keep the 8-dim invariant). Bumping any
one stales every prior cached answer that consulted the vec
backend, exactly as bumping the chunker stales every prior FTS5
hit. The vec table itself stays in place but is treated as
"cold": query-time lookups filter on `embedder_version =
current` and ignore older rows; a `make rebuild-vec` target
re-embeds.
### 6.1 Distance metric — the silent-correctness field
Switching distance metric without re-embedding silently changes
ranking. A query that returned chunk A as top-1 under cosine may
return chunk B under L2 if the embeddings are not unit-normalized.
Because nothing else in the pipeline observes the change (the vec
backend just emits ranked Hits; the verifier and cache_key are
ranking-blind), a metric-only flip would be **invisible drift** —
exactly the failure mode CLAUDE.md "Versioned defaults" exists to
prevent. Hence its inclusion in the policy hash.
The recommended canonical config:
```text
1. embedder produces unit-normalized vectors (||v|| = 1)
2. ingest stores vectors as-is (no further normalize)
3. query stores metric = "l2_normalized" (in policy hash)
4. SQL uses vec_distance_l2() (== cosine ranking,
no per-query norm,
SIMD-friendly)
```
`metric = "cosine"` (i.e. `vec_distance_cosine()` at query time)
is the alternative when the embedder output is **not** guaranteed
unit-normalized; pays a per-row norm computation in exchange for
correctness regardless of input vector magnitude. Slower, never
needed if §1 holds.
`metric = "dot"` is for pre-scaled int8 / float16 where unit
normalization is impossible — the embedder writer is expected to
have rescaled the vectors so dot-product **ranks** identically to
cosine on the original float32 normalized vectors. Documented in
the embedder version note; bump `embedder-version` if the rescale
recipe changes.
`metric = "hamming"` is the only correct option for binary
quantization — cosine on bit-vectors is undefined. Switching from
binary + Hamming to int8 + L2 is a **two-field policy bump**
(quantization AND distance-metric), and re-embeds nothing because
the underlying float vectors weren't stored — `make rebuild-vec`
must re-embed from source.
### 6.2 Quantization × metric compatibility matrix
| Quantization | Valid metrics | Default |
|---|---|---|
| float32 (normalized) | `l2_normalized`, `cosine` | `l2_normalized` |
| float32 (unnormalized) | `cosine` | `cosine` |
| int8 (pre-scaled) | `dot`, `l2_normalized` | `dot` |
| binary | `hamming` | `hamming` |
Phase 1 enforces the matrix at ingest: a config that pairs `binary
+ cosine` errors out before any chunk is embedded. The error
message names the matrix entry that would resolve it.
---
## 7. SearchBackend integration — where the code lands
Existing surface (`arborist/search/base.py`):
```python
class SearchBackend(ABC):
name: str
audit_mode: AuditMode
def search(self, query: str, limit: int = 20) -> list[Hit]: ...
```
Phase 1 adds `arborist/search/vec.py` mirroring `fts5.py`:
```python
class VecBackend(SearchBackend):
name = "vec"
audit_mode = AuditMode.UNGROUNDED # same as FTS5 — soft signal
def __init__(self, conn, embedder):
super().__init__(conn)
self.embedder = embedder # callable: str -> np.ndarray
def search(self, query, limit=20):
qv = self.embedder(query)
rows = self.conn.execute(
"SELECT chunk_id, distance FROM chunk_vecs "
"WHERE embedding MATCH ? AND k = ? "
"ORDER BY distance",
(qv.tobytes(), limit),
).fetchall()
# JOIN back to chunks + documents, build Hits with the same
# _build_snippet helper FTS5 already uses.
...
```
The merge with FTS5 happens in `arborist/qa/query.py` where the
four FTS5 routes already merge today. RRF is a six-line addition
once both rankings exist.
Optional-dep guard at import time (mirrors `[html]`):
```python
try:
import sqlite_vec
_VEC_AVAILABLE = True
except ImportError:
_VEC_AVAILABLE = False
```
CLI surface only exposes `--retrieval=vec` or `--retrieval=hybrid`
when `_VEC_AVAILABLE`, otherwise the flag errors with an install
hint. Same pattern as `selectolax` for the HTML source.
---
## 8. Phased plan with measurement gates
### Phase 0 — this ticket (doc)
- Decide embedder (path 1 or path 2, named model).
- Decide default quantization + dimension (recommendation:
**int8 × 384 + flat** as the speed/quality default; **binary ×
768 + flat** as the storage-conservative alternative).
- Run the 1k-chunk smoke test under WAL + `synchronous=NORMAL`.
- Write the bench protocol: which fixtures, what signal floor.
Lift target: ≥ 5pp recall@20 on bench-emergent random-word
fixtures (bench-maxing.md signal floor).
### Phase 1 — code (gated on Phase 0 sign-off)
- `arborist/search/vec.py` with optional-dep guard.
- `chunk_vecs` virtual table created lazily on first ingest with
`--embed`.
- `arborist ingest --embed` ingest-side flag (default off).
- `make rebuild-vec` Makefile target.
- Rebuilds populate `chunk_vecs` for all current `chunks` rows.
- Bench: vec-only retrieval vs FTS5-only on a single shard.
**Phase 1 succeeds if** vec-only matches FTS5-only within ±5pp
on STRICT-rate AND beats FTS5 by ≥ 5pp on at least one
bench-emergent semantic-allusion fixture.
### Phase 2 — hybrid fusion (gated on Phase 1 success)
- RRF merge in `arborist/qa/query.py` between FTS5 routes and
vec route.
- Bench: hybrid vs FTS5-only.
- **Phase 2 succeeds if** hybrid lifts STRICT-rate by ≥ 5pp over
FTS5-only with no UNGROUNDED-rate regression.
- If Phase 2 fails by < 5pp lift: park the vec layer as opt-in
per-call (`--retrieval=vec`) but do not make hybrid the default.
The 2.4 GB tax is not paid for a sub-signal-floor lift.
### Phase 3 — `governance_policy_hash` wiring (mechanical)
- Fold the five vec-version fields (§6) into the policy hash.
- Document in CLAUDE.md "Versioned defaults" and "Schema
invariants" sections.
- Update `make chain-check-shards` semantics if needed (it should
not need changes — vec data is sibling, like `concept_relations`).
---
## 9. Decision tree on quantization
```text
Phase 0 budget: ≤ 15 % storage tax (≤ 5.7 GB across all shards).
If Phase 1 bench shows int8 × 384 hits the recall target
→ ship int8 × 384 + flat as default.
→ 6 % storage tax. Best speed/quality for the budget.
If Phase 1 bench shows int8 × 384 misses the target
AND int8 × 768 hits it
→ ship int8 × 768 + flat.
→ 13 % storage tax. Borderline acceptable.
If Phase 1 bench shows int8 × 768 also misses
→ DO NOT ship float32 anywhere — too expensive.
→ Park the ticket. Reopen when a smaller, sharper embedder lands
(research-bench problem, not an arborist problem).
If storage budget tightens later
→ fall back to binary × 768 + flat. < 2 % tax,
~10–20 % recall hit, still wins on the semantic-allusion
fixtures the lexical pipeline misses entirely.
```
---
## 10. Risks and open questions
1. **`sqlite-vec` pre-v1 instability.** Mitigation: pin tightly,
treat upgrades as schema-bump events, run the 1k-chunk smoke
test after every dep bump. Phase 0 deliverable.
2. **Embedder model drift.** A new HuggingFace revision of
`bge-small` invalidates every prior embedding. Mitigation: pin
model+revision via `embedder_version` in §6, track upgrades
under `make rebuild-vec`.
3. **Re-embed cost on chunker bump.** `tok-512-v1 → v2` already
stales every FTS5 chunk. Adds re-embed work proportional to
chunk count. At 6.24 M chunks × ~10 ms/chunk on a CPU
bge-small ≈ 17 hours single-threaded; parallelizes trivially.
Already a known-large cost; vec adds ~10–20 % to it.
4. **Verifier semantics unchanged.** Vec lifts retrieval recall;
the lexical verifier (quote / span / entity / paraphrase)
stays binary. A vec-retrieved chunk that the verifier cannot
ground still lands UNGROUNDED. Worth confirming on the bench:
does vec recall translate to STRICT-rate lift, or just to more
UNGROUNDED hits being dropped at verification time?
5. **Title-relevance hard check (Rule 8) interaction.** Vec is
precisely the retrieval mode most likely to surface
structurally-unrelated cited chunks (the spin-glass / QCD
case). Phase 1 bench must include the existing title-mismatch
fixtures and confirm vec hits get caught by Rule 8 like any
other retrieval-driven hallucination candidate. If vec hits
bypass Rule 8 somehow, that is a Phase 1 blocker.
6. **`edges` is the dominant cost (38 % per §3), not FTS5.**
Confirmed with the corpus-wide row count: **90.6 M edges
across 3.47 M documents = ~26 edges/document.** If storage
matters more than retrieval quality, compacting `edges` is a
higher-leverage optimization than anything vec-related (a
single int8 × 384 vec layer adds 6 % tax; trimming edges by
10 % saves 3.8 % at zero quality cost). Out of scope for this
ticket but worth a reminder when we benchmark.
7. **Concurrent-Hermes bottleneck (#000037 §11) is irrelevant
here** as long as we use a local embedder (path 1 in §5). If
we go path 2 or 3, vec ingest competes with inference traffic
and the Prometheus-Σ controller (#000037) would need to
throttle ingest under load.
---
## 11. What this ticket does NOT do
- Does not commit to shipping a vec backend. Phase 0 is doc-only;
every later phase has a measurable gate.
- Does not bump `cache_key` dimensions. Vec config folds into
`governance_policy_hash`, same pattern as every prior versioned
default.
- Does not change the verifier or the audit chain. Vec is a soft
signal at the retrieval layer only; proof-path stays identical.
- Does not replace any FTS5 route. §4 settles this: vec is a
fifth route, never a substitute.
- Does not implement the unconscious falsification sweep
(#000037 Phase 1). That is a separate ticket; vec just makes
semantic neighbors easier to surface, which is one capability
the sweep can use later.
---
## 12. References
- This ticket: doc-only Phase 0.
- `arborist/search/base.py` — `SearchBackend` ABC, the integration
surface.
- `arborist/search/fts5.py` — the contract the vec backend mirrors.
- `arborist/qa/query.py` — where RRF merge would land.
- CLAUDE.md "Retrieval pipeline" — the four-route baseline.
- CLAUDE.md "Soft hash vs hard hash" — the rule that licenses vec
retrieval as long as it never enters the proof path.
- CLAUDE.md "Versioned defaults" — the pattern §6 follows.
- #000018 (concept_relations) — the 1.6 % storage-tax precedent.
- #000037 §11 — the Hermes ~4-concurrent constraint that drives
the "embed at ingest, not at query" rule.
- `~/.arborist/shards/` — 4 × 9.6 GB shards measured 2026-05-09;
the per-table breakdown in §3 is `dbstat`-derived from
`000.db`.
- Upstream: <https://github.com/asg017/sqlite-vec> (v0.1.9 at time
of writing; expect breaking changes pre-v1).