docs(L5): Reverse RAG framing + no-embeddings + per-1k COGS at $0.33/kWh

- Frame the solution as a Reverse RAG (Merkle Providence Reverse RAG) with
  a link to the whitepaper (unfirehose.com/merkle-providence-reverse-rag-
  whitepaper — note: published on unfirehose, not uncloseai).
- New differentiator: NO vector embeddings — retrieval is lexical-first
  (FTS5 BM25 + Merkle), dense-vector optional + off by default; embedding
  10M docs costs 10-100x more/doc + a vector index to store/maintain. A
  big part of why COGS is low.
- COGS framed per 1,000 answers, labeled @ $0.33/kWh (intro + diagram cost
  node). Cost node clarified: no embeddings, NO reasoning (reasoning is the
  thing that would cost 4-6x, which we skip). Retrieval node + mapping
  table updated to lexical-first / no vector index.
This commit is contained in:
russell@unturf.com 2026-05-21 15:49:12 -04:00
parent e8bc5c2220
commit 89ce211077
No known key found for this signature in database

View file

@ -6,12 +6,16 @@ made the rounds online:
*"Design a RAG pipeline for 10M docs with zero hallucination."*
The canonical answer is a ten-box pipeline — ingest/normalize, hybrid
BM25+embedding retrieval, ANN+rerank, source-confidence scoring,
constrained generation, citation-backed responses, plus evals, caching and
observability. arborist already implements every one of those boxes — and
goes three steps further, which is what actually buys *zero hallucination*
and a near-zero bill:
arborist answers it as a **Reverse RAG** — the Merkle Providence Reverse
RAG architecture (`whitepaper`_): instead of trusting a model and hoping
it stays grounded, every claim is *bound back* to source spans by Merkle
proof, and what cannot be bound is refused. The canonical answer is a
ten-box pipeline — ingest/normalize, hybrid BM25+embedding retrieval,
ANN+rerank, source-confidence scoring, constrained generation,
citation-backed responses, plus evals, caching and observability.
arborist already implements every one of those boxes — and goes four
steps further, which is what actually buys *zero hallucination* and a
near-zero bill:
#. **A deterministic verifier, not a model confidence score.** "Zero
hallucination" is not a threshold you tune — it is a property you prove.
@ -21,10 +25,20 @@ and a near-zero bill:
#. **A Merkle-bound cache that skips the GPU.** A hot answer is a
content-addressed providence record with a Merkle proof — it replays
with **zero GPU joules** and never re-enters the cost.
#. **Measured energy COGS.** Constrained, non-reasoning generation
(cheap prefill + short decode, no test-time-compute chains) lands a
grounded answer for **~$0.070.16 per 1,000 answers** of GPU
electricity (see :doc:`bench`).
#. **No vector embeddings.** Retrieval is **lexical-first** (FTS5 BM25 +
Merkle proofs); dense-vector semantic search is an *optional* layer,
**disabled by default**. Embedding 10M docs would cost **10100× more
per document** to ingest and a whole vector index to store, maintain
and re-embed on drift — arborist skips all of it. That asymmetry is a
large part of why the bill is what it is.
#. **Measured energy COGS — per 1,000 answers.** Because the pipeline is
a Reverse RAG with no embedding step and no reasoning chains (cheap
prefill + short decode), the GPU cost of a *grounded* answer is small
enough to **measure per thousand**: **~$0.070.16 per 1,000 answers
at $0.33/kWh** (see :doc:`bench`). A cache hit skips the GPU entirely,
so it never enters that count.
.. _whitepaper: https://unfirehose.com/merkle-providence-reverse-rag-whitepaper
Pipeline
--------
@ -71,7 +85,7 @@ Pipeline
subgraph cluster_retr {
label="3 RETRIEVAL + RERANK (4 routes, merged)";
style=dashed; color="#00aa00";
retr [label="FTS5: BM25 body / title-LIKE /\ncore-keyword (TF-IDF) / phrase-pattern\n-> merge"];
retr [label="FTS5 LEXICAL-FIRST (no vector index)\nBM25 body / title-LIKE /\ncore-keyword (TF-IDF) / phrase-pattern\n-> merge"];
rerank [label="sqrt body-coverage rerank /\ntitle boost / rivalry exclusion +\nsynonym / per-source cap"];
ctx [label="context assembly\nwikitext -> base prose /\nper-mode budget"];
retr -> rerank -> ctx;
@ -82,7 +96,7 @@ Pipeline
style=dashed; color="#aa00aa";
gen [label="claim_lattice (pointer IDs ->\nruntime-interpolated spans)\nOR quote - no test-time compute",
style=filled, fillcolor="#fff0ff"];
cost [label="GPU: cheap prefill + short decode\n~$0.07-0.16 / 1k answers\n(reasoning chains cost 4-6x)",
cost [label="GPU: cheap prefill + short decode\n~$0.07-0.16 / 1k answers @ $0.33/kWh\nno embeddings · NO reasoning (would cost 4-6x)",
shape=note, style=filled, fillcolor="#ffffcc"];
gen -> cost [style=dotted, arrowhead=none];
}
@ -138,8 +152,9 @@ How arborist maps onto (and extends) the canonical design
- **Merkle commitment** + idempotent content-addressing + lossless
``supersedes`` version history
* - Hybrid retrieval (BM25 + embeddings)
- 4-route FTS5 merge (``qa/query.py``)
- phrase-pattern route (closes the allusion gap) + rivalry exclusion
- 4-route FTS5 merge, **lexical-first** (``qa/query.py``)
- **no embeddings needed** — dense-vector optional + off by default
(10-100x cheaper ingest, no vector index); phrase-pattern route
* - ANN + reranking
- sqrt body-coverage rerank + title boost
- corpus-derived synonym/rivalry layer, no embedding index to drift