docs(L5): Reverse RAG framing + no-embeddings + per-1k COGS at $0.33/kWh
- Frame the solution as a Reverse RAG (Merkle Providence Reverse RAG) with a link to the whitepaper (unfirehose.com/merkle-providence-reverse-rag- whitepaper — note: published on unfirehose, not uncloseai). - New differentiator: NO vector embeddings — retrieval is lexical-first (FTS5 BM25 + Merkle), dense-vector optional + off by default; embedding 10M docs costs 10-100x more/doc + a vector index to store/maintain. A big part of why COGS is low. - COGS framed per 1,000 answers, labeled @ $0.33/kWh (intro + diagram cost node). Cost node clarified: no embeddings, NO reasoning (reasoning is the thing that would cost 4-6x, which we skip). Retrieval node + mapping table updated to lexical-first / no vector index.
This commit is contained in:
parent
e8bc5c2220
commit
89ce211077
1 changed files with 29 additions and 14 deletions
|
|
@ -6,12 +6,16 @@ made the rounds online:
|
|||
|
||||
*"Design a RAG pipeline for 10M docs with zero hallucination."*
|
||||
|
||||
The canonical answer is a ten-box pipeline — ingest/normalize, hybrid
|
||||
BM25+embedding retrieval, ANN+rerank, source-confidence scoring,
|
||||
constrained generation, citation-backed responses, plus evals, caching and
|
||||
observability. arborist already implements every one of those boxes — and
|
||||
goes three steps further, which is what actually buys *zero hallucination*
|
||||
and a near-zero bill:
|
||||
arborist answers it as a **Reverse RAG** — the Merkle Providence Reverse
|
||||
RAG architecture (`whitepaper`_): instead of trusting a model and hoping
|
||||
it stays grounded, every claim is *bound back* to source spans by Merkle
|
||||
proof, and what cannot be bound is refused. The canonical answer is a
|
||||
ten-box pipeline — ingest/normalize, hybrid BM25+embedding retrieval,
|
||||
ANN+rerank, source-confidence scoring, constrained generation,
|
||||
citation-backed responses, plus evals, caching and observability.
|
||||
arborist already implements every one of those boxes — and goes four
|
||||
steps further, which is what actually buys *zero hallucination* and a
|
||||
near-zero bill:
|
||||
|
||||
#. **A deterministic verifier, not a model confidence score.** "Zero
|
||||
hallucination" is not a threshold you tune — it is a property you prove.
|
||||
|
|
@ -21,10 +25,20 @@ and a near-zero bill:
|
|||
#. **A Merkle-bound cache that skips the GPU.** A hot answer is a
|
||||
content-addressed providence record with a Merkle proof — it replays
|
||||
with **zero GPU joules** and never re-enters the cost.
|
||||
#. **Measured energy COGS.** Constrained, non-reasoning generation
|
||||
(cheap prefill + short decode, no test-time-compute chains) lands a
|
||||
grounded answer for **~$0.07–0.16 per 1,000 answers** of GPU
|
||||
electricity (see :doc:`bench`).
|
||||
#. **No vector embeddings.** Retrieval is **lexical-first** (FTS5 BM25 +
|
||||
Merkle proofs); dense-vector semantic search is an *optional* layer,
|
||||
**disabled by default**. Embedding 10M docs would cost **10–100× more
|
||||
per document** to ingest and a whole vector index to store, maintain
|
||||
and re-embed on drift — arborist skips all of it. That asymmetry is a
|
||||
large part of why the bill is what it is.
|
||||
#. **Measured energy COGS — per 1,000 answers.** Because the pipeline is
|
||||
a Reverse RAG with no embedding step and no reasoning chains (cheap
|
||||
prefill + short decode), the GPU cost of a *grounded* answer is small
|
||||
enough to **measure per thousand**: **~$0.07–0.16 per 1,000 answers
|
||||
at $0.33/kWh** (see :doc:`bench`). A cache hit skips the GPU entirely,
|
||||
so it never enters that count.
|
||||
|
||||
.. _whitepaper: https://unfirehose.com/merkle-providence-reverse-rag-whitepaper
|
||||
|
||||
Pipeline
|
||||
--------
|
||||
|
|
@ -71,7 +85,7 @@ Pipeline
|
|||
subgraph cluster_retr {
|
||||
label="3 RETRIEVAL + RERANK (4 routes, merged)";
|
||||
style=dashed; color="#00aa00";
|
||||
retr [label="FTS5: BM25 body / title-LIKE /\ncore-keyword (TF-IDF) / phrase-pattern\n-> merge"];
|
||||
retr [label="FTS5 LEXICAL-FIRST (no vector index)\nBM25 body / title-LIKE /\ncore-keyword (TF-IDF) / phrase-pattern\n-> merge"];
|
||||
rerank [label="sqrt body-coverage rerank /\ntitle boost / rivalry exclusion +\nsynonym / per-source cap"];
|
||||
ctx [label="context assembly\nwikitext -> base prose /\nper-mode budget"];
|
||||
retr -> rerank -> ctx;
|
||||
|
|
@ -82,7 +96,7 @@ Pipeline
|
|||
style=dashed; color="#aa00aa";
|
||||
gen [label="claim_lattice (pointer IDs ->\nruntime-interpolated spans)\nOR quote - no test-time compute",
|
||||
style=filled, fillcolor="#fff0ff"];
|
||||
cost [label="GPU: cheap prefill + short decode\n~$0.07-0.16 / 1k answers\n(reasoning chains cost 4-6x)",
|
||||
cost [label="GPU: cheap prefill + short decode\n~$0.07-0.16 / 1k answers @ $0.33/kWh\nno embeddings · NO reasoning (would cost 4-6x)",
|
||||
shape=note, style=filled, fillcolor="#ffffcc"];
|
||||
gen -> cost [style=dotted, arrowhead=none];
|
||||
}
|
||||
|
|
@ -138,8 +152,9 @@ How arborist maps onto (and extends) the canonical design
|
|||
- **Merkle commitment** + idempotent content-addressing + lossless
|
||||
``supersedes`` version history
|
||||
* - Hybrid retrieval (BM25 + embeddings)
|
||||
- 4-route FTS5 merge (``qa/query.py``)
|
||||
- phrase-pattern route (closes the allusion gap) + rivalry exclusion
|
||||
- 4-route FTS5 merge, **lexical-first** (``qa/query.py``)
|
||||
- **no embeddings needed** — dense-vector optional + off by default
|
||||
(10-100x cheaper ingest, no vector index); phrase-pattern route
|
||||
* - ANN + reranking
|
||||
- sqrt body-coverage rerank + title boost
|
||||
- corpus-derived synonym/rivalry layer, no embedding index to drift
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue