trim Makefile (-6 redundant targets) + better RTD docs
Makefile cuts (64 → 58 documented targets): - ingest-cur-parallel, ingest-old-parallel: parallel-shared mode superseded by attached (no WAL contention) - distill-shards: sequential never preferred over parallel variant - bench-qa-quick: bench-qa-smoke covers same use case (~30s vs ~10s) - ingest-grok, ingest-grok-media: single-DB grok rare; -attached is canonical path All cuts land in code that the underlying CLI still exposes — operators who need the dropped variant call '.venv/bin/aborist ingest --shard ...' directly. No behavior loss, just shortcut removal. Docs improvements: - New Concepts page (docs/_source/concepts.rst): orientation on what aborist is, three layers (surface/core/providence), Merkle commitment, 8-dim cache key, audit chain, trichotomy + four-rung ladder, layered verifier, falsification state, sidecars. Embeds module-graph and verifier-ladder SVG diagrams. - New Cookbook page (docs/_source/cookbook.rst): 8 recipes — recrawl, falsify, ingest-self-providence, mixed-corpus query, LLM endpoint override, integrity after bulk ops, bench, retrieval tuning. - Quickstart embeds query-pipeline SVG diagram. - docs/_source/diagrams symlinks to docs/diagrams so Sphinx can include the SVGs (was orphaned, only README referenced them). Better Makefile RTD page (docs/_source/_ext/makefile_targets.py): - Group by workflow phase (Setup → Fetch → Ingest → Distill → Query → Verify → Operations → Tests → Docs → Clean) instead of alphabetical prefix. Tells a new operator the order they'd actually run things. - Phase descriptions added; targets prefixed with 'make ' for copy-paste. - Uncategorized leftover surfaces missing entries in PHASES list.
This commit is contained in:
parent
09fb5e4f2d
commit
850e59d16e
7 changed files with 496 additions and 61 deletions
143
docs/_source/cookbook.rst
Normal file
143
docs/_source/cookbook.rst
Normal file
|
|
@ -0,0 +1,143 @@
|
|||
Cookbook
|
||||
========
|
||||
|
||||
Recipes for common workflows beyond the quickstart. Each starts from
|
||||
a working aborist install (``make bootstrap`` already run) and a
|
||||
populated shards directory under ``~/.aborist/shards/``.
|
||||
|
||||
Re-crawl a website to detect changes
|
||||
-------------------------------------
|
||||
|
||||
After ``make crawl-ingest`` lands a site into a shard, the per-page
|
||||
ETag and Last-Modified headers are kept in ``document_http_meta``. A
|
||||
re-crawl can ask "did anything change?" without downloading bodies.
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
make crawl-ingest URL=https://example.com DEPTH=2 # initial crawl
|
||||
# ...later...
|
||||
make recrawl-check DOMAIN=example.com # conditional HEAD per page
|
||||
|
||||
Each URL is classified ``fresh`` (304), ``stale`` (200 with new body),
|
||||
``gone`` (404/410), or ``unreachable``. One tiny round-trip per URL,
|
||||
no body transfer when content is unchanged.
|
||||
|
||||
Falsify a wrong answer (audit-preserving)
|
||||
------------------------------------------
|
||||
|
||||
The verifier called something STRICT but you know it's wrong. Mark
|
||||
the record falsified — it stays in the DB so downstream consumers
|
||||
that referenced it can still trace history.
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
make query Q="When did X happen?" # see the answer + cache_key
|
||||
make inspect KEY=<cache_key> # diagnose unverified spans
|
||||
make falsify KEY=<cache_key> REASON='wrong year — sources cited 1942 not 1944'
|
||||
|
||||
Future lookups skip records whose ``falsification_state != 'live'``.
|
||||
A ``falsify`` audit event records the act; the chain stays intact.
|
||||
|
||||
Promote your own past answers into the corpus
|
||||
----------------------------------------------
|
||||
|
||||
After enough STRICT answers accumulate, treat them as a derived
|
||||
source. The ``providence`` ingest path promotes mature STRICT records
|
||||
into the document corpus where retrieval can pick them up.
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
make ingest-self-providence KG_SECONDS=86400 # only records ≥1 day old
|
||||
|
||||
The ``KG_SECONDS`` (kindergarten window) prevents the system from
|
||||
trusting freshly-cached answers as ground truth before they've had
|
||||
time to fail. See :doc:`api/qa` for the ``ProvidenceSource`` impl.
|
||||
|
||||
Query across mixed corpora
|
||||
--------------------------
|
||||
|
||||
Every shard under ``~/.aborist/shards/`` is queried automatically.
|
||||
Mix Wikipedia, your Grok export, a crawled site, and your own git
|
||||
repos in one query — retrieval ranks across all of them.
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
make ingest-cur-attached # Wikipedia
|
||||
make ingest-grok-attached GROK_EXPORT=$HOME/Downloads/<uuid> # Grok chats
|
||||
make crawl-ingest URL=https://russell.ballestrini.net DEPTH=2 # personal site
|
||||
make ingest-git GIT_REPO=$HOME/git/myproject # source code
|
||||
|
||||
make query Q="how does my project handle authentication?"
|
||||
|
||||
Each shard contributes hits; the query path's title-relevance + body
|
||||
coverage rerank lets neologisms in your private corpus outrank generic
|
||||
Wikipedia matches.
|
||||
|
||||
Override the LLM endpoint
|
||||
-------------------------
|
||||
|
||||
Default points at ``https://hermes.ai.unturf.com/v1`` (Hermes-3-8B,
|
||||
no auth). Point at any OpenAI-compatible endpoint via env:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
export ABORIST_LLM_ENDPOINT="https://your-vllm.example.com/v1"
|
||||
export ABORIST_LLM_MODEL="meta-llama/Llama-3.1-70B-Instruct"
|
||||
export ABORIST_LLM_API_KEY="..." # optional; many vLLM deploys are open
|
||||
make query Q="..."
|
||||
|
||||
The model id folds into ``model_profile_hash`` (one of the 8 cache
|
||||
key dimensions), so swapping models invalidates prior cache hits on
|
||||
lookup — no risk of serving an answer one model produced under
|
||||
another model's identity.
|
||||
|
||||
Verify shard integrity after a bulk operation
|
||||
----------------------------------------------
|
||||
|
||||
Any state-changing op (mass falsify, hash bump, schema migration)
|
||||
should be followed by:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
make chain-check-shards # 0 chain breaks per shard = intact
|
||||
make analyze-shards # compression spectrum + audit integrity
|
||||
make verify-shards # round-trip Merkle proofs on a sample
|
||||
|
||||
Chain breaks are the loudest possible signal. Run these before
|
||||
declaring an op successful.
|
||||
|
||||
Run the QA bench and read the results
|
||||
--------------------------------------
|
||||
|
||||
The QA bench measures how often the verifier says STRICT vs HYBRID
|
||||
vs UNGROUNDED across a fixed question set, per answer mode.
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
make bench-qa-smoke # ~30s, 5 questions × 3 modes
|
||||
make bench-qa BENCH_QA_N=3 # full sweep, 3 samples each
|
||||
|
||||
Output lands in ``bench/qa_results/<utc-stamp>.{jsonl,md}``. The
|
||||
markdown file has the summary table; the JSONL has every per-question
|
||||
record for drill-down.
|
||||
|
||||
Resume an interrupted bench:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
make bench-qa --resume bench/qa_results/<previous-utc-stamp>.jsonl
|
||||
|
||||
Same ``--seed`` is required for shuffled-task-order alignment.
|
||||
|
||||
Tune retrieval per-question
|
||||
---------------------------
|
||||
|
||||
When a query returns the wrong sources, ``K=`` injects extra retrieval
|
||||
keywords without changing what the LLM sees as the question:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
make query Q="What did Orwell mean by always at war?" K="1984 Oceania Eastasia"
|
||||
|
||||
Provenance gap on this is tracked in
|
||||
:doc:`api/qa` (``aborist.qa.query``).
|
||||
Loading…
Add table
Add a link
Reference in a new issue