Commit graph

6 commits

Author SHA1 Message Date
656b573198
modified: .gitignore
modified:   Makefile
	modified:   arborist/cli.py
	new file:   arborist/qa/progress.py
	modified:   arborist/qa/query.py
	new file:   arborist/qa/witness.py
	modified:   bench/results/real-shard-baseline.json
	modified:   bench/results/real-shard-baseline.md
	modified:   docs/TICKETS.md
	new file:   docs/tickets/ticket-000028-multi-modality-witness.md
	new file:   greatest-live-rock-and-roll-song-ever-played.md
	new file:   tests/test_witness.py
2026-05-08 16:38:09 -04:00
546fa6d690
docs: auto-generate Makefile reference page for RTD
Adds a Sphinx extension at docs/_source/_ext/makefile_targets.py that
parses the project Makefile's '## description' annotations and writes
docs/_source/api/makefile.rst at build time. Same convention 'make help'
uses, so the reference stays in sync with the source.

Generated page is grouped by target prefix (fetch-, ingest-, distill-,
docs-, etc.) and rendered as a list-table. Shows on RTD alongside the
autodoc API modules.

Generated file is gitignored — RTD regenerates on every build.
2026-05-04 08:26:27 -04:00
8e9e993571
docs: add Read the Docs configuration
Add .readthedocs.yaml at project root so docs.aborist (or
aborist.readthedocs.io) can build the Sphinx API reference on push.

Configuration:
- Ubuntu 24.04, Python 3.13
- Sphinx config: docs/_source/conf.py
- Build deps: docs/_source/requirements.txt (sphinx, furo)
- Install package with [html, wikitext] extras so autodoc imports succeed

Cleanup:
- Untrack docs/_source/_build/ (generated artifact, RTD builds on push)
- Add _build/ to .gitignore
- Create docs/_source/_static/ placeholder (Sphinx convention)
- Remove non-functional html_sidebars override from conf.py
  (furo defaults are better; build now produces 0 warnings vs 29)
2026-05-04 08:17:24 -04:00
b99aa8d300
chore: gitignore .claude/, drop accidentally-staged worktree pointers
The previous commit (588aa45) accidentally git-added four
.claude/worktrees/agent-* gitlinks via `git add -A`. Those are
session-local Claude Code worktree pointers (a tooling artifact),
not source. Removing them from the index and adding `.claude/` to
.gitignore so future sessions don't trip the same way.
2026-04-30 15:11:48 -04:00
2601575f15
bench: QA-quality sweep harness with n-sample variance
Adds bench/qa_sweep.py + bench/qa_questions.txt + `make bench-qa` target.
Sweeps a fixed question set through quote and claim_lattice_pointer modes,
N samples per cell (default 3), each sample burns the cached record so
Hermes nondeterminism becomes the variance source.

Outputs:
  bench/qa_results/<utc>.jsonl   one row per (mode, question, sample)
  bench/qa_results/<utc>.md      summary + per-question vote counts

Question set spans the failure-mode shapes the verifier needs to handle:
narrow factoid, broad descriptive ("tell me about X"), entity list,
relationship, and out-of-corpus (should land UNGROUNDED honestly).

Methodology gap this closes: at n=1 the strict-rate swings ±20pp on a
fixed prompt purely from sampling noise. n=3 makes 5pp deltas legible.
2026-04-30 11:46:43 -04:00
856b3116d7
phase 0 explore: aborist core + sources + distill + evict
A content-addressed, Merkle-committed document store implementing the
runtime spec from Merkle Providence Reverse RAG (April 2026 whitepaper)
and Merkle-AGI v9.8 admissibility ledger. Ports proxy.unturf.com Go
merkle conventions to Python: non-commutative HashCombine with 0x03
prefix, explicit IsLeft per sibling, self-duplicate odd elements.

What's in:

- merkle.py — proof generation/verification, JSON serialization
- store.py — v9.8 SQLite schema: 8-dim providence_cache key,
  falsification_state, append-only audit chain, surface/core kind,
  hot/warm/cold tier, derivations, edges
- ingest.py — Source -> normalize -> chunk -> merkle -> upsert,
  idempotent on document_root collision
- search/ — SearchBackend ABC with explicit AuditMode (STRICT/HYBRID/
  VISUAL), FTS5 backend returning VISUAL hits
- sources/ — wikipedia.py (streaming bz2/MySQL extended-INSERT parser
  for 2003-era cur dumps); html_page.py (selectolax + httpx, robots.txt
  honored automatically)
- distill/ — Distiller ABC + first-sentence-v1 stub. Runner generates
  per-contributing-chunk Merkle proofs binding cores back to source
  document_root.
- evict.py — hot->cold demote (NULLs content, drops FTS row, retains
  leaf_hash). rehydrate() refetches via source pipeline; matching root
  restores content, mismatching root marks providence stale and writes
  rehydrate_drift event. Cores never evict.
- cli.py — ingest / search / verify / stats / distill / evict /
  rehydrate
- 31 tests covering merkle round-trip, ingest+audit, chunker version
  binding, html parse, distillation proof verification, evict+
  rehydrate including drift detection.

Smoke: 503 Wikipedia 2003-05-16 + 3 fox-owned HTML pages ingested,
478 cores produced (24 surface->core merkle dedups), 7 chunks evicted
to cold and round-tripped via rehydrate, 987 audit events chained 0
breaks.
2026-04-27 07:53:18 -04:00