russell@unturf.com
a056acfb5b
The 2003-05-16 archive ships three files:
20030516_cur_tablesql.bz2 82 MB current snapshot (single revision/page)
old_tablesqlbz2.1 640 MiB \
old_tablesqlbz2.2 252 MiB / split halves of old (full revision
history). Concatenate before bzcat.
Generalize the parser:
WikipediaSqlDump(table='cur'|'old') — shared statement parser, single
column-position contract for the
first 4 fields (id/ns/title/text)
WikipediaCurDump — back-compat wrapper, table='cur'
WikipediaOldDump — new, table='old'; old has no is_redirect, every
revision is real
Old rows surface old_id and old_timestamp via Document.extra so a
downstream pass can sort revisions chronologically before re-ingesting
through the supersedes-edge path.
Makefile gains:
fetch-cur / fetch-old / fetch (both)
ingest-cur / ingest-old / ingest (cur default)
WP_OLD target concatenates the two split parts
CLI ingest --source now accepts wikipedia_cur or wikipedia_old.
Smoke (real dump): 5 revisions of "AtlasShrugged/Companies" yielded
correctly with old_id=2..10, timestamps from January 2002.
|
||
|---|---|---|
| aborist | ||
| tests | ||
| .gitignore | ||
| LICENSE | ||
| Makefile | ||
| pyproject.toml | ||
| README.md | ||
aborist
An arborist for trees and forests of cross-linked information.
Aborist ingests documents into a content-addressed, Merkle-committed store. Every
document carries a URI for backtrack and cross-link. Search hooks return hits with
declared audit modes (STRICT / HYBRID / VISUAL) so callers never overclaim
what a result actually proves.
Design
Aborist implements the runtime spec from Merkle-AGI v9.8 / Merkle Providence
Reverse RAG. The 8-dim admissibility key (source_root, question_hash,
model_profile_hash, conversation_hash, governance_policy_hash,
schema_version, canonicalization_version, chunking_version) plus
falsification state ensures cached records are never reused under drift.
Two layered document kinds:
- surface — diverse ingested content (full chunks, FTS-indexed)
- core — distilled records (haiku/equation/snippet) Merkle-signed back to
source surface roots via
derivations
Reversible eviction via chunk tier ∈ {hot, warm, cold}. Cold = leaf hash
- URI only; rehydratable from URI, identity verifiable via leaf hash.
Quick start
make all # bootstrap + fetch + ingest + verify + stats
make search Q='…'
License
AGPL-3.0-only.