Commit graph

8 commits

Author SHA1 Message Date
b711215f11
docs: pagers — rename 'not grounded' → 'ungrounded' (agreed label set: grounded / partly grounded / ungrounded)
'not grounded' was a drift from the agreed three-label vocabulary. Single-word labels are cleaner in tables, diagrams, and prose.
2026-05-14 12:56:13 -04:00
75ae470581
docs: rewrite arborist-one-pager + arborist-two-pager for external readers — drop ticket refs, schema versions, and internal vocabulary; reframe around user value (verified answers, fabricated-citation prevention, replay)
Old drafts read as internal substrate notes. Rewrites lead with what the system does for a consumer or evaluator and what it costs to run, with no references to internal tickets, table names, schema-version strings, governance hash dimensions, or per-record audit-mode tokens. Appendix diagrams updated in lockstep: friendly labels ("grounded / partly grounded / not grounded") replace the schema-column trichotomy, layer names paraphrased away from SURFACE/CORE/PROVIDENCE.
2026-05-14 10:02:08 -04:00
0aada29a8e
docs: arborist-one-pager + arborist-two-pager — Dav1d/fox-signoff summaries with letterhead, license, and 2 strategic appendix diagrams
1-pager (docs/_source/arborist-one-pager.rst, 1 page) for AI-literate readers: the trichotomy, the 8-dim cache key, CTI synthetic-elision-impossible, soft-channel separation, real-traffic bench numbers (mis-cite 100% @ 0% FP, warrant 92/92, quote 0.54 STRICT-rate).

2-pager (docs/_source/arborist-two-pager.rst, 3 pages = 2 body + 1 appendix) for technical reviewers: letterhead, Permacomputer Preamble license box, six numbered sections, plus appendix figures (pager-arch-stack 3-layer architecture, pager-verifier-flow question→pointer→verifier→trichotomy).

Both pages live under docs/_source/ so the same RST renders into the Sphinx readthedocs site (toctree caption "Summary pages" added to docs/_source/index.rst) AND into standalone PDFs via rst2pdf (docs/pager.style, lazy install into .venv).

Makefile targets: docs-one-pager, docs-two-pager, docs-pagers, docs-pagers-clean. Diagrams render through the existing DOT pipeline.
2026-05-14 09:48:50 -04:00
6e42dbf026
docs/diagrams: rename aborist-modules → arborist-modules (post-rename cleanup)
The top-level module diagram still used the old aborist- (one R) prefix
from before the package rename. arborist.unturf.com served from the
symlinked docs/_source/diagrams/, so the live URL
arborist.unturf.com/en/diagrams/arborist-modules.svg 404'd because the
file on disk was aborist-modules.svg.

Other diagram base names (ingest-pipeline, mesh-*, query-pipeline,
verifier-ladder) were never aborist-prefixed; only this one needed it.
2026-05-08 07:50:28 -04:00
8d6961fcc1
aborist/arborist
modified:   .gitlab-ci.yml
	modified:   bench/qa_questions.txt
	modified:   bench/qa_sweep.py
	modified:   bench/run.sh
	modified:   docs/TICKETS.md
	modified:   docs/_source/README.md
	modified:   docs/_source/_ext/makefile_targets.py
	modified:   docs/_source/api/cli.rst
	modified:   docs/_source/api/distill.rst
	modified:   docs/_source/api/mesh.rst
	modified:   docs/_source/api/qa.rst
	modified:   docs/_source/api/retrieval.rst
	modified:   docs/_source/api/storage.rst
	modified:   docs/_source/api/substrate.rst
	modified:   docs/_source/concepts.rst
	modified:   docs/_source/conf.py
	modified:   docs/_source/cookbook.rst
	modified:   docs/_source/index.rst
	modified:   docs/_source/license.rst
	modified:   docs/_source/quickstart.rst
	modified:   docs/bench-maxing.md
	modified:   docs/benchmarks.md
	modified:   docs/cti-architecture.md
	modified:   docs/diagrams/aborist-modules.dot
	modified:   docs/diagrams/aborist-modules.svg
	modified:   docs/diagrams/mesh-data-flow.dot
	modified:   docs/diagrams/mesh-epoch-lifecycle.dot
	modified:   docs/diagrams/mesh-epoch-lifecycle.svg
	modified:   docs/diagrams/mesh-group-decisions.dot
	modified:   docs/diagrams/mesh-group-decisions.svg
	modified:   docs/diagrams/mesh-identity-stack.dot
	modified:   docs/diagrams/mesh-secret-envelope.dot
	modified:   docs/mesh.md
	modified:   docs/qa-modes-bench.md
	modified:   docs/seven-point-program.md
	modified:   docs/tickets/ticket-000001-retrieval-keywords-audit-gap.md
	modified:   docs/tickets/ticket-000002-reference-frame-polarity-contract.md
	modified:   docs/tickets/ticket-000003-anchor-class-warrant.md
	modified:   docs/tickets/ticket-000005-label-ladder-migration.md
	modified:   docs/tickets/ticket-000006-bench-emergent-findings.md
	modified:   docs/tickets/ticket-000007-query-layer-hyphen-fold.md
	modified:   docs/tickets/ticket-000008-broad-quantifier-preflight-guard.md
	modified:   docs/tickets/ticket-000009-quantifier-preflight-dag-binding.md
	modified:   docs/tickets/ticket-000010-metacognition-preflight-guard.md
	modified:   docs/tickets/ticket-000011-soft-preflight-hint-sidecar.md
	modified:   scripts/backfill_concepts.py
	modified:   scripts/bench_emergent.py
	modified:   tests/crawler/test_async_web_fetcher.py
	modified:   tests/crawler/test_bridge.py
	modified:   tests/crawler/test_web_fetch.py
	modified:   tests/test_bench_qa_sweep.py
	modified:   tests/test_burn.py
	modified:   tests/test_burn_doc.py
	modified:   tests/test_claim_lattice.py
	modified:   tests/test_cli_render.py
	modified:   tests/test_compress.py
	modified:   tests/test_concepts.py
	modified:   tests/test_dag.py
	modified:   tests/test_directives.py
	modified:   tests/test_distill.py
	modified:   tests/test_distill_recursive.py
	modified:   tests/test_evict.py
	modified:   tests/test_frame.py
	modified:   tests/test_grok_source.py
	modified:   tests/test_html_source.py
	modified:   tests/test_ingest.py
	modified:   tests/test_inspect.py
	modified:   tests/test_journal.py
	modified:   tests/test_keys.py
	modified:   tests/test_llm_context_base.py
	modified:   tests/test_merkle.py
	modified:   tests/test_mesh.py
	modified:   tests/test_mesh_aead.py
	modified:   tests/test_mesh_chain.py
	modified:   tests/test_mesh_cli.py
	modified:   tests/test_mesh_cli_pull.py
	modified:   tests/test_mesh_wire.py
	modified:   tests/test_mesh_wire_e2e.py
	modified:   tests/test_metacognition.py
	modified:   tests/test_migration_audit_mode.py
	modified:   tests/test_providence_source.py
	modified:   tests/test_qa.py
	modified:   tests/test_qa_quality_live.py
	modified:   tests/test_quantifier_caps.py
	modified:   tests/test_quantifier_classifier.py
	modified:   tests/test_quantifier_phase4.py
	modified:   tests/test_quantifier_reminder.py
	modified:   tests/test_query.py
	modified:   tests/test_reclassify.py
	modified:   tests/test_repair.py
	modified:   tests/test_resume.py
	modified:   tests/test_snapshot.py
	modified:   tests/test_soft_preflight.py
	modified:   tests/test_tfidf.py
	modified:   tests/test_vcs_source.py
	modified:   tests/test_verify.py
	modified:   tests/test_verify_json.py
	modified:   tests/test_versioned_ingest.py
	modified:   tests/test_warrant.py
	modified:   tests/test_wikipedia_old.py
	modified:   tests/test_wikipedia_xml.py
	modified:   tests/test_wikitext.py
2026-05-07 09:31:49 -04:00
de07ad9392
docs: distill #000008+#000009+#000010 into core docs + diagrams
Three Explore agents fanned out in parallel for a docs/ + diagrams/
+ code-comment audit against the shipped state of the three
preflight tickets. This commit lands all the alignment fixes.

Core docs updates:

  CLAUDE.md
    - dag.py module description: stage counts now read
      "7/8 quote · 9/10 CTI · 3 reject" reflecting #000009 preflight
      stage + reject-broad early-return shape.

  docs/cti-architecture.md §2.2 + §2.3
    - §2.3 Merkle-AGI-DAG section rewritten: documents all five DAG
      shapes (legacy 7/9, post-#000009 8/10, reject-broad 3),
      describes the preflight stage's 5 nested CTI clauses
      (classifier / answer_contract / prompt_contract /
      evidence_contract / policy_refs), pins
      PREFLIGHT_NODE_VERSION = "preflight-node-v1", states the
      audit-replay payoff.
    - §2.2 CTI section: adds the four new modules
      (quantifier, model_profiles, quantifier_reminder,
      metacognition) as code anchors. Notes that pre-answer
      preflight contract extends CTI upstream of retrieval.

  docs/seven-point-program.md
    - D3 status ½ → ¾ — pre-answer preflight contract landed via
      #000008 + #000010. Code anchors + pinning tests updated.
    - D4 status ½ → ¾ — preflight stage adds upstream control
      commitment to the run-DAG. Code anchors include
      build_reject_run_dag + preflight_node_hash.
    - Status snapshot table: tickets column now references
      #000008/#000009/#000010 against D1/D3/D4 directives.
    - "Post-landing addendum (2026-05-03 / 2026-05-04)" subsection
      summarises all three tickets + their commit shas + final
      test count (993 passing, up from 734).

  docs/modules.md
    - Q&A pipeline table: added 4 new modules (quantifier.py,
      model_profiles.py, quantifier_reminder.py, metacognition.py).
      dag.py row updated to "7/8 quote · 9/10 CTI · 3 reject".
    - dag.py subsection rewritten: documents all 5 DAG shapes,
      describes the preflight payload's 5 clauses + question_state.
    - 4 new module subsections (quantifier / model_profiles /
      quantifier_reminder / metacognition) explaining each
      module's purpose, signature, and how it feeds the run-DAG
      preflight clause.

Diagram updates:

  docs/diagrams/query-pipeline.dot + .svg
    - New "PREFLIGHT (#000008 + #000010)" node inserted between
      cache_check and concepts_lookup.
    - New "REJECT-BROAD" node showing the 3-stage minimal DAG
      escape path.
    - render node label extended with the audit-line tail token
      catalog.

  docs/diagrams/aborist-modules.dot + .svg
    - 4 new qa_* nodes in the retrieval & verifier cluster.
    - 8 new edges: qa_query/qa_runner each call into all 4
      preflight modules; qa_dag has dotted edges to qa_quantifier
      + qa_metacognition (preflight clause sources).
    - qa_dag label updated to mention preflight_node_hash + 5 clauses.

  docs/diagrams/verifier-ladder.dot + .svg
    - Soft-demote violations list extended: BROAD_QUANTIFIER_RUNAWAY
      / CAP_APPLIED / SCOPE_UNBOUND, FORMAT_COLLAPSED, BARE_NAME_CLAIM.
    - New "AUDIT-LINE TAILS" annotation node listing all 11 tail
      tokens (#000008 broad-* + #000010 metacog + classic verifier).
    - Dashed edges from each rung to tails note showing tails
      compose onto labels.

Code-side stale-comment fixes (caught by 3rd Explore agent):

  aborist/qa/keys.py:218
    - "The four fields" → "The seven fields"; mention #000010 adds
      six more for metacognition.
  aborist/qa/query.py:2644
    - 7-stage / 9-stage comment expanded to enumerate all four
      base+preflight shapes plus the 3-stage reject path.
  aborist/qa/runner.py:835
    - same expansion as query.py for runner.ask() callsite.

mesh-*.dot, ingest-pipeline.dot, qa-modes-bench.md, bench-maxing.md,
bench-emergent-design.md, verifier-semantic-gap-design.md,
self-reference-design.md, concept-relations-design.md confirmed
orthogonal — no edits needed.

993 tests still passing (no behavior change). 7 files modified
across docs/ + 3 dot diagrams + 3 SVGs + 4 code-comment fixes.
2026-05-03 19:18:22 -04:00
326badf6d8
docs: README label refresh + per-module reference + Graphviz diagrams
Three things in one commit because they're tightly coupled (README
points at the diagrams; diagrams index in modules/index.md points
back at README; module pages embed the diagrams).

(1) README — label refresh:
    - Quickstart label changed from STRICT/HYBRID/UNGROUNDED to the
      four-rung ladder POINTER-LINKED → ANCHOR-WARRANTED →
      EVIDENCE-WARRANTED → UNGROUNDED with -PARTIAL suffix on HYBRID.
    - Verifier section spells out both layers (schema trichotomy +
      display ladder), the seven hard checks of claim_lattice, and
      the five anchor classes of warrant.
    - Architecture tree updated: concepts/ package added, qa/
      sub-modules expanded (warrant, evidence, parse_claims, dag),
      verify.py described as quote/span/entity/paraphrase + claim_lattice.
    - Concept overlay description updated for corpus-derived layer
      (concept_relations table, link_reciprocity extractor, 1.6%
      tax cite).
    - Test count: 326+ → 641+.

(2) docs/diagrams/ — Graphviz dot sources:
    - aborist-modules.dot — top-level package graph (substrate /
      storage / sources / retrieval / qa / mesh / cli)
    - query-pipeline.dot — question → cache → retrieval → LLM →
      verify → render → cache write, with phase budgets
    - ingest-pipeline.dot — source doc → canonicalize → chunk →
      Merkle → upsert (+ optional distill)
    - verifier-ladder.dot — (audit_mode, violations) → display rung
      decision tree
    Existing mesh-*.dot kept as-is. Makefile `make docs` target
    extended to also emit .svg alongside the existing .png so the
    diagrams render in markdown viewers.

(3) docs/modules/ — per-module reference pages:
    - index.md (links to every diagram + every module page)
    - merkle.md, document.md, store.md, ingest.md, evict.md,
      sources.md, search.md, concepts.md, qa.md, distill.md,
      wikitext.md
    Each page is a one-screenful concise reference: what the
    module is for, public API, key invariants, embedded diagrams
    where useful, link to source. Mesh stays at the existing
    docs/mesh.md + docs/mesh-deploy.md (already comprehensive).

Tests: 641 passed (no code change).
2026-05-01 23:19:01 -04:00
c7275b5618
docs: mesh multiplayer protocol with dot diagrams
Adds docs/mesh.md and five graphviz diagrams covering the federation
layer: identity stack, epoch state machine, per-member secret envelope,
gossip wire contract, and operator decision tree. Pins the protocol
contract for the upcoming HTTP wire (mesh sync, mesh serve).

Makefile gets a 'docs' target with pattern rule so PNG renders are
incremental from .dot sources.
2026-04-28 12:54:36 -04:00