Commit graph

4 commits

Author SHA1 Message Date
8d6961fcc1
aborist/arborist
modified:   .gitlab-ci.yml
	modified:   bench/qa_questions.txt
	modified:   bench/qa_sweep.py
	modified:   bench/run.sh
	modified:   docs/TICKETS.md
	modified:   docs/_source/README.md
	modified:   docs/_source/_ext/makefile_targets.py
	modified:   docs/_source/api/cli.rst
	modified:   docs/_source/api/distill.rst
	modified:   docs/_source/api/mesh.rst
	modified:   docs/_source/api/qa.rst
	modified:   docs/_source/api/retrieval.rst
	modified:   docs/_source/api/storage.rst
	modified:   docs/_source/api/substrate.rst
	modified:   docs/_source/concepts.rst
	modified:   docs/_source/conf.py
	modified:   docs/_source/cookbook.rst
	modified:   docs/_source/index.rst
	modified:   docs/_source/license.rst
	modified:   docs/_source/quickstart.rst
	modified:   docs/bench-maxing.md
	modified:   docs/benchmarks.md
	modified:   docs/cti-architecture.md
	modified:   docs/diagrams/aborist-modules.dot
	modified:   docs/diagrams/aborist-modules.svg
	modified:   docs/diagrams/mesh-data-flow.dot
	modified:   docs/diagrams/mesh-epoch-lifecycle.dot
	modified:   docs/diagrams/mesh-epoch-lifecycle.svg
	modified:   docs/diagrams/mesh-group-decisions.dot
	modified:   docs/diagrams/mesh-group-decisions.svg
	modified:   docs/diagrams/mesh-identity-stack.dot
	modified:   docs/diagrams/mesh-secret-envelope.dot
	modified:   docs/mesh.md
	modified:   docs/qa-modes-bench.md
	modified:   docs/seven-point-program.md
	modified:   docs/tickets/ticket-000001-retrieval-keywords-audit-gap.md
	modified:   docs/tickets/ticket-000002-reference-frame-polarity-contract.md
	modified:   docs/tickets/ticket-000003-anchor-class-warrant.md
	modified:   docs/tickets/ticket-000005-label-ladder-migration.md
	modified:   docs/tickets/ticket-000006-bench-emergent-findings.md
	modified:   docs/tickets/ticket-000007-query-layer-hyphen-fold.md
	modified:   docs/tickets/ticket-000008-broad-quantifier-preflight-guard.md
	modified:   docs/tickets/ticket-000009-quantifier-preflight-dag-binding.md
	modified:   docs/tickets/ticket-000010-metacognition-preflight-guard.md
	modified:   docs/tickets/ticket-000011-soft-preflight-hint-sidecar.md
	modified:   scripts/backfill_concepts.py
	modified:   scripts/bench_emergent.py
	modified:   tests/crawler/test_async_web_fetcher.py
	modified:   tests/crawler/test_bridge.py
	modified:   tests/crawler/test_web_fetch.py
	modified:   tests/test_bench_qa_sweep.py
	modified:   tests/test_burn.py
	modified:   tests/test_burn_doc.py
	modified:   tests/test_claim_lattice.py
	modified:   tests/test_cli_render.py
	modified:   tests/test_compress.py
	modified:   tests/test_concepts.py
	modified:   tests/test_dag.py
	modified:   tests/test_directives.py
	modified:   tests/test_distill.py
	modified:   tests/test_distill_recursive.py
	modified:   tests/test_evict.py
	modified:   tests/test_frame.py
	modified:   tests/test_grok_source.py
	modified:   tests/test_html_source.py
	modified:   tests/test_ingest.py
	modified:   tests/test_inspect.py
	modified:   tests/test_journal.py
	modified:   tests/test_keys.py
	modified:   tests/test_llm_context_base.py
	modified:   tests/test_merkle.py
	modified:   tests/test_mesh.py
	modified:   tests/test_mesh_aead.py
	modified:   tests/test_mesh_chain.py
	modified:   tests/test_mesh_cli.py
	modified:   tests/test_mesh_cli_pull.py
	modified:   tests/test_mesh_wire.py
	modified:   tests/test_mesh_wire_e2e.py
	modified:   tests/test_metacognition.py
	modified:   tests/test_migration_audit_mode.py
	modified:   tests/test_providence_source.py
	modified:   tests/test_qa.py
	modified:   tests/test_qa_quality_live.py
	modified:   tests/test_quantifier_caps.py
	modified:   tests/test_quantifier_classifier.py
	modified:   tests/test_quantifier_phase4.py
	modified:   tests/test_quantifier_reminder.py
	modified:   tests/test_query.py
	modified:   tests/test_reclassify.py
	modified:   tests/test_repair.py
	modified:   tests/test_resume.py
	modified:   tests/test_snapshot.py
	modified:   tests/test_soft_preflight.py
	modified:   tests/test_tfidf.py
	modified:   tests/test_vcs_source.py
	modified:   tests/test_verify.py
	modified:   tests/test_verify_json.py
	modified:   tests/test_versioned_ingest.py
	modified:   tests/test_warrant.py
	modified:   tests/test_wikipedia_old.py
	modified:   tests/test_wikipedia_xml.py
	modified:   tests/test_wikitext.py
2026-05-07 09:31:49 -04:00
de07ad9392
docs: distill #000008+#000009+#000010 into core docs + diagrams
Three Explore agents fanned out in parallel for a docs/ + diagrams/
+ code-comment audit against the shipped state of the three
preflight tickets. This commit lands all the alignment fixes.

Core docs updates:

  CLAUDE.md
    - dag.py module description: stage counts now read
      "7/8 quote · 9/10 CTI · 3 reject" reflecting #000009 preflight
      stage + reject-broad early-return shape.

  docs/cti-architecture.md §2.2 + §2.3
    - §2.3 Merkle-AGI-DAG section rewritten: documents all five DAG
      shapes (legacy 7/9, post-#000009 8/10, reject-broad 3),
      describes the preflight stage's 5 nested CTI clauses
      (classifier / answer_contract / prompt_contract /
      evidence_contract / policy_refs), pins
      PREFLIGHT_NODE_VERSION = "preflight-node-v1", states the
      audit-replay payoff.
    - §2.2 CTI section: adds the four new modules
      (quantifier, model_profiles, quantifier_reminder,
      metacognition) as code anchors. Notes that pre-answer
      preflight contract extends CTI upstream of retrieval.

  docs/seven-point-program.md
    - D3 status ½ → ¾ — pre-answer preflight contract landed via
      #000008 + #000010. Code anchors + pinning tests updated.
    - D4 status ½ → ¾ — preflight stage adds upstream control
      commitment to the run-DAG. Code anchors include
      build_reject_run_dag + preflight_node_hash.
    - Status snapshot table: tickets column now references
      #000008/#000009/#000010 against D1/D3/D4 directives.
    - "Post-landing addendum (2026-05-03 / 2026-05-04)" subsection
      summarises all three tickets + their commit shas + final
      test count (993 passing, up from 734).

  docs/modules.md
    - Q&A pipeline table: added 4 new modules (quantifier.py,
      model_profiles.py, quantifier_reminder.py, metacognition.py).
      dag.py row updated to "7/8 quote · 9/10 CTI · 3 reject".
    - dag.py subsection rewritten: documents all 5 DAG shapes,
      describes the preflight payload's 5 clauses + question_state.
    - 4 new module subsections (quantifier / model_profiles /
      quantifier_reminder / metacognition) explaining each
      module's purpose, signature, and how it feeds the run-DAG
      preflight clause.

Diagram updates:

  docs/diagrams/query-pipeline.dot + .svg
    - New "PREFLIGHT (#000008 + #000010)" node inserted between
      cache_check and concepts_lookup.
    - New "REJECT-BROAD" node showing the 3-stage minimal DAG
      escape path.
    - render node label extended with the audit-line tail token
      catalog.

  docs/diagrams/aborist-modules.dot + .svg
    - 4 new qa_* nodes in the retrieval & verifier cluster.
    - 8 new edges: qa_query/qa_runner each call into all 4
      preflight modules; qa_dag has dotted edges to qa_quantifier
      + qa_metacognition (preflight clause sources).
    - qa_dag label updated to mention preflight_node_hash + 5 clauses.

  docs/diagrams/verifier-ladder.dot + .svg
    - Soft-demote violations list extended: BROAD_QUANTIFIER_RUNAWAY
      / CAP_APPLIED / SCOPE_UNBOUND, FORMAT_COLLAPSED, BARE_NAME_CLAIM.
    - New "AUDIT-LINE TAILS" annotation node listing all 11 tail
      tokens (#000008 broad-* + #000010 metacog + classic verifier).
    - Dashed edges from each rung to tails note showing tails
      compose onto labels.

Code-side stale-comment fixes (caught by 3rd Explore agent):

  aborist/qa/keys.py:218
    - "The four fields" → "The seven fields"; mention #000010 adds
      six more for metacognition.
  aborist/qa/query.py:2644
    - 7-stage / 9-stage comment expanded to enumerate all four
      base+preflight shapes plus the 3-stage reject path.
  aborist/qa/runner.py:835
    - same expansion as query.py for runner.ask() callsite.

mesh-*.dot, ingest-pipeline.dot, qa-modes-bench.md, bench-maxing.md,
bench-emergent-design.md, verifier-semantic-gap-design.md,
self-reference-design.md, concept-relations-design.md confirmed
orthogonal — no edits needed.

993 tests still passing (no behavior change). 7 files modified
across docs/ + 3 dot diagrams + 3 SVGs + 4 code-comment fixes.
2026-05-03 19:18:22 -04:00
326badf6d8
docs: README label refresh + per-module reference + Graphviz diagrams
Three things in one commit because they're tightly coupled (README
points at the diagrams; diagrams index in modules/index.md points
back at README; module pages embed the diagrams).

(1) README — label refresh:
    - Quickstart label changed from STRICT/HYBRID/UNGROUNDED to the
      four-rung ladder POINTER-LINKED → ANCHOR-WARRANTED →
      EVIDENCE-WARRANTED → UNGROUNDED with -PARTIAL suffix on HYBRID.
    - Verifier section spells out both layers (schema trichotomy +
      display ladder), the seven hard checks of claim_lattice, and
      the five anchor classes of warrant.
    - Architecture tree updated: concepts/ package added, qa/
      sub-modules expanded (warrant, evidence, parse_claims, dag),
      verify.py described as quote/span/entity/paraphrase + claim_lattice.
    - Concept overlay description updated for corpus-derived layer
      (concept_relations table, link_reciprocity extractor, 1.6%
      tax cite).
    - Test count: 326+ → 641+.

(2) docs/diagrams/ — Graphviz dot sources:
    - aborist-modules.dot — top-level package graph (substrate /
      storage / sources / retrieval / qa / mesh / cli)
    - query-pipeline.dot — question → cache → retrieval → LLM →
      verify → render → cache write, with phase budgets
    - ingest-pipeline.dot — source doc → canonicalize → chunk →
      Merkle → upsert (+ optional distill)
    - verifier-ladder.dot — (audit_mode, violations) → display rung
      decision tree
    Existing mesh-*.dot kept as-is. Makefile `make docs` target
    extended to also emit .svg alongside the existing .png so the
    diagrams render in markdown viewers.

(3) docs/modules/ — per-module reference pages:
    - index.md (links to every diagram + every module page)
    - merkle.md, document.md, store.md, ingest.md, evict.md,
      sources.md, search.md, concepts.md, qa.md, distill.md,
      wikitext.md
    Each page is a one-screenful concise reference: what the
    module is for, public API, key invariants, embedded diagrams
    where useful, link to source. Mesh stays at the existing
    docs/mesh.md + docs/mesh-deploy.md (already comprehensive).

Tests: 641 passed (no code change).
2026-05-01 23:19:01 -04:00
c7275b5618
docs: mesh multiplayer protocol with dot diagrams
Adds docs/mesh.md and five graphviz diagrams covering the federation
layer: identity stack, epoch state machine, per-member secret envelope,
gossip wire contract, and operator decision tree. Pins the protocol
contract for the upcoming HTTP wire (mesh sync, mesh serve).

Makefile gets a 'docs' target with pattern rule so PNG renders are
incremental from .dot sources.
2026-04-28 12:54:36 -04:00