Commit graph

4 commits

Author SHA1 Message Date
8d6961fcc1
aborist/arborist
modified:   .gitlab-ci.yml
	modified:   bench/qa_questions.txt
	modified:   bench/qa_sweep.py
	modified:   bench/run.sh
	modified:   docs/TICKETS.md
	modified:   docs/_source/README.md
	modified:   docs/_source/_ext/makefile_targets.py
	modified:   docs/_source/api/cli.rst
	modified:   docs/_source/api/distill.rst
	modified:   docs/_source/api/mesh.rst
	modified:   docs/_source/api/qa.rst
	modified:   docs/_source/api/retrieval.rst
	modified:   docs/_source/api/storage.rst
	modified:   docs/_source/api/substrate.rst
	modified:   docs/_source/concepts.rst
	modified:   docs/_source/conf.py
	modified:   docs/_source/cookbook.rst
	modified:   docs/_source/index.rst
	modified:   docs/_source/license.rst
	modified:   docs/_source/quickstart.rst
	modified:   docs/bench-maxing.md
	modified:   docs/benchmarks.md
	modified:   docs/cti-architecture.md
	modified:   docs/diagrams/aborist-modules.dot
	modified:   docs/diagrams/aborist-modules.svg
	modified:   docs/diagrams/mesh-data-flow.dot
	modified:   docs/diagrams/mesh-epoch-lifecycle.dot
	modified:   docs/diagrams/mesh-epoch-lifecycle.svg
	modified:   docs/diagrams/mesh-group-decisions.dot
	modified:   docs/diagrams/mesh-group-decisions.svg
	modified:   docs/diagrams/mesh-identity-stack.dot
	modified:   docs/diagrams/mesh-secret-envelope.dot
	modified:   docs/mesh.md
	modified:   docs/qa-modes-bench.md
	modified:   docs/seven-point-program.md
	modified:   docs/tickets/ticket-000001-retrieval-keywords-audit-gap.md
	modified:   docs/tickets/ticket-000002-reference-frame-polarity-contract.md
	modified:   docs/tickets/ticket-000003-anchor-class-warrant.md
	modified:   docs/tickets/ticket-000005-label-ladder-migration.md
	modified:   docs/tickets/ticket-000006-bench-emergent-findings.md
	modified:   docs/tickets/ticket-000007-query-layer-hyphen-fold.md
	modified:   docs/tickets/ticket-000008-broad-quantifier-preflight-guard.md
	modified:   docs/tickets/ticket-000009-quantifier-preflight-dag-binding.md
	modified:   docs/tickets/ticket-000010-metacognition-preflight-guard.md
	modified:   docs/tickets/ticket-000011-soft-preflight-hint-sidecar.md
	modified:   scripts/backfill_concepts.py
	modified:   scripts/bench_emergent.py
	modified:   tests/crawler/test_async_web_fetcher.py
	modified:   tests/crawler/test_bridge.py
	modified:   tests/crawler/test_web_fetch.py
	modified:   tests/test_bench_qa_sweep.py
	modified:   tests/test_burn.py
	modified:   tests/test_burn_doc.py
	modified:   tests/test_claim_lattice.py
	modified:   tests/test_cli_render.py
	modified:   tests/test_compress.py
	modified:   tests/test_concepts.py
	modified:   tests/test_dag.py
	modified:   tests/test_directives.py
	modified:   tests/test_distill.py
	modified:   tests/test_distill_recursive.py
	modified:   tests/test_evict.py
	modified:   tests/test_frame.py
	modified:   tests/test_grok_source.py
	modified:   tests/test_html_source.py
	modified:   tests/test_ingest.py
	modified:   tests/test_inspect.py
	modified:   tests/test_journal.py
	modified:   tests/test_keys.py
	modified:   tests/test_llm_context_base.py
	modified:   tests/test_merkle.py
	modified:   tests/test_mesh.py
	modified:   tests/test_mesh_aead.py
	modified:   tests/test_mesh_chain.py
	modified:   tests/test_mesh_cli.py
	modified:   tests/test_mesh_cli_pull.py
	modified:   tests/test_mesh_wire.py
	modified:   tests/test_mesh_wire_e2e.py
	modified:   tests/test_metacognition.py
	modified:   tests/test_migration_audit_mode.py
	modified:   tests/test_providence_source.py
	modified:   tests/test_qa.py
	modified:   tests/test_qa_quality_live.py
	modified:   tests/test_quantifier_caps.py
	modified:   tests/test_quantifier_classifier.py
	modified:   tests/test_quantifier_phase4.py
	modified:   tests/test_quantifier_reminder.py
	modified:   tests/test_query.py
	modified:   tests/test_reclassify.py
	modified:   tests/test_repair.py
	modified:   tests/test_resume.py
	modified:   tests/test_snapshot.py
	modified:   tests/test_soft_preflight.py
	modified:   tests/test_tfidf.py
	modified:   tests/test_vcs_source.py
	modified:   tests/test_verify.py
	modified:   tests/test_verify_json.py
	modified:   tests/test_versioned_ingest.py
	modified:   tests/test_warrant.py
	modified:   tests/test_wikipedia_old.py
	modified:   tests/test_wikipedia_xml.py
	modified:   tests/test_wikitext.py
2026-05-07 09:31:49 -04:00
b48646bc0c
mesh: sync default-holds records younger than 1 hour (kindergarten window)
Fox 2026-04-29: "part of the gossip protocol should be some default
delay that allows a user to catch and burn kindergarteners before
they are synced." Currently `mesh sync` enumerates the most-recent
N items with no age filter — a doc ingested 30 seconds ago goes
out on the next sync, & if a peer ingests it before the operator
notices a problem, burn no longer suffices (peer has its own copy).

Adds a sender-side kindergarten window:

  --kindergarten-seconds N   default 3600 (1 hour)

Records younger than `now - N` are held back from broadcast. Both
ANNOUNCE_ROOT (filtered on documents.ingest_ts) & ANNOUNCE_FALSIFICATION
(filtered on falsifications.at). N=0 = broadcast everything (cron-
friendly opt-out for operators preferring immediate propagation).

Result JSON now reports the held counts so the operator can see
what the window protected:

  kindergarten_seconds: 3600
  announced_roots: 12
  kindergarten_held_roots: 3
  announced_falsifications: 0
  kindergarten_held_falsifications: 1

Sender-side discipline only — the receiver has no view into when
the sender created the record, so it can't enforce. Adding a
created_at on the envelope would let receivers reject too-fresh
gossip, but that's a future protocol bump (envelopes today don't
carry sender wall-clock; ts is the send time).

Tests:
  - existing tests now pass `--kindergarten-seconds 0` so freshly-
    ingested fixtures broadcast immediately for the test
  - new test_sync_kindergarten_holds_fresh_records: a 30-second-old
    doc is held; an artificially-aged doc broadcasts. announced=1,
    held>=1.
  - new test_sync_kindergarten_zero_broadcasts_everything: explicit
    opt-out works.

342 passed, 1 skipped.
2026-04-29 16:07:24 -04:00
01340013dd
mesh: sync now also broadcasts ANNOUNCE_FALSIFICATION by default
Per fox 2026-04-29: the wire protocol has had ANNOUNCE_FALSIFICATION
since the foundation commit (and MeshWireClient.announce_falsification
since AEAD landed), but the user-facing `mesh sync` only ever fired
ANNOUNCE_ROOT. Falsifications local to one peer never reached others
unless an operator hand-rolled a Python script.

Now `mesh sync` enumerates BOTH categories:
  - ANNOUNCE_ROOT       most-recent --limit documents (existing path)
  - ANNOUNCE_FALSIFICATION  most-recent --limit falsifications (new)

Receivers verify Ed25519 sig + per-peer chain-of-claims as before,
write one mesh_received audit event per accepted envelope. Result
JSON now reports both counts:

  announced_roots: N      (was: announced)
  announced_falsifications: N
  sent_roots: [...]       (was: sent)
  sent_falsifications: [...]

Two opt-out flags so operators can scope the broadcast:
  --no-roots               only push falsifications
  --no-falsifications      only push roots

Burns are deliberately NOT propagated. Burn semantics are local
kindergarten cleanup ("delete a leaf I shouldn't have written") —
other peers may have legitimately ingested the doc independently.
Falsify is the audit-preserving alternative whose broadcast IS the
right cross-peer signal for "this answer is wrong."

Caveat (deferred): no per-peer dedup state yet. Re-running sync
re-broadcasts the same most-recent N falsifications; receivers get
duplicate mesh_received audit-log entries (no state corruption,
just log noise). A `mesh_sync_state` table tracking
last_falsify_announced_ts per peer URL is the natural follow-up
when the falsification volume grows.

Tests:
  - existing test_sync_announces_local_roots updated for new field
    names (announced_roots, announced_falsifications)
  - new test_sync_announces_falsifications: bob falsifies, syncs,
    alice's chain has the ANNOUNCE_FALSIFICATION envelope
  - new test_sync_no_falsifications_flag_skips_them: --no-falsifications
    skips the broadcast cleanly

339 passed, 1 skipped.
2026-04-29 15:40:53 -04:00
5427a61768
mesh: 'serve' and 'sync' CLI verbs over the wire layer
Wires the gossip-wire core (commit f141bab) into the user-facing CLI:

  aborist mesh serve --host 127.0.0.1 --port 8400
    Spawns MeshWireServer, blocks until SIGINT. Refuses if mesh isn't
    initialized or .enabled is off — same gate every other mesh verb
    enforces. Stdout is unbuffered JSON status lines.

  aborist mesh sync --peer http://other.example:8400 [--limit N]
    Enumerates local documents (most-recent N, default 100), fires one
    ANNOUNCE_ROOT per document via MeshWireClient, reports counts of
    acknowledged vs. errored. Pulling missing roots back from the peer
    is a v2 addition — this verb pushes only.

Tests (6): gating refusals on uninitialized + disabled mesh; sync
round-trip with two peers proving Alice's audit chain gains one
'mesh_received' per Bob announce; unreachable-peer reporting; serve
smoke-test via start_in_thread + GET /mesh/info.
2026-04-28 17:05:31 -04:00