engineering/arborist
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
arborist/tests
History
russell@unturf.com 60b5748ff9
fan-out: ForkScore CLI · authorship warrant ladder · 5F Phase 1c 
Three streams in one commit since they're independent and each is
small.

#000012 Phase 1b — ForkScore CLI surface
========================================

`arborist v8 score` already existed; this adds `--out` for JSON-
artifact emission so CI / downstream graders / mesh peers can
ingest without parsing stdout. New Makefile targets:

- `make bench-fork-baseline` — pins current bench-suite output as
  the ForkScore parent (one-shot per iteration).
- `make bench-fork-score` — runs bench-suite again, scores child
  vs pinned parent, writes bench/results/fork_score_report.json.
  Exit 1 on REJECT so CI can gate.

`FORK_PARENT` / `FORK_CHILD` / `FORK_REPORT` env-vars override
default paths. New regression test pins the --out contract:
stdout and file are byte-identical artifacts; --out auto-creates
parent directories.

#000026 Phase 3 — authorship warrant ladder
============================================

Sidecar classifier in `arborist/qa/warrant_authorship.py`. Six
tiers strongest-to-weakest: AUTHOR_PACKAGE_METADATA →
AUTHOR_REPOSITORY_OWNER → AUTHOR_PAGE_BYLINE →
AUTHOR_PRIMARY_PAGE_TITLE → AUTHOR_COPYRIGHT_FOOTER →
AUTHOR_SECONDARY_SOURCE. Plus NO_AUTHORSHIP_SIGNAL when the
question doesn't smell like authorship (sidecar stays quiet).

Detector regexes for each tier:
- Tier 1: `author = "X"` simple form + TOML inline-table
  `authors = [{ name = "X" }]` form (PEP 621).
- Tier 2: github.com / gitlab.com / codeberg.org / bitbucket.org
  URL pattern.
- Tier 3: "By NAME" / "Author: NAME" prose + <meta name="author">.
  Inline-flag regex keeps the prefix case-insensitive while the
  capitalized-name capture stays case-sensitive.
- Tier 4: cited evidence is the entity's own primary page (host
  tokens overlap title + answer; third-party indexers like
  wikipedia.org explicitly excluded).
- Tier 5: `© NAME` / `Copyright YYYY NAME` (the current
  `virt-back` warrant).
- Tier 6: fall-through when authorship-shaped question hits cited
  evidence with no direct markers.

Sidecar discipline: never enters proof path; never raises;
returns dict with `tier`, `tier_rank` (1=strongest, 99=quiet),
`signals`, `candidate_names`, `note`. 20 tests cover each tier
+ noise filtering + sidecar contract + tier-ordering (strongest
wins when multiple fire).

Wiring into `arborist inspect` sidecar output + audit-line
render-tail is queued as a follow-up — sidecar itself ready.

#000025 5F Phase 1c — fixture catalog expansion
================================================

Synthetic side of all five 5F sub-batteries expanded 10 → 30:

  function       — varied claim_count, pointer_set, threshold cases
  falsification  — 13 violation tags (WARRANT_MISSING, TITLE_MISMATCH,
                   FORMAT_COLLAPSED, NO_EVIDENCE_POINTER, BARE_NAME_CLAIM,
                   LAZY_ANCHOR_DEMOTED, etc.) + 7 fail cases
  feedback-loop  — 10 chain templates × 2 cycles
  finetuning     — 20 capability transitions across all 5S/5T/5F/5R
                   sub-batteries + canonical math/logic
  formulate      — 12 lattice shapes × 2 (with deliberate fail cases)

150/150 fixtures pass through `bench-5f-*` runners.
test_session_integration.py total updated 462 → 562. Pinned
test_5f_*_runs counts updated 10 → 30 across all assertions.

Tests
=====

Full suite: 1388 passed, 36 skipped (was 1367; +21 — 20 warrant
tests + 1 ForkScore --out test).
2026-05-09 12:14:38 -04:00
..
crawler aborist/arborist 2026-05-07 09:31:49 -04:00
fixtures verify: strip wikitext from context before substring matching 2026-04-28 15:48:07 -04:00
__init__.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_bench_batteries.py fan-out: ForkScore CLI · authorship warrant ladder · 5F Phase 1c 2026-05-09 12:14:38 -04:00
test_bench_qa_sweep.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_burn.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_burn_doc.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_canonical_cache.py #000027 + #000028: canonical projections persist; STRICT-WITNESSED reachable 2026-05-09 11:37:06 -04:00
test_canonical_projection.py #000027 + #000028: canonical projections persist; STRICT-WITNESSED reachable 2026-05-09 11:37:06 -04:00
test_capital.py capital: land ticket #000020 (8-capital-form cost ledger) 2026-05-07 16:41:48 -04:00
test_claim_lattice.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_claim_pack.py ticket #000029: claim-pack source for axiom/theorem JSON bundles 2026-05-09 11:44:08 -04:00
test_cli_render.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_cli_session.py tests: unit + integration + functional coverage for this session's surface 2026-05-07 21:15:14 -04:00
test_compress.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_concepts.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_dag.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_directives.py #000027 + #000028: canonical projections persist; STRICT-WITNESSED reachable 2026-05-09 11:37:06 -04:00
test_distill.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_distill_recursive.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_evict.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_frame.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_grok_source.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_html_source.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_ingest.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_inspect.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_journal.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_keys.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_llm_context_base.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_loss_report.py loss_report: land ticket #000022 (adapter LossReport sidecar) 2026-05-07 17:58:50 -04:00
test_memory_root.py memory: land ticket #000017 (memory-root lifelong learning summary) 2026-05-07 16:46:41 -04:00
test_merkle.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh_aead.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh_chain.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh_cli.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh_cli_pull.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh_wire.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh_wire_e2e.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_metacognition.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_migration_audit_mode.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_pi_star.py pi_star: time-series-quantized@v1 graduates + Substrate docs section 2026-05-08 09:33:19 -04:00
test_providence_source.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_qa.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_qa_quality_live.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_quantifier_caps.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_quantifier_classifier.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_quantifier_phase4.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_quantifier_reminder.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_query.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_reclassify.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_repair.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_resume.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_selfmodel.py selfmodel: land ticket #000014 (identity record + falsification) 2026-05-07 16:36:34 -04:00
test_session_integration.py fan-out: ForkScore CLI · authorship warrant ladder · 5F Phase 1c 2026-05-09 12:14:38 -04:00
test_session_migrations.py tests: unit + integration + functional coverage for this session's surface 2026-05-07 21:15:14 -04:00
test_snapshot.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_soft_preflight.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_store_migration_memoization.py store: per-process migration memoization (#000026 Phase 1) 2026-05-08 12:48:48 -04:00
test_tfidf.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_v8_fork_score.py fan-out: ForkScore CLI · authorship warrant ladder · 5F Phase 1c 2026-05-09 12:14:38 -04:00
test_vcs_source.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_verify.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_verify_json.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_versioned_ingest.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_warrant.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_warrant_authorship.py fan-out: ForkScore CLI · authorship warrant ladder · 5F Phase 1c 2026-05-09 12:14:38 -04:00
test_wikipedia_old.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_wikipedia_xml.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_wikitext.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_witness.py #000027 + #000028: canonical projections persist; STRICT-WITNESSED reachable 2026-05-09 11:37:06 -04:00