arborist/tests
russell@unturf.com 8da29fde69
ticket #000044: AUTOCOUNT doc-drift discipline + code-fence parser fix
Lock the AUTOCOUNT regression-test pattern as the design log
canonical record. Previously declined when surface was 1-metric
+ 29 tags; now mature enough (4 metrics + 58 tags + 1 same-day
drift-catch since landing) to formalize.

== Ticket content ==

10 sections covering:

1. Why this exists — the 4-drift-day baseline (6cbbf95 / 14bcb99 /
   5c21e83 / 30a9488) that motivated mechanization. Five-step
   walk through justifying each choice (Step 5 last).
2. Format — `<!--AUTOCOUNT:metric:path-->N<!--/AUTOCOUNT-->`.
3. Four supported metrics with examples + skip semantics:
   `tests`, `fixture-rows`, `db-rows`, `db-where`.
4. Skip-on-absence — operator state (shards, qa.db) absence is a
   logged skip, not a fail. Smoke verified 2026-05-10 with
   HOME=/tmp/empty.
5. What NOT to tag — closed-ticket point-in-time snapshots,
   aggregate floors ("2000+"), historical journey arcs.
6. Install discipline at write time + at refresh time.
7. Future metrics deferred (file-lines, gh-pr-comments-count,
   module-loc, commit-hash-exists) with the "add a metric"
   recipe.
8. Empirical baseline at landing (3 test functions, 58 active
   tagged claims across 8 doc files, harness runtime 2-4s).
9. Scope boundaries — does NOT auto-rewrite, does NOT validate
   prose quality, does NOT scan docstrings, does NOT lock
   values, does NOT add deps.
10. References — every landing commit + sister doc.

Closed at landing (status quo since fc5ba50 2026-05-10 morning;
this ticket is retroactive design log per the convention "every
ticket flips to `closed · landed in commit <sha>` when the work
ships").

== Code-fence parser fix ==

Adding the ticket itself surfaced an oversight: my AUTOCOUNT
examples in §3.3 + §3.4 used literal tag pairs in ``` fenced
code blocks. The parser was reading them as live claims and
firing on the illustrative `db-rows:002.db:concept_relations`
claim (compared 1234 vs live 72576 — both meaningless because
it's an example).

Fix: `_strip_fenced_code_blocks` substitutes the body of every
triple-backtick block with newlines before regex scanning. Line
numbers stay aligned (newline-preserving substitution); tags
inside fences are skipped because their parent text no longer
matches the regex.

Both helper functions (`_iter_claims` and the well-formed-tags
test) walk through the stripped text, so the strip discipline
is consistent across all three test functions.

== TICKETS.md index ==

Added #000044 row marked closed with the 5-commit landing trail.
Bumped Next ID 000044 → 000045.

== Verification ==

  $ pytest tests/test_doc_counts.py
  3 passed in 2.80s

  $ pytest tests/ -q
  2337 passed, 37 skipped in 108.29s

Hygiene: fox's in-flight changes to arborist/qa/runner.py +
arborist/substrate/prometheus.py + tests/test_prometheus*.py
left untouched in working tree.
2026-05-10 18:24:50 -04:00
..
crawler aborist/arborist 2026-05-07 09:31:49 -04:00
fixtures verify: strip wikitext from context before substring matching 2026-04-28 15:48:07 -04:00
__init__.py phase 0 explore: aborist core + sources + distill + evict 2026-04-27 07:53:18 -04:00
test_aliases.py tests/aliases: gap-fill list_term_aliases + tokenizer helpers (10 new tests) 2026-05-10 12:35:33 -04:00
test_anchor_prg.py tests: rename v8 → substrate + close 9-item checklist gaps across 3 files 2026-05-10 13:35:52 -04:00
test_bench_batteries.py bench/fixtures/5f: harvest 40 corpus-derived falsification fixtures (#000037#000025) 2026-05-10 17:47:07 -04:00
test_bench_qa_sweep.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_burn.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_burn_doc.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_canonical_cache.py #000027 + #000028: canonical projections persist; STRICT-WITNESSED reachable 2026-05-09 11:37:06 -04:00
test_canonical_projection.py ticket #000030 Phases 1+2: algebra-symbolic@v1 + calculus-derivative@v1 2026-05-09 12:35:18 -04:00
test_capital.py capital: land ticket #000020 (8-capital-form cost ledger) 2026-05-07 16:41:48 -04:00
test_claim_lattice.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_claim_pack.py ticket #000029: claim-pack source for axiom/theorem JSON bundles 2026-05-09 11:44:08 -04:00
test_cli_render.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_cli_session.py tests: unit + integration + functional coverage for this session's surface 2026-05-07 21:15:14 -04:00
test_cli_smoke.py tests/cli_smoke: 68 argparse-construction smoke tests for the full CLI surface 2026-05-10 13:16:06 -04:00
test_compress.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_concepts.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_concepts_extract.py tests/concepts/extract: 20 tests for synonym/idf/fts5 extractors (was zero coverage) 2026-05-10 12:47:58 -04:00
test_dag.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_demo_plot.py fan-out: witness audit + 5F extractor + function-sampled demo + docs 2026-05-09 13:19:30 -04:00
test_directives.py #000027 + #000028: canonical projections persist; STRICT-WITNESSED reachable 2026-05-09 11:37:06 -04:00
test_distill.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_distill_recursive.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_doc_counts.py ticket #000044: AUTOCOUNT doc-drift discipline + code-fence parser fix 2026-05-10 18:24:50 -04:00
test_evict.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_fork_score.py tests/fork_score: 18 tests for v8 ForkScore (#000012 Phase 1a — was zero coverage) 2026-05-10 12:38:39 -04:00
test_frame.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_grok_source.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_html_source.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_ingest.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_inspect.py fan-out: warrant ladder wiring · witness follow-ups · 5F Phase 1d 2026-05-09 12:42:56 -04:00
test_journal.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_keys.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_llm_context_base.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_loss_report.py loss_report: land ticket #000022 (adapter LossReport sidecar) 2026-05-07 17:58:50 -04:00
test_memory_root.py memory: land ticket #000017 (memory-root lifelong learning summary) 2026-05-07 16:46:41 -04:00
test_merkle.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh_aead.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh_chain.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh_cli.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh_cli_pull.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh_wire.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_mesh_wire_e2e.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_metacognition.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_migration_audit_mode.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_phi_alignment_probe.py tests: rename v8 → substrate + close 9-item checklist gaps across 3 files 2026-05-10 13:35:52 -04:00
test_pi_star.py fan-out: 5 π* graduations close the registry chapter 2026-05-09 13:04:43 -04:00
test_pi_star_algebra_symbolic.py ticket #000030 Phases 1+2: algebra-symbolic@v1 + calculus-derivative@v1 2026-05-09 12:35:18 -04:00
test_pi_star_algebra_symbolic_simplified.py ticket #000030 Phases 1b+3 + open #000031 2026-05-09 12:50:46 -04:00
test_pi_star_arithmetic.py tests/pi_star_arithmetic: 56 KATs for exact-rational arithmetic kernel 2026-05-10 15:54:37 -04:00
test_pi_star_calculus_derivative.py ticket #000030 Phases 1+2: algebra-symbolic@v1 + calculus-derivative@v1 2026-05-09 12:35:18 -04:00
test_pi_star_calculus_integral.py ticket #000030 Phases 1b+3 + open #000031 2026-05-09 12:50:46 -04:00
test_pi_star_code.py tests/pi_star_code: 32 KATs for Python-AST kernel 2026-05-10 15:54:49 -04:00
test_pi_star_combinatorics.py ticket #000032: combinatorics@v1 π* (pure-integer counting kernel) 2026-05-09 13:20:29 -04:00
test_pi_star_compositions.py fan-out: close #000030 · composition fixtures · witness end-to-end 2026-05-09 13:29:59 -04:00
test_pi_star_logic.py tests/pi_star_logic: 53 KATs for propositional CNF kernel 2026-05-10 15:54:43 -04:00
test_pi_star_phase_3_to_7.py fan-out: 5 π* graduations close the registry chapter 2026-05-09 13:04:43 -04:00
test_pi_star_protocol_and_registry.py tests/pi_star: 21 tests for protocol + registry (foundation, was untested) 2026-05-10 12:44:13 -04:00
test_pi_star_tabular.py fan-out: 5 π* graduations close the registry chapter 2026-05-09 13:04:43 -04:00
test_pi_star_time_series.py tests/pi_star_time_series: 35 KATs for temporal-signal kernel 2026-05-10 15:54:55 -04:00
test_prometheus.py #000037 Phase 1 continued: §14 row 4 + §13 step 11 + weight-tunable gates 2026-05-10 16:56:18 -04:00
test_prometheus_audit.py arborist/substrate/prometheus_audit: Phase 2 advisory audit writes (#000037) 2026-05-10 16:32:02 -04:00
test_providence_source.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_qa.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_qa_progress.py tests/qa/progress: 31 tests for Progress emitter (was zero coverage) 2026-05-10 12:45:32 -04:00
test_qa_prompts.py tests/qa/prompts: 20 tests pinning load-bearing system prompts (was zero coverage) 2026-05-10 12:50:20 -04:00
test_qa_quality_live.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_quantifier_caps.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_quantifier_classifier.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_quantifier_phase4.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_quantifier_reminder.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_query.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_reclassify.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_repair.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_resume.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_search_fts5.py search/fts5: progressive-AND fallback + DF filter at OR-pool 2026-05-09 18:31:24 -04:00
test_selfmodel.py selfmodel: land ticket #000014 (identity record + falsification) 2026-05-07 16:36:34 -04:00
test_session_integration.py #000025 Phase 1e: 5F falsification motif coverage (§10.12 closed) 2026-05-10 16:04:34 -04:00
test_session_migrations.py tests: unit + integration + functional coverage for this session's surface 2026-05-07 21:15:14 -04:00
test_snapshot.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_soft_preflight.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_store_migration_memoization.py store: per-process migration memoization (#000026 Phase 1) 2026-05-08 12:48:48 -04:00
test_substrate_fork_score.py tests: rename v8 → substrate + close 9-item checklist gaps across 3 files 2026-05-10 13:35:52 -04:00
test_t3_bound_calculator.py tests/t3_bound_calculator: close the two checklist gaps from §3 audit 2026-05-10 13:26:30 -04:00
test_textbook_tex.py textbooks: TeX-source ingest closes pillars I + IV (Hilbert + Boole) 2026-05-09 16:06:53 -04:00
test_textbooks_manifest.py tests/textbooks_manifest: 43 tests for license-discipline gate (was zero coverage) 2026-05-10 13:02:17 -04:00
test_tfidf.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_vcs_source.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_verify.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_verify_json.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_versioned_ingest.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_warrant.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_warrant_authorship.py fan-out: warrant ladder wiring · witness follow-ups · 5F Phase 1d 2026-05-09 12:42:56 -04:00
test_warrant_chain.py ticket #000031 Phase 3: warrant-chain-aware verifier suppresses WARRANT_MISSING 2026-05-10 09:17:48 -04:00
test_warrant_resolver.py ticket #000031 follow-ups B-1 + B-2: alias attribution + source-side title author 2026-05-10 09:35:49 -04:00
test_weights.py tests/weights: 16 tests for WeightSet defaults + from_dict adapter 2026-05-10 12:41:49 -04:00
test_wikipedia_old.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_wikipedia_xml.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_wikitext.py aborist/arborist 2026-05-07 09:31:49 -04:00
test_witness.py fan-out: warrant ladder wiring · witness follow-ups · 5F Phase 1d 2026-05-09 12:42:56 -04:00
test_witness_to_5f.py fan-out: witness audit + 5F extractor + function-sampled demo + docs 2026-05-09 13:19:30 -04:00
test_world_namespace.py #000013 closed: v7-W spatial-temporal substrate paper + namespace 2026-05-09 15:00:05 -04:00