arborist/docs
russell@unturf.com 6c6defbcb2
tests/doc_counts: db-where metric + tag prometheus controller (#000037 Phase 1+2)
Fan-out follow-up: extends AUTOCOUNT with a new metric for filtered
SQL-row claims, then tags fox's prometheus controller test surfaces
shipped this evening under #000037.

== Task 3: db-where metric ==

New metric ``db-where`` for tagging single-column equality
predicates. Target syntax::

    <table>?<column>=<value>
    <shard>:<table>?<column>=<value>

Resolves to ``SELECT COUNT(*) FROM <table> WHERE <column> = ?``
with ``<value>`` bound as a SQL parameter (no string
interpolation), so author typos or stray content can't escape
the predicate. Column + table names validated as bare
identifiers before string-interpolating into the query template;
sqlite3 connection opens with ``mode=ro`` URI flag.

Same skip-on-absence semantics as ``db-rows``: missing DB or
table yields a logged skip note, not a test failure. Sentinel
returns reuse the same _DB_MISSING / _TABLE_MISSING / _DB_ERROR
constants.

Smoke verified::

    _live_db_where('documents?source_type=claim_pack')      → 92
    _live_db_where('documents?source_type=wikipedia_xml')   → 866782
    _live_db_where('001.db:documents?source_type=wikipedia_xml')
                                                            → 867695
    _live_db_where('documents')  # malformed (no ?)         → -4

Tagged claims using the new metric (cookbook):

- L36 ``92 records total`` for the claim-pack source
- L349 ``The 92 chains have three quality tiers``

Both resolve to ``documents WHERE source_type='claim_pack'``
in shard ``000.db`` — the live count of claim-pack records.

L5 ``18/92 → 92/92`` historical narrative left untagged
(expressing a journey arc, not current state).

== Task 2: tag prometheus controller test surfaces ==

fox shipped two test files this evening under #000037 that
weren't previously inventoried in any reference doc:

- ``tests/test_prometheus.py`` — 36 tests covering Phase 1
  controller (commits ``f625cac`` + ``f9f5ae4``). Verifier-style
  discipline (NOT calculator pattern — it's a pure-function
  state-machine controller with no closed-form math).
- ``tests/test_prometheus_audit.py`` — 14 tests covering Phase 2
  ``controller_events`` sibling table (commit ``a786d6d``). Pins
  no-chain-mutation invariant (advisory writes never enter
  audit_events.event_hash preimage).

Added two paragraphs to cookbook §"Substrate-paper-spec'd
primitives" describing the test discipline + algorithmic surfaces
each pins. Renamed section header from
"(#000012 + #000018 + #000034)" to
"(#000012 + #000018 + #000034 + #000037)" to keep the
ticket-set roster current.

Two new rows in the test/code-density table:

  | substrate/prometheus.py       | 893 | 804 (36 tests) | 0.90 |
  | substrate/prometheus_audit.py | 200 | 388 (14 tests) | 1.94 |

All four counts AUTOCOUNT-tagged (2 inline prose + 2 table rows
= 4 new tags). prometheus_audit's ratio of 1.94 is high because
the test file pins a lot of write-path invariants for what is
nominally a small (200 LOC) sibling-table module — appropriate
for foundation-level audit-discipline code.

== Task 4: test_full_suite_total_fixture_count flake — investigated ==

Earlier today's transient ``-x`` flake (1 failed, 1799 passed)
did NOT reproduce in current tree state (2328 passed, 37 skipped
under same flags). Root-cause investigation:

- ``_DEFAULT_FIXTURES`` is read-only at module scope; no test
  mutates it.
- The test reads JSONL fixtures from ``bench/fixtures/`` via
  ``_run_one``; those files weren't being written by parallel
  tests.

Hypothesis (not confirmed, since flake didn't repro): transient
filesystem state during heavy-parallel-commit window
(``f625cac`` / ``a786d6d`` / ``6142437`` / ``f9f5ae4`` all
landed in succession around 17:23-17:24 EDT 2026-05-10 while my
test run was in flight). No structural defect identified. If
flake recurs, capture stdout + filesystem state at failure time
to confirm.

== Task 1: Walked fox's 6 evening commits via Explore agent ==

Agent reported what shipped under #000037 + #000012 evening
push (Prometheus-Σ Phases 0/1/2/dry-run + v8 consensus paper).
Findings used to drive task 2 above. Notable design choices
worth surfacing as reference:

- prometheus.py is **pure function** (no DB / LLM / scheduler);
  returns advisory ``ControllerDecision`` + optional proposal
  records, never mutations. Verifier-style test discipline.
- ``controller_events`` sibling table never enters
  ``audit_events.event_hash`` preimage — audit chain unaffected.
- Phase 3 sleep-sweep scheduler **deferred** in favor of
  read-only dry-run simulator surfacing 5 design constraints
  (chunk_size = Hermes concurrency NOT candidate pool;
  capital_cost flat=1.0 needs split; τ_qa per audit_mode;
  Target B = 4.4% canonical-shape match; quarantined-row veto
  exercises end-to-end). Calibration substrate for eventual
  scheduler.
- Per-branch controller latency 12.5 µs at chunk_size=4 → not
  the bottleneck; Hermes witness fan-out is.

v8 consensus paper at ``docs/_source/merkle-agi-v8-consensus.rst``
(834 lines; 11 parts) closes the loop from single-validator
Proof-of-Upgrade to multi-validator BFT selection. Phase 1c
(branch-set persistence) remains proposed-not-opened.

== Coverage ==

  Total tags after this commit:   54 (was 49; +5)
  Tags by metric:
    tests:           45 (+4 new prometheus + table rows)
    fixture-rows:     2
    db-rows:          3
    db-where:         2 (new metric, both 92 claim-pack)

  Files with tags:
    docs/warrant-substrate-cookbook.md             32 (+5)
    docs/calculator-test-patterns.md                8
    docs/soft-hash-channel-analysis.md              5
    docs/tickets/ticket-000006-bench-emergent...    4
    docs/seven-point-program.md                     3
    docs/tickets/ticket-000035-prg-choice-phi-prg.md 2

== Verification ==

  $ pytest tests/test_doc_counts.py -v
  3 passed in 2.60s

  $ pytest tests/ -q
  2328 passed, 37 skipped in 106.06s
2026-05-10 17:51:08 -04:00
..
_source #000012 Phase 1b: Merkle-AGI v8 consensus paper landed 2026-05-10 16:12:00 -04:00
diagrams docs/diagrams: rename aborist-modules → arborist-modules (post-rename cleanup) 2026-05-08 07:50:28 -04:00
tickets update everything with today's learnings — math review forward + Phase 3 dry-run amend 2026-05-10 16:57:19 -04:00
bench-maxing.md aborist/arborist 2026-05-07 09:31:49 -04:00
benchmarks.md aborist/arborist 2026-05-07 09:31:49 -04:00
calculator-test-patterns.md docs/calculator-test-patterns: 3 new patterns from π* kernel work + 4 exemplar files 2026-05-10 16:33:19 -04:00
cti-architecture.md aborist/arborist 2026-05-07 09:31:49 -04:00
mesh.md aborist/arborist 2026-05-07 09:31:49 -04:00
pi-star-composition.md pi_star: land ticket #000015 (π* domain library + composition algebra) 2026-05-07 16:51:33 -04:00
qa-modes-bench.md aborist/arborist 2026-05-07 09:31:49 -04:00
seven-point-program.md tests/doc_counts: regression test for numeric claims in docs/ (4x drift fix) 2026-05-10 16:15:52 -04:00
soft-hash-channel-analysis.md tests/doc_counts: regression test for numeric claims in docs/ (4x drift fix) 2026-05-10 16:15:52 -04:00
soft-hash-channel-t3-bound.md soft-hash T3 bound: pre-review polish pass for external cryptographer 2026-05-10 15:59:16 -04:00
spec-methodology.md docs: land ticket #000019 (spec methodology for π*, V, policy fields) 2026-05-07 16:53:28 -04:00
TICKETS.md docs/TICKETS: refresh #000006 status row + calculator-test-patterns.md description 2026-05-10 17:00:37 -04:00
tool-action-dag-design.md docs: add tool-action-dag-design.md research path (pre-ticket) 2026-05-07 19:47:50 -04:00
v7w-frontier-catalog.md #000013 closed: v7-W spatial-temporal substrate paper + namespace 2026-05-09 15:00:05 -04:00
v8-fork-score.md CLI: arborist v8 score → arborist substrate score 2026-05-10 09:12:34 -04:00
warrant-substrate-cookbook.md tests/doc_counts: db-where metric + tag prometheus controller (#000037 Phase 1+2) 2026-05-10 17:51:08 -04:00
zk-frontier-bench.md #000016 parked: ZK frontier-proof bench plan + wire protocol 2026-05-09 15:05:08 -04:00
zk-wire-protocol.md #000016 parked: ZK frontier-proof bench plan + wire protocol 2026-05-09 15:05:08 -04:00