engineering/arborist
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

tests/doc_counts: db-where metric + tag prometheus controller (#000037 Phase 1+2)

Browse source 
Fan-out follow-up: extends AUTOCOUNT with a new metric for filtered
SQL-row claims, then tags fox's prometheus controller test surfaces
shipped this evening under #000037.

== Task 3: db-where metric ==

New metric ``db-where`` for tagging single-column equality
predicates. Target syntax::

    <table>?<column>=<value>
    <shard>:<table>?<column>=<value>

Resolves to ``SELECT COUNT(*) FROM <table> WHERE <column> = ?``
with ``<value>`` bound as a SQL parameter (no string
interpolation), so author typos or stray content can't escape
the predicate. Column + table names validated as bare
identifiers before string-interpolating into the query template;
sqlite3 connection opens with ``mode=ro`` URI flag.

Same skip-on-absence semantics as ``db-rows``: missing DB or
table yields a logged skip note, not a test failure. Sentinel
returns reuse the same _DB_MISSING / _TABLE_MISSING / _DB_ERROR
constants.

Smoke verified::

    _live_db_where('documents?source_type=claim_pack')      → 92
    _live_db_where('documents?source_type=wikipedia_xml')   → 866782
    _live_db_where('001.db:documents?source_type=wikipedia_xml')
                                                            → 867695
    _live_db_where('documents')  # malformed (no ?)         → -4

Tagged claims using the new metric (cookbook):

- L36 ``92 records total`` for the claim-pack source
- L349 ``The 92 chains have three quality tiers``

Both resolve to ``documents WHERE source_type='claim_pack'``
in shard ``000.db`` — the live count of claim-pack records.

L5 ``18/92 → 92/92`` historical narrative left untagged
(expressing a journey arc, not current state).

== Task 2: tag prometheus controller test surfaces ==

fox shipped two test files this evening under #000037 that
weren't previously inventoried in any reference doc:

- ``tests/test_prometheus.py`` — 36 tests covering Phase 1
  controller (commits ``f625cac`` + ``f9f5ae4``). Verifier-style
  discipline (NOT calculator pattern — it's a pure-function
  state-machine controller with no closed-form math).
- ``tests/test_prometheus_audit.py`` — 14 tests covering Phase 2
  ``controller_events`` sibling table (commit ``a786d6d``). Pins
  no-chain-mutation invariant (advisory writes never enter
  audit_events.event_hash preimage).

Added two paragraphs to cookbook §"Substrate-paper-spec'd
primitives" describing the test discipline + algorithmic surfaces
each pins. Renamed section header from
"(#000012 + #000018 + #000034)" to
"(#000012 + #000018 + #000034 + #000037)" to keep the
ticket-set roster current.

Two new rows in the test/code-density table:

  | substrate/prometheus.py       | 893 | 804 (36 tests) | 0.90 |
  | substrate/prometheus_audit.py | 200 | 388 (14 tests) | 1.94 |

All four counts AUTOCOUNT-tagged (2 inline prose + 2 table rows
= 4 new tags). prometheus_audit's ratio of 1.94 is high because
the test file pins a lot of write-path invariants for what is
nominally a small (200 LOC) sibling-table module — appropriate
for foundation-level audit-discipline code.

== Task 4: test_full_suite_total_fixture_count flake — investigated ==

Earlier today's transient ``-x`` flake (1 failed, 1799 passed)
did NOT reproduce in current tree state (2328 passed, 37 skipped
under same flags). Root-cause investigation:

- ``_DEFAULT_FIXTURES`` is read-only at module scope; no test
  mutates it.
- The test reads JSONL fixtures from ``bench/fixtures/`` via
  ``_run_one``; those files weren't being written by parallel
  tests.

Hypothesis (not confirmed, since flake didn't repro): transient
filesystem state during heavy-parallel-commit window
(``f625cac`` / ``a786d6d`` / ``6142437`` / ``f9f5ae4`` all
landed in succession around 17:23-17:24 EDT 2026-05-10 while my
test run was in flight). No structural defect identified. If
flake recurs, capture stdout + filesystem state at failure time
to confirm.

== Task 1: Walked fox's 6 evening commits via Explore agent ==

Agent reported what shipped under #000037 + #000012 evening
push (Prometheus-Σ Phases 0/1/2/dry-run + v8 consensus paper).
Findings used to drive task 2 above. Notable design choices
worth surfacing as reference:

- prometheus.py is **pure function** (no DB / LLM / scheduler);
  returns advisory ``ControllerDecision`` + optional proposal
  records, never mutations. Verifier-style test discipline.
- ``controller_events`` sibling table never enters
  ``audit_events.event_hash`` preimage — audit chain unaffected.
- Phase 3 sleep-sweep scheduler **deferred** in favor of
  read-only dry-run simulator surfacing 5 design constraints
  (chunk_size = Hermes concurrency NOT candidate pool;
  capital_cost flat=1.0 needs split; τ_qa per audit_mode;
  Target B = 4.4% canonical-shape match; quarantined-row veto
  exercises end-to-end). Calibration substrate for eventual
  scheduler.
- Per-branch controller latency 12.5 µs at chunk_size=4 → not
  the bottleneck; Hermes witness fan-out is.

v8 consensus paper at ``docs/_source/merkle-agi-v8-consensus.rst``
(834 lines; 11 parts) closes the loop from single-validator
Proof-of-Upgrade to multi-validator BFT selection. Phase 1c
(branch-set persistence) remains proposed-not-opened.

== Coverage ==

  Total tags after this commit:   54 (was 49; +5)
  Tags by metric:
    tests:           45 (+4 new prometheus + table rows)
    fixture-rows:     2
    db-rows:          3
    db-where:         2 (new metric, both 92 claim-pack)

  Files with tags:
    docs/warrant-substrate-cookbook.md             32 (+5)
    docs/calculator-test-patterns.md                8
    docs/soft-hash-channel-analysis.md              5
    docs/tickets/ticket-000006-bench-emergent...    4
    docs/seven-point-program.md                     3
    docs/tickets/ticket-000035-prg-choice-phi-prg.md 2

== Verification ==

  $ pytest tests/test_doc_counts.py -v
  3 passed in 2.60s

  $ pytest tests/ -q
  2328 passed, 37 skipped in 106.06s
This commit is contained in:
russell@unturf.com 2026-05-10 17:51:08 -04:00
parent ff1752c383
commit 6c6defbcb2
No known key found for this signature in database
2 changed files with 103 additions and 12 deletions
Download patch file Download diff file Expand all files Collapse all files

29
docs/warrant-substrate-cookbook.md
View file

@ -33,9 +33,10 @@ patterns that keep the chains audit-clean.
## Pillar-by-pillar substrate map
Eight pillars in the claim-pack source (`#000029`); 92 records
total. As of 2026-05-10, every record resolves to at least one
substrate in `~/.arborist/crawl/textbook_*.db`.
Eight pillars in the claim-pack source (`#000029`);
<!--AUTOCOUNT:db-where:documents?source_type=claim_pack-->92<!--/AUTOCOUNT-->
records total. As of 2026-05-10, every record resolves to at
least one substrate in `~/.arborist/crawl/textbook_*.db`.
### Pillar I — Logic (13 records)
@ -345,7 +346,7 @@ Full cascade order, tightest → loosest:
## Honest tier breakdown
The 92 chains have three quality tiers:
The <!--AUTOCOUNT:db-where:documents?source_type=claim_pack-->92<!--/AUTOCOUNT--> chains have three quality tiers:
- **Direct primary (~25 records)** — citation matches the literal
textbook ingested. Hilbert pillar IV (18), Goldstein → Newton
@ -503,7 +504,7 @@ than waiting for bench-time STRICT-rate drift to surface it.
stripped. Pinned so a future "simplify the cascade" PR can't
silently regress.
### Substrate-paper-spec'd primitives (#000012 + #000018 + #000034)
### Substrate-paper-spec'd primitives (#000012 + #000018 + #000034 + #000037)
- `tests/test_anchor_prg.py` — <!--AUTOCOUNT:tests:tests/test_anchor_prg.py-->27<!--/AUTOCOUNT--> tests for φ_PRG HMAC-SHA-512
expansion (#000035 Phase 1). Covers KAT regression, hand-formula
@ -541,6 +542,22 @@ than waiting for bench-time STRICT-rate drift to surface it.
- `tests/test_pi_star_protocol_and_registry.py` — <!--AUTOCOUNT:tests:tests/test_pi_star_protocol_and_registry.py-->21<!--/AUTOCOUNT--> tests for
`PiStar` Protocol contract + registry mutation discipline
(no public unregister; `name@version` content-pinned).
- `tests/test_prometheus.py` — <!--AUTOCOUNT:tests:tests/test_prometheus.py-->36<!--/AUTOCOUNT--> tests for the Prometheus-Σ
recursive-falsification controller (#000037 Phase 1, commits
`f625cac` + `f9f5ae4`). Verifier-style discipline (NOT
calculator pattern — it's a state machine, not closed-form
math): pins §16.2 named contracts (17), boundary veto-class
cases (5), §14 exception-matrix dispatch including row 4
Hermes-saturation guard, §13 step 11 falsification-fixture
proposal, §15 weight-tunable entropy + memory gates,
ESCALATE > QUARANTINE > REJECT priority ordering.
- `tests/test_prometheus_audit.py` — <!--AUTOCOUNT:tests:tests/test_prometheus_audit.py-->14<!--/AUTOCOUNT--> tests for the
`controller_events` sibling table (#000037 Phase 2, commit
`a786d6d`). Pins migration idempotency, no-chain-mutation
invariant (advisory writes never enter `audit_events.event_hash`
preimage), `(event_kind, body_hash)` UNIQUE dedupe contract,
query paths for the three event kinds (`controller_decision`,
`controller_difficulty`, `controller_budget_allocation`).
### Q&A / verifier scaffolding
@ -577,6 +594,8 @@ than waiting for bench-time STRICT-rate drift to surface it.
| qa/prompts.py | 153 | 207 (<!--AUTOCOUNT:tests:tests/test_qa_prompts.py-->20<!--/AUTOCOUNT--> tests) | 1.35 |
| concepts/extract.py | 288 | 278 (<!--AUTOCOUNT:tests:tests/test_concepts_extract.py-->20<!--/AUTOCOUNT--> tests) | 0.97 |
| textbooks_manifest.py | 236 | 463 (<!--AUTOCOUNT:tests:tests/test_textbooks_manifest.py-->43<!--/AUTOCOUNT--> tests) | 1.96 |
| substrate/prometheus.py | 893 | 804 (<!--AUTOCOUNT:tests:tests/test_prometheus.py-->36<!--/AUTOCOUNT--> tests) | 0.90 |
| substrate/prometheus_audit.py | 200 | 388 (<!--AUTOCOUNT:tests:tests/test_prometheus_audit.py-->14<!--/AUTOCOUNT--> tests) | 1.94 |
Pattern: small contract-defining modules (weights, prompts,
warrant_chain) get 1.4-3.6× test LOC because they're foundation;