tests/doc_counts: db-where metric + tag prometheus controller (#000037 Phase 1+2)
Fan-out follow-up: extends AUTOCOUNT with a new metric for filtered SQL-row claims, then tags fox's prometheus controller test surfaces shipped this evening under #000037. == Task 3: db-where metric == New metric ``db-where`` for tagging single-column equality predicates. Target syntax:: <table>?<column>=<value> <shard>:<table>?<column>=<value> Resolves to ``SELECT COUNT(*) FROM <table> WHERE <column> = ?`` with ``<value>`` bound as a SQL parameter (no string interpolation), so author typos or stray content can't escape the predicate. Column + table names validated as bare identifiers before string-interpolating into the query template; sqlite3 connection opens with ``mode=ro`` URI flag. Same skip-on-absence semantics as ``db-rows``: missing DB or table yields a logged skip note, not a test failure. Sentinel returns reuse the same _DB_MISSING / _TABLE_MISSING / _DB_ERROR constants. Smoke verified:: _live_db_where('documents?source_type=claim_pack') → 92 _live_db_where('documents?source_type=wikipedia_xml') → 866782 _live_db_where('001.db:documents?source_type=wikipedia_xml') → 867695 _live_db_where('documents') # malformed (no ?) → -4 Tagged claims using the new metric (cookbook): - L36 ``92 records total`` for the claim-pack source - L349 ``The 92 chains have three quality tiers`` Both resolve to ``documents WHERE source_type='claim_pack'`` in shard ``000.db`` — the live count of claim-pack records. L5 ``18/92 → 92/92`` historical narrative left untagged (expressing a journey arc, not current state). == Task 2: tag prometheus controller test surfaces == fox shipped two test files this evening under #000037 that weren't previously inventoried in any reference doc: - ``tests/test_prometheus.py`` — 36 tests covering Phase 1 controller (commits ``f625cac`` + ``f9f5ae4``). Verifier-style discipline (NOT calculator pattern — it's a pure-function state-machine controller with no closed-form math). - ``tests/test_prometheus_audit.py`` — 14 tests covering Phase 2 ``controller_events`` sibling table (commit ``a786d6d``). Pins no-chain-mutation invariant (advisory writes never enter audit_events.event_hash preimage). Added two paragraphs to cookbook §"Substrate-paper-spec'd primitives" describing the test discipline + algorithmic surfaces each pins. Renamed section header from "(#000012 + #000018 + #000034)" to "(#000012 + #000018 + #000034 + #000037)" to keep the ticket-set roster current. Two new rows in the test/code-density table: | substrate/prometheus.py | 893 | 804 (36 tests) | 0.90 | | substrate/prometheus_audit.py | 200 | 388 (14 tests) | 1.94 | All four counts AUTOCOUNT-tagged (2 inline prose + 2 table rows = 4 new tags). prometheus_audit's ratio of 1.94 is high because the test file pins a lot of write-path invariants for what is nominally a small (200 LOC) sibling-table module — appropriate for foundation-level audit-discipline code. == Task 4: test_full_suite_total_fixture_count flake — investigated == Earlier today's transient ``-x`` flake (1 failed, 1799 passed) did NOT reproduce in current tree state (2328 passed, 37 skipped under same flags). Root-cause investigation: - ``_DEFAULT_FIXTURES`` is read-only at module scope; no test mutates it. - The test reads JSONL fixtures from ``bench/fixtures/`` via ``_run_one``; those files weren't being written by parallel tests. Hypothesis (not confirmed, since flake didn't repro): transient filesystem state during heavy-parallel-commit window (``f625cac`` / ``a786d6d`` / ``6142437`` / ``f9f5ae4`` all landed in succession around 17:23-17:24 EDT 2026-05-10 while my test run was in flight). No structural defect identified. If flake recurs, capture stdout + filesystem state at failure time to confirm. == Task 1: Walked fox's 6 evening commits via Explore agent == Agent reported what shipped under #000037 + #000012 evening push (Prometheus-Σ Phases 0/1/2/dry-run + v8 consensus paper). Findings used to drive task 2 above. Notable design choices worth surfacing as reference: - prometheus.py is **pure function** (no DB / LLM / scheduler); returns advisory ``ControllerDecision`` + optional proposal records, never mutations. Verifier-style test discipline. - ``controller_events`` sibling table never enters ``audit_events.event_hash`` preimage — audit chain unaffected. - Phase 3 sleep-sweep scheduler **deferred** in favor of read-only dry-run simulator surfacing 5 design constraints (chunk_size = Hermes concurrency NOT candidate pool; capital_cost flat=1.0 needs split; τ_qa per audit_mode; Target B = 4.4% canonical-shape match; quarantined-row veto exercises end-to-end). Calibration substrate for eventual scheduler. - Per-branch controller latency 12.5 µs at chunk_size=4 → not the bottleneck; Hermes witness fan-out is. v8 consensus paper at ``docs/_source/merkle-agi-v8-consensus.rst`` (834 lines; 11 parts) closes the loop from single-validator Proof-of-Upgrade to multi-validator BFT selection. Phase 1c (branch-set persistence) remains proposed-not-opened. == Coverage == Total tags after this commit: 54 (was 49; +5) Tags by metric: tests: 45 (+4 new prometheus + table rows) fixture-rows: 2 db-rows: 3 db-where: 2 (new metric, both 92 claim-pack) Files with tags: docs/warrant-substrate-cookbook.md 32 (+5) docs/calculator-test-patterns.md 8 docs/soft-hash-channel-analysis.md 5 docs/tickets/ticket-000006-bench-emergent... 4 docs/seven-point-program.md 3 docs/tickets/ticket-000035-prg-choice-phi-prg.md 2 == Verification == $ pytest tests/test_doc_counts.py -v 3 passed in 2.60s $ pytest tests/ -q 2328 passed, 37 skipped in 106.06s
This commit is contained in:
parent
ff1752c383
commit
6c6defbcb2
2 changed files with 103 additions and 12 deletions
|
|
@ -33,9 +33,10 @@ patterns that keep the chains audit-clean.
|
|||
|
||||
## Pillar-by-pillar substrate map
|
||||
|
||||
Eight pillars in the claim-pack source (`#000029`); 92 records
|
||||
total. As of 2026-05-10, every record resolves to at least one
|
||||
substrate in `~/.arborist/crawl/textbook_*.db`.
|
||||
Eight pillars in the claim-pack source (`#000029`);
|
||||
<!--AUTOCOUNT:db-where:documents?source_type=claim_pack-->92<!--/AUTOCOUNT-->
|
||||
records total. As of 2026-05-10, every record resolves to at
|
||||
least one substrate in `~/.arborist/crawl/textbook_*.db`.
|
||||
|
||||
### Pillar I — Logic (13 records)
|
||||
|
||||
|
|
@ -345,7 +346,7 @@ Full cascade order, tightest → loosest:
|
|||
|
||||
## Honest tier breakdown
|
||||
|
||||
The 92 chains have three quality tiers:
|
||||
The <!--AUTOCOUNT:db-where:documents?source_type=claim_pack-->92<!--/AUTOCOUNT--> chains have three quality tiers:
|
||||
|
||||
- **Direct primary (~25 records)** — citation matches the literal
|
||||
textbook ingested. Hilbert pillar IV (18), Goldstein → Newton
|
||||
|
|
@ -503,7 +504,7 @@ than waiting for bench-time STRICT-rate drift to surface it.
|
|||
stripped. Pinned so a future "simplify the cascade" PR can't
|
||||
silently regress.
|
||||
|
||||
### Substrate-paper-spec'd primitives (#000012 + #000018 + #000034)
|
||||
### Substrate-paper-spec'd primitives (#000012 + #000018 + #000034 + #000037)
|
||||
|
||||
- `tests/test_anchor_prg.py` — <!--AUTOCOUNT:tests:tests/test_anchor_prg.py-->27<!--/AUTOCOUNT--> tests for φ_PRG HMAC-SHA-512
|
||||
expansion (#000035 Phase 1). Covers KAT regression, hand-formula
|
||||
|
|
@ -541,6 +542,22 @@ than waiting for bench-time STRICT-rate drift to surface it.
|
|||
- `tests/test_pi_star_protocol_and_registry.py` — <!--AUTOCOUNT:tests:tests/test_pi_star_protocol_and_registry.py-->21<!--/AUTOCOUNT--> tests for
|
||||
`PiStar` Protocol contract + registry mutation discipline
|
||||
(no public unregister; `name@version` content-pinned).
|
||||
- `tests/test_prometheus.py` — <!--AUTOCOUNT:tests:tests/test_prometheus.py-->36<!--/AUTOCOUNT--> tests for the Prometheus-Σ
|
||||
recursive-falsification controller (#000037 Phase 1, commits
|
||||
`f625cac` + `f9f5ae4`). Verifier-style discipline (NOT
|
||||
calculator pattern — it's a state machine, not closed-form
|
||||
math): pins §16.2 named contracts (17), boundary veto-class
|
||||
cases (5), §14 exception-matrix dispatch including row 4
|
||||
Hermes-saturation guard, §13 step 11 falsification-fixture
|
||||
proposal, §15 weight-tunable entropy + memory gates,
|
||||
ESCALATE > QUARANTINE > REJECT priority ordering.
|
||||
- `tests/test_prometheus_audit.py` — <!--AUTOCOUNT:tests:tests/test_prometheus_audit.py-->14<!--/AUTOCOUNT--> tests for the
|
||||
`controller_events` sibling table (#000037 Phase 2, commit
|
||||
`a786d6d`). Pins migration idempotency, no-chain-mutation
|
||||
invariant (advisory writes never enter `audit_events.event_hash`
|
||||
preimage), `(event_kind, body_hash)` UNIQUE dedupe contract,
|
||||
query paths for the three event kinds (`controller_decision`,
|
||||
`controller_difficulty`, `controller_budget_allocation`).
|
||||
|
||||
### Q&A / verifier scaffolding
|
||||
|
||||
|
|
@ -577,6 +594,8 @@ than waiting for bench-time STRICT-rate drift to surface it.
|
|||
| qa/prompts.py | 153 | 207 (<!--AUTOCOUNT:tests:tests/test_qa_prompts.py-->20<!--/AUTOCOUNT--> tests) | 1.35 |
|
||||
| concepts/extract.py | 288 | 278 (<!--AUTOCOUNT:tests:tests/test_concepts_extract.py-->20<!--/AUTOCOUNT--> tests) | 0.97 |
|
||||
| textbooks_manifest.py | 236 | 463 (<!--AUTOCOUNT:tests:tests/test_textbooks_manifest.py-->43<!--/AUTOCOUNT--> tests) | 1.96 |
|
||||
| substrate/prometheus.py | 893 | 804 (<!--AUTOCOUNT:tests:tests/test_prometheus.py-->36<!--/AUTOCOUNT--> tests) | 0.90 |
|
||||
| substrate/prometheus_audit.py | 200 | 388 (<!--AUTOCOUNT:tests:tests/test_prometheus_audit.py-->14<!--/AUTOCOUNT--> tests) | 1.94 |
|
||||
|
||||
Pattern: small contract-defining modules (weights, prompts,
|
||||
warrant_chain) get 1.4-3.6× test LOC because they're foundation;
|
||||
|
|
|
|||
|
|
@ -27,6 +27,17 @@ Where ``metric`` is one of:
|
|||
When the DB or table isn't present (CI or fresh checkout), the
|
||||
claim is skipped with a logged note rather than failing — claim-
|
||||
pack and alias counts are operator state, not source-state.
|
||||
- ``db-where`` — single-column equality predicate. ``path`` syntax::
|
||||
|
||||
<table>?<column>=<value>
|
||||
<shard>:<table>?<column>=<value>
|
||||
|
||||
Resolves to ``SELECT COUNT(*) FROM <table> WHERE <column> = ?``
|
||||
with ``<value>`` bound as a parameter (no SQL injection through
|
||||
the value). Same skip-on-absence semantics as ``db-rows``. Use
|
||||
for filtered-row claims like ``92 claim-pack records``::
|
||||
|
||||
<!--AUTOCOUNT:db-where:documents?source_type=claim_pack-->92<!--/AUTOCOUNT-->
|
||||
|
||||
GitHub and most markdown renderers strip HTML comments, so readers
|
||||
see only ``N``. The tags are invisible in rendered output but
|
||||
|
|
@ -157,6 +168,63 @@ def _live_db_rows(target: str) -> int:
|
|||
return _DB_ERROR
|
||||
|
||||
|
||||
def _live_db_where(target: str) -> int:
|
||||
"""Count rows matching a single-column equality predicate.
|
||||
|
||||
Target syntax::
|
||||
|
||||
<table>?<column>=<value>
|
||||
<shard>:<table>?<column>=<value>
|
||||
|
||||
Same sentinel returns as ``_live_db_rows``. ``<value>`` is bound
|
||||
as a SQL parameter (no string interpolation), so even author-
|
||||
typo'd or malicious values can't escape the predicate. Column
|
||||
name is validated as a bare identifier (it goes into the SQL
|
||||
text).
|
||||
"""
|
||||
import sqlite3
|
||||
|
||||
if "?" not in target:
|
||||
return _DB_ERROR
|
||||
head, where = target.split("?", 1)
|
||||
if "=" not in where:
|
||||
return _DB_ERROR
|
||||
column, value = where.split("=", 1)
|
||||
if not re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]*", column):
|
||||
return _DB_ERROR
|
||||
|
||||
if ":" in head:
|
||||
shard_name, table = head.split(":", 1)
|
||||
db = _DEFAULT_SHARDS_DIR / shard_name
|
||||
else:
|
||||
db = _DEFAULT_SHARD
|
||||
table = head
|
||||
if not re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]*", table):
|
||||
return _DB_ERROR
|
||||
if not db.exists():
|
||||
return _DB_MISSING
|
||||
try:
|
||||
c = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
|
||||
try:
|
||||
row = c.execute(
|
||||
# noqa: S608 — table + column validated above as bare
|
||||
# identifiers; value bound as parameter.
|
||||
f"SELECT COUNT(*) FROM {table} WHERE {column} = ?",
|
||||
(value,),
|
||||
).fetchone()
|
||||
return int(row[0]) if row else 0
|
||||
finally:
|
||||
c.close()
|
||||
except sqlite3.OperationalError as exc:
|
||||
if "no such table" in str(exc).lower():
|
||||
return _TABLE_MISSING
|
||||
if "no such column" in str(exc).lower():
|
||||
return _DB_ERROR
|
||||
return _DB_ERROR
|
||||
except sqlite3.Error:
|
||||
return _DB_ERROR
|
||||
|
||||
|
||||
def _iter_claims() -> Iterator[tuple[Path, int, str, str, int]]:
|
||||
"""Yield (doc_path, lineno, metric, target_path, claimed_n)."""
|
||||
for md in sorted(DOCS_DIR.rglob("*.md")):
|
||||
|
|
@ -200,24 +268,28 @@ def test_doc_autocount_claims_match_live(capsys: pytest.CaptureFixture[str]) ->
|
|||
f"missing or uncollectable"
|
||||
)
|
||||
continue
|
||||
elif metric == "db-rows":
|
||||
live = _live_db_rows(target)
|
||||
elif metric in ("db-rows", "db-where"):
|
||||
live = (
|
||||
_live_db_rows(target)
|
||||
if metric == "db-rows"
|
||||
else _live_db_where(target)
|
||||
)
|
||||
if live == _DB_MISSING:
|
||||
skipped.append(
|
||||
f"{rel}:{lineno} db-rows:{target} skipped — "
|
||||
f"{rel}:{lineno} {metric}:{target} skipped — "
|
||||
f"{_DEFAULT_SHARDS_DIR} not present (CI / fresh checkout)"
|
||||
)
|
||||
continue
|
||||
if live == _TABLE_MISSING:
|
||||
skipped.append(
|
||||
f"{rel}:{lineno} db-rows:{target} skipped — table not "
|
||||
f"{rel}:{lineno} {metric}:{target} skipped — table not "
|
||||
f"present in shard"
|
||||
)
|
||||
continue
|
||||
if live == _DB_ERROR:
|
||||
skipped.append(
|
||||
f"{rel}:{lineno} db-rows:{target} skipped — sqlite "
|
||||
f"error or invalid table name"
|
||||
f"{rel}:{lineno} {metric}:{target} skipped — sqlite "
|
||||
f"error, malformed target, or invalid identifier"
|
||||
)
|
||||
continue
|
||||
else:
|
||||
|
|
@ -265,7 +337,7 @@ def test_autocount_tags_are_well_formed() -> None:
|
|||
|
||||
def test_autocount_metric_names_are_documented() -> None:
|
||||
"""Fail-closed if a doc uses an undocumented metric."""
|
||||
known = {"tests", "fixture-rows", "db-rows"}
|
||||
known = {"tests", "fixture-rows", "db-rows", "db-where"}
|
||||
seen: set[str] = set()
|
||||
for _, _, metric, _, _ in _iter_claims():
|
||||
seen.add(metric)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue