ticket #000044: AUTOCOUNT doc-drift discipline + code-fence parser fix

Lock the AUTOCOUNT regression-test pattern as the design log
canonical record. Previously declined when surface was 1-metric
+ 29 tags; now mature enough (4 metrics + 58 tags + 1 same-day
drift-catch since landing) to formalize.

== Ticket content ==

10 sections covering:

1. Why this exists — the 4-drift-day baseline (6cbbf95 / 14bcb99 /
   5c21e83 / 30a9488) that motivated mechanization. Five-step
   walk through justifying each choice (Step 5 last).
2. Format — `<!--AUTOCOUNT:metric:path-->N<!--/AUTOCOUNT-->`.
3. Four supported metrics with examples + skip semantics:
   `tests`, `fixture-rows`, `db-rows`, `db-where`.
4. Skip-on-absence — operator state (shards, qa.db) absence is a
   logged skip, not a fail. Smoke verified 2026-05-10 with
   HOME=/tmp/empty.
5. What NOT to tag — closed-ticket point-in-time snapshots,
   aggregate floors ("2000+"), historical journey arcs.
6. Install discipline at write time + at refresh time.
7. Future metrics deferred (file-lines, gh-pr-comments-count,
   module-loc, commit-hash-exists) with the "add a metric"
   recipe.
8. Empirical baseline at landing (3 test functions, 58 active
   tagged claims across 8 doc files, harness runtime 2-4s).
9. Scope boundaries — does NOT auto-rewrite, does NOT validate
   prose quality, does NOT scan docstrings, does NOT lock
   values, does NOT add deps.
10. References — every landing commit + sister doc.

Closed at landing (status quo since fc5ba50 2026-05-10 morning;
this ticket is retroactive design log per the convention "every
ticket flips to `closed · landed in commit <sha>` when the work
ships").

== Code-fence parser fix ==

Adding the ticket itself surfaced an oversight: my AUTOCOUNT
examples in §3.3 + §3.4 used literal tag pairs in ``` fenced
code blocks. The parser was reading them as live claims and
firing on the illustrative `db-rows:002.db:concept_relations`
claim (compared 1234 vs live 72576 — both meaningless because
it's an example).

Fix: `_strip_fenced_code_blocks` substitutes the body of every
triple-backtick block with newlines before regex scanning. Line
numbers stay aligned (newline-preserving substitution); tags
inside fences are skipped because their parent text no longer
matches the regex.

Both helper functions (`_iter_claims` and the well-formed-tags
test) walk through the stripped text, so the strip discipline
is consistent across all three test functions.

== TICKETS.md index ==

Added #000044 row marked closed with the 5-commit landing trail.
Bumped Next ID 000044 → 000045.

== Verification ==

  $ pytest tests/test_doc_counts.py
  3 passed in 2.80s

  $ pytest tests/ -q
  2337 passed, 37 skipped in 108.29s

Hygiene: fox's in-flight changes to arborist/qa/runner.py +
arborist/substrate/prometheus.py + tests/test_prometheus*.py
left untouched in working tree.
This commit is contained in:
russell@unturf.com 2026-05-10 18:24:50 -04:00
parent 3b301267ff
commit 8da29fde69
No known key found for this signature in database
3 changed files with 334 additions and 4 deletions

View file

@ -225,10 +225,42 @@ def _live_db_where(target: str) -> int:
return _DB_ERROR
def _strip_fenced_code_blocks(text: str) -> str:
"""Replace contents of triple-backtick fenced code blocks with newlines.
Tags inside fenced blocks are documentation examples, not live claims —
line numbers must stay aligned (so failure messages point at the right
line) and the rest of the file must be untouched. Substituting newlines
for the block bodies keeps line offsets exact and removes the example
tag pairs from the regex scan.
Only handles the ```` ``` ```` form. Inline backticks (``code``) don't
get fenced semantics anyway — they're single-line and the regex
already requires the matching close tag on the same string.
"""
out: list[str] = []
in_fence = False
for line in text.splitlines(keepends=True):
stripped = line.lstrip()
if stripped.startswith("```"):
in_fence = not in_fence
out.append("\n")
continue
if in_fence:
out.append("\n")
else:
out.append(line)
return "".join(out)
def _iter_claims() -> Iterator[tuple[Path, int, str, str, int]]:
"""Yield (doc_path, lineno, metric, target_path, claimed_n)."""
"""Yield (doc_path, lineno, metric, target_path, claimed_n).
Tags inside ``` fenced code blocks are skipped — those are
documentation examples, not live claims.
"""
for md in sorted(DOCS_DIR.rglob("*.md")):
text = md.read_text()
text = _strip_fenced_code_blocks(md.read_text())
for m in AUTOCOUNT_RE.finditer(text):
lineno = text.count("\n", 0, m.start()) + 1
yield (
@ -316,7 +348,7 @@ def test_autocount_tags_are_well_formed() -> None:
close_re = re.compile(r"<!--\s*/AUTOCOUNT\s*-->")
issues: list[str] = []
for md in sorted(DOCS_DIR.rglob("*.md")):
text = md.read_text()
text = _strip_fenced_code_blocks(md.read_text())
opens = len(open_re.findall(text))
closes = len(close_re.findall(text))
full = len(AUTOCOUNT_RE.findall(text))