docs/cookbook: appendix — test-coverage cross-reference (2026-05-10)

Adds a per-discipline test-file index to the cookbook so audit
reviewers can click through from a substrate discipline to the
unit-test pinning that prevents silent regression. Eight
sub-sections matching the live discipline groups:

  - citation/term alias mechanism (#000041 + #000042) —
    test_aliases.py 28 tests
  - warrant-resolver chain (#000031 P1+2+3) —
    test_warrant_resolver.py + test_warrant_chain.py
  - textbook ingest license-discipline gate —
    test_textbooks_manifest.py 43 tests
  - cascade tuning (#000040 + parenthetical/OR-fallback) —
    _phrase_from_parenthetical regression guard
  - substrate-paper-spec'd primitives (#000012 + #000018 +
    #000034 + #000036) — 6 test files
  - Q&A / verifier scaffolding — test_qa_progress.py +
    test_qa_prompts.py
  - concept-relations write-side — test_concepts_extract.py

Plus a unit-test-density-vs-code table: small contract-defining
modules (weights, prompts, warrant_chain) sit at 1.4-3.6× test
LOC because they're foundation; larger modules (warrant_resolver,
aliases) settle at 0.5-1.0× because they're more code-with-tests-
per-feature than contract-with-tests-per-rule.

Code-review heuristic surfaced for future shifts: new
substrate-paper-spec'd primitive without ≥1× test/code ratio
is suspect on landing.
This commit is contained in:
russell@unturf.com 2026-05-10 13:11:08 -04:00
parent 35aeebbb24
commit 6aca7d91d7
No known key found for this signature in database

View file

@ -434,3 +434,125 @@ arborist warrant-resolve --use-aliases --write
`_phrase_from_parenthetical` + `resolve_chunks`
- `arborist/qa/aliases.py` — citation + term alias schema +
CLI add/list/remove
## Appendix — test-coverage cross-reference (2026-05-10)
Per-discipline regression-guard test files. Each test file pins
contract phrases / numerical invariants so a future PR that
silently drops a discipline rule fires a unit-test failure rather
than waiting for bench-time STRICT-rate drift to surface it.
### Citation-alias / term-alias mechanism (#000041 + #000042)
- `tests/test_aliases.py` — 28 tests covering
`add_citation_alias` / `add_term_alias` audit-fail-closed
(empty `decision_by` raises ValueError), domain isolation
(same term different domain stays distinct), lowercase
normalization (Term/Term/Geometry → all lowercased on
insert), bidirectional `lookup_term_aliases`,
`expand_query_with_term_aliases` quoted-phrase preservation,
fail-closed on unreachable DB, parenthesis-as-token tokenizer
contract (OR-expansion round-trip), and `list_term_aliases`
filter-by-domain / term-substring.
### Warrant-resolver chain (#000031 Phase 1+2+3)
- `tests/test_warrant_resolver.py` — Citation parsing (3
patterns: "Title by Author", multi-author Oxford comma,
semicolon-separated multi-cite); cascade builder (5-query
variants: title-phrase, parenthetical-phrase, AND-top-5,
OR-top-3, legacy); `via_citation_alias` floor propagation
through ResolutionMatch (B-1 attribution fix).
- `tests/test_warrant_chain.py` — 9 tests covering
`warrant_chain_lookup` (process_id LIKE filter, missing-table
fail-closed, +alias variant matching), `has_warrant_chain`
empty-set short-circuit, verifier suppression of
WARRANT_MISSING when chain present, verdict-field
`warrant_proven_claim_idxs` always present (schema guarantee).
### Textbook ingest license-discipline gate
- `tests/test_textbooks_manifest.py` — 43 tests covering every
license token in `_ALLOWED_LICENSES` (parametrized 12-token
sweep), placeholder rows allowed without URLs, disallowed
license + emit URLs raises, CLI dispatch, return-code
contract (0 ok / 2 missing-arg / 3 license-fail / 4
unknown-id), and the live manifest itself validates clean.
### Cascade tuning (#000040 + parenthetical/OR-fallback)
- `tests/test_warrant_resolver.py::_phrase_from_parenthetical`
— extracts `"non negativity"` from
"Kolmogorov's First Axiom (Non-Negativity)"; the discriminator
is INSIDE the parenthetical that earlier resolver versions
stripped. Pinned so a future "simplify the cascade" PR can't
silently regress.
### Substrate-paper-spec'd primitives (#000012 + #000018 + #000034)
- `tests/test_anchor_prg.py` — 27 tests for φ_PRG HMAC-SHA-512
expansion (#000035 Phase 1).
- `tests/test_phi_alignment_probe.py` — 14 tests for φ_linear
Hessian-alignment probe (#000034 Phase 1a).
- `tests/test_t3_bound_calculator.py` — 51 tests for the
T3 per-window covert-channel bound calculator (#000036 §11);
pins the closed-form B1/B2/B3 formulas, monotonicity in each
input, recommendation-text mode transitions, and the §11
worked-example bit-for-bit (with doc-calibration update
surfaced through the test).
- `tests/test_fork_score.py` — 18 tests for v8 ForkScore
(#000012 Phase 1a); pins SIGNAL_FLOOR (5pp) + HARD_REGRESSION_FLOOR
(5pp), score = sum-of-breakdown closure, security_risk inert
under default iota=0 (opt-in), NEG_INF_REGRESSION hard-reject.
- `tests/test_weights.py` — 16 tests for WeightSet defaults
(each weight value pinned to its docstring rationale; PR that
flips alpha=1.0→0.5 fires this test), greek-letter and
Python-safe key aliases, frozen-dataclass invariant.
- `tests/test_pi_star_protocol_and_registry.py` — 21 tests for
`PiStar` Protocol contract + registry mutation discipline
(no public unregister; `name@version` content-pinned).
### Q&A / verifier scaffolding
- `tests/test_qa_progress.py` — 31 tests for the Progress
emitter (env / TTY / cli-override precedence; truthy/falsy
spelling matrix; fail-closed on missing `.isatty`).
- `tests/test_qa_prompts.py` — 20 tests pinning load-bearing
system prompts (worked-example presence, two-pointer cap,
pointer-mode no-quote-instruction discipline,
JSON-mode first-char-`{` / last-char-`}` rule); a silent
prompt edit that drops a critical instruction would otherwise
surface only as a STRICT-rate drop in bench.
### Concept-relations write-side
- `tests/test_concepts_extract.py` — 20 tests for synonym /
IDF / FTS5-titles extractors (`_title_tokens` stopword
+ length-floor + dedupe; `EXTRACTORS` registry contract;
`link_reciprocity_synonym` idempotency + self-overlap
exclusion; `backfill_documents_fts` NULL-title skip).
### Unit-test density vs production-code size
| domain | LOC tested | test LOC | test/code ratio |
|---|---|---|---|
| aliases.py | 512 | 469 (28 tests) | 0.92 |
| warrant_resolver.py | ~800 | ~430 (combined) | 0.54 |
| warrant_chain.py | 89 | 320 (9 tests) | 3.6 |
| t3_bound_calculator.py | 249 | 351 (51 tests) | 1.4 |
| fork_score.py | 298 | 403 (18 tests) | 1.35 |
| weights.py | 73 | 180 (16 tests) | 2.5 |
| pi_star/protocol+registry | 124 | 280 (21 tests) | 2.3 |
| qa/progress.py | 85 | 226 (31 tests) | 2.7 |
| qa/prompts.py | 153 | 207 (20 tests) | 1.35 |
| concepts/extract.py | 288 | 278 (20 tests) | 0.97 |
| textbooks_manifest.py | 236 | 463 (43 tests) | 1.96 |
Pattern: small contract-defining modules (weights, prompts,
warrant_chain) get 1.4-3.6× test LOC because they're foundation;
larger modules (warrant_resolver, aliases) settle around
0.5-1.0× because they're more code-with-tests-per-feature than
contract-with-tests-per-rule.
The ratio is a code-review heuristic: a new substrate-paper-spec'd
primitive without ≥1× test/code ratio is suspect on landing.