docs/cookbook: appendix — test-coverage cross-reference (2026-05-10)
Adds a per-discipline test-file index to the cookbook so audit reviewers can click through from a substrate discipline to the unit-test pinning that prevents silent regression. Eight sub-sections matching the live discipline groups: - citation/term alias mechanism (#000041 + #000042) — test_aliases.py 28 tests - warrant-resolver chain (#000031 P1+2+3) — test_warrant_resolver.py + test_warrant_chain.py - textbook ingest license-discipline gate — test_textbooks_manifest.py 43 tests - cascade tuning (#000040 + parenthetical/OR-fallback) — _phrase_from_parenthetical regression guard - substrate-paper-spec'd primitives (#000012 + #000018 + #000034 + #000036) — 6 test files - Q&A / verifier scaffolding — test_qa_progress.py + test_qa_prompts.py - concept-relations write-side — test_concepts_extract.py Plus a unit-test-density-vs-code table: small contract-defining modules (weights, prompts, warrant_chain) sit at 1.4-3.6× test LOC because they're foundation; larger modules (warrant_resolver, aliases) settle at 0.5-1.0× because they're more code-with-tests- per-feature than contract-with-tests-per-rule. Code-review heuristic surfaced for future shifts: new substrate-paper-spec'd primitive without ≥1× test/code ratio is suspect on landing.
This commit is contained in:
parent
35aeebbb24
commit
6aca7d91d7
1 changed files with 122 additions and 0 deletions
|
|
@ -434,3 +434,125 @@ arborist warrant-resolve --use-aliases --write
|
|||
`_phrase_from_parenthetical` + `resolve_chunks`
|
||||
- `arborist/qa/aliases.py` — citation + term alias schema +
|
||||
CLI add/list/remove
|
||||
|
||||
## Appendix — test-coverage cross-reference (2026-05-10)
|
||||
|
||||
Per-discipline regression-guard test files. Each test file pins
|
||||
contract phrases / numerical invariants so a future PR that
|
||||
silently drops a discipline rule fires a unit-test failure rather
|
||||
than waiting for bench-time STRICT-rate drift to surface it.
|
||||
|
||||
### Citation-alias / term-alias mechanism (#000041 + #000042)
|
||||
|
||||
- `tests/test_aliases.py` — 28 tests covering
|
||||
`add_citation_alias` / `add_term_alias` audit-fail-closed
|
||||
(empty `decision_by` raises ValueError), domain isolation
|
||||
(same term different domain stays distinct), lowercase
|
||||
normalization (Term/Term/Geometry → all lowercased on
|
||||
insert), bidirectional `lookup_term_aliases`,
|
||||
`expand_query_with_term_aliases` quoted-phrase preservation,
|
||||
fail-closed on unreachable DB, parenthesis-as-token tokenizer
|
||||
contract (OR-expansion round-trip), and `list_term_aliases`
|
||||
filter-by-domain / term-substring.
|
||||
|
||||
### Warrant-resolver chain (#000031 Phase 1+2+3)
|
||||
|
||||
- `tests/test_warrant_resolver.py` — Citation parsing (3
|
||||
patterns: "Title by Author", multi-author Oxford comma,
|
||||
semicolon-separated multi-cite); cascade builder (5-query
|
||||
variants: title-phrase, parenthetical-phrase, AND-top-5,
|
||||
OR-top-3, legacy); `via_citation_alias` floor propagation
|
||||
through ResolutionMatch (B-1 attribution fix).
|
||||
- `tests/test_warrant_chain.py` — 9 tests covering
|
||||
`warrant_chain_lookup` (process_id LIKE filter, missing-table
|
||||
fail-closed, +alias variant matching), `has_warrant_chain`
|
||||
empty-set short-circuit, verifier suppression of
|
||||
WARRANT_MISSING when chain present, verdict-field
|
||||
`warrant_proven_claim_idxs` always present (schema guarantee).
|
||||
|
||||
### Textbook ingest license-discipline gate
|
||||
|
||||
- `tests/test_textbooks_manifest.py` — 43 tests covering every
|
||||
license token in `_ALLOWED_LICENSES` (parametrized 12-token
|
||||
sweep), placeholder rows allowed without URLs, disallowed
|
||||
license + emit URLs raises, CLI dispatch, return-code
|
||||
contract (0 ok / 2 missing-arg / 3 license-fail / 4
|
||||
unknown-id), and the live manifest itself validates clean.
|
||||
|
||||
### Cascade tuning (#000040 + parenthetical/OR-fallback)
|
||||
|
||||
- `tests/test_warrant_resolver.py::_phrase_from_parenthetical`
|
||||
— extracts `"non negativity"` from
|
||||
"Kolmogorov's First Axiom (Non-Negativity)"; the discriminator
|
||||
is INSIDE the parenthetical that earlier resolver versions
|
||||
stripped. Pinned so a future "simplify the cascade" PR can't
|
||||
silently regress.
|
||||
|
||||
### Substrate-paper-spec'd primitives (#000012 + #000018 + #000034)
|
||||
|
||||
- `tests/test_anchor_prg.py` — 27 tests for φ_PRG HMAC-SHA-512
|
||||
expansion (#000035 Phase 1).
|
||||
- `tests/test_phi_alignment_probe.py` — 14 tests for φ_linear
|
||||
Hessian-alignment probe (#000034 Phase 1a).
|
||||
- `tests/test_t3_bound_calculator.py` — 51 tests for the
|
||||
T3 per-window covert-channel bound calculator (#000036 §11);
|
||||
pins the closed-form B1/B2/B3 formulas, monotonicity in each
|
||||
input, recommendation-text mode transitions, and the §11
|
||||
worked-example bit-for-bit (with doc-calibration update
|
||||
surfaced through the test).
|
||||
- `tests/test_fork_score.py` — 18 tests for v8 ForkScore
|
||||
(#000012 Phase 1a); pins SIGNAL_FLOOR (5pp) + HARD_REGRESSION_FLOOR
|
||||
(5pp), score = sum-of-breakdown closure, security_risk inert
|
||||
under default iota=0 (opt-in), NEG_INF_REGRESSION hard-reject.
|
||||
- `tests/test_weights.py` — 16 tests for WeightSet defaults
|
||||
(each weight value pinned to its docstring rationale; PR that
|
||||
flips alpha=1.0→0.5 fires this test), greek-letter and
|
||||
Python-safe key aliases, frozen-dataclass invariant.
|
||||
- `tests/test_pi_star_protocol_and_registry.py` — 21 tests for
|
||||
`PiStar` Protocol contract + registry mutation discipline
|
||||
(no public unregister; `name@version` content-pinned).
|
||||
|
||||
### Q&A / verifier scaffolding
|
||||
|
||||
- `tests/test_qa_progress.py` — 31 tests for the Progress
|
||||
emitter (env / TTY / cli-override precedence; truthy/falsy
|
||||
spelling matrix; fail-closed on missing `.isatty`).
|
||||
- `tests/test_qa_prompts.py` — 20 tests pinning load-bearing
|
||||
system prompts (worked-example presence, two-pointer cap,
|
||||
pointer-mode no-quote-instruction discipline,
|
||||
JSON-mode first-char-`{` / last-char-`}` rule); a silent
|
||||
prompt edit that drops a critical instruction would otherwise
|
||||
surface only as a STRICT-rate drop in bench.
|
||||
|
||||
### Concept-relations write-side
|
||||
|
||||
- `tests/test_concepts_extract.py` — 20 tests for synonym /
|
||||
IDF / FTS5-titles extractors (`_title_tokens` stopword
|
||||
+ length-floor + dedupe; `EXTRACTORS` registry contract;
|
||||
`link_reciprocity_synonym` idempotency + self-overlap
|
||||
exclusion; `backfill_documents_fts` NULL-title skip).
|
||||
|
||||
### Unit-test density vs production-code size
|
||||
|
||||
| domain | LOC tested | test LOC | test/code ratio |
|
||||
|---|---|---|---|
|
||||
| aliases.py | 512 | 469 (28 tests) | 0.92 |
|
||||
| warrant_resolver.py | ~800 | ~430 (combined) | 0.54 |
|
||||
| warrant_chain.py | 89 | 320 (9 tests) | 3.6 |
|
||||
| t3_bound_calculator.py | 249 | 351 (51 tests) | 1.4 |
|
||||
| fork_score.py | 298 | 403 (18 tests) | 1.35 |
|
||||
| weights.py | 73 | 180 (16 tests) | 2.5 |
|
||||
| pi_star/protocol+registry | 124 | 280 (21 tests) | 2.3 |
|
||||
| qa/progress.py | 85 | 226 (31 tests) | 2.7 |
|
||||
| qa/prompts.py | 153 | 207 (20 tests) | 1.35 |
|
||||
| concepts/extract.py | 288 | 278 (20 tests) | 0.97 |
|
||||
| textbooks_manifest.py | 236 | 463 (43 tests) | 1.96 |
|
||||
|
||||
Pattern: small contract-defining modules (weights, prompts,
|
||||
warrant_chain) get 1.4-3.6× test LOC because they're foundation;
|
||||
larger modules (warrant_resolver, aliases) settle around
|
||||
0.5-1.0× because they're more code-with-tests-per-feature than
|
||||
contract-with-tests-per-rule.
|
||||
|
||||
The ratio is a code-review heuristic: a new substrate-paper-spec'd
|
||||
primitive without ≥1× test/code ratio is suspect on landing.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue