# Ticket #000004 — Directive coverage in bench summary

**Status:** closed · landed in commit `acd1f9c` (2026-05-01)
**Opened:** 2026-05-01
**Closed:** 2026-05-01
**Directive:** [D8](seven-point-program.md) — Automate only after the
invariants are test-pinned. Directive coverage in the bench output is
the substrate that makes D1-D7 enforcement visible per-run.
**Scope:** Wire the seven-point program into `bench/qa_sweep.py` so
each row carries a `directive_compliance` dict and the markdown
summary emits a per-mode coverage table.
**Audience:** fox + future blackops shifts.
**Hard constraint:** the bench output never edits the run-DAG, the
audit chain, or the providence_cache. Directive compliance lives
alongside the existing per-row signals (deflection, capacity,
strict-rate); never feeds back into the proof path.

---

## Why this ticket exists

The seven-point program is governance prose unless the bench
enforces it per-run. Without per-row directive compliance, an
operator can't tell whether a strict-rate climb came from real
substrate work or from accidentally weakening one of the directives.

Per the five-step algorithm step 5 ("automate only after test-
pinning"): the bench harness IS the automation substrate, so it
must enforce the directives before any feature gets layered on top.

## Implementation summary (landed `acd1f9c`)

`bench/qa_sweep.py`:

- `_directive_compliance(answer_mode, result, err)` — pure helper
  computing per-row pass/fail for the directives whose pinning is
  observable from a single bench row (D2, D3, D4, D6, D7). D1, D5,
  D8 are global properties of the substrate; tracked once in the
  program doc rather than per-row.
- Each `_run_one` result row gains a `directive_compliance: dict`
  field.
- `_summarize` aggregates per-row booleans into per-mode
  `directive_pass: dict[str, int]` counts.
- `_render_markdown` emits a `## directive coverage (seven-point
  program)` section with a per-mode per-directive table showing
  `count/N (pct%)`.

Per-row checks:

| ID | Check                                                              |
|----|---------------------------------------------------------------------|
| D2 | `answer_mode in ("claim_lattice_pointer", "claim_lattice")`         |
| D3 | lattice mode + `run_dag_root` populated (full Module L pending #000002) |
| D4 | `run_dag_root` populated (retrieval-plan hash pending #000001)      |
| D6 | warrant ran (lattice mode today; per-shape gating pending #000003)  |
| D7 | `audit_mode in {STRICT, HYBRID, UNGROUNDED}` (renderer pinned in `test_cli_render.py`) |

## Tests

5 tests in `tests/test_bench_qa_sweep.py`:

- `test_summarize_aggregates_directive_pass_counts_per_mode`
- `test_render_markdown_directive_coverage_section`
- `test_directive_compliance_helper_marks_quote_mode_d2_false`
- `test_directive_compliance_helper_marks_lattice_mode_d2_true`
- `test_directive_compliance_returns_empty_on_error_row`

## Closure rationale

Filed as a closed ticket immediately so the design log captures the
artifact alongside the work. The convention from `docs/TICKETS.md`:
"Closed tickets stay in place as the design log; do not delete
tickets — they are the design log."

Future evolutions (per-row D1/D5 tracking, error-row directive
weighting, cross-mode comparison) get their own tickets. This one
documents the v1 substrate.
