docs: document scanner improvement loop in CLAUDE.md
This commit is contained in:
parent
0ac47a5b75
commit
89f094d242
1 changed files with 44 additions and 0 deletions
44
CLAUDE.md
44
CLAUDE.md
|
|
@ -123,6 +123,50 @@ qBittorrent, Transmission, Deluge, aria2.
|
|||
|
||||
**No stone unturned complete as of 2026-03-29.** All 320 defect directories have a patch or CLEAN marker. 629 confirmed sites, 571 UNDF assigned. Next wave: deeper scans on single-defect targets (allegro5, caddy, ceph, grafana, etc.) or clone new targets not yet in defects/.
|
||||
|
||||
## Test Harness and Scanner — How They Improve
|
||||
|
||||
**The scanner does not improve on its own.** It improves when we do real work:
|
||||
|
||||
1. **Find a new hit** — scan a target, find CWE-407 or other MOAD pattern
|
||||
2. **Write the patch** — `defects/{project}/patch/{slug}.patch` with `# UNDF: ...` header
|
||||
3. **Write the tests** — unit in `tests/unit/`, integration in `tests/integration/`, functional in `tests/functional/`
|
||||
4. **Add intel** — run `generate_undf.py`, push to undefect.com
|
||||
5. **Update CLAUDE.md** — if a new defect pattern or fix signature was involved, add it to `tests/scan_verify.py` `DEFECTIVE_SIGS` / `FIX_SIGS` so that pattern promotes from WARN to PASS on future scans
|
||||
|
||||
**If a patch shows WARN in `make scan-verify`:**
|
||||
- Check whether its defective pattern needs a new entry in `DEFECTIVE_SIGS`
|
||||
- Check whether its fix pattern needs a new entry in `FIX_SIGS`
|
||||
- Add the pattern, re-run, confirm it promotes to PASS
|
||||
- Commit the pattern addition alongside the new patch/test
|
||||
|
||||
**The three feedback loops:**
|
||||
```
|
||||
New hit found
|
||||
→ patch written
|
||||
→ tests pass
|
||||
→ scan_verify.py PASS
|
||||
→ UNDF post on undefect.com
|
||||
→ scanner coverage grows by 1
|
||||
|
||||
WARN entry spotted
|
||||
→ read the patch to find the unrecognized pattern
|
||||
→ add pattern to DEFECTIVE_SIGS or FIX_SIGS
|
||||
→ re-run make scan-verify, confirm PASS
|
||||
→ scanner improves for all future patches using that pattern
|
||||
|
||||
NO_PATCH entry seen
|
||||
→ write the missing patch for that defect dir
|
||||
→ coverage-check passes for that entry
|
||||
→ scanner coverage grows
|
||||
```
|
||||
|
||||
**Current state (2026-04-12):**
|
||||
- 1258 UNDF entries in registry
|
||||
- 454 PASS, 441 WARN, 12 FAIL, 300 NO_PATCH, 51 PENDING
|
||||
- FAIL = real stub patches needing actual diff content (thunderbird 1-6, systemd-0003, etc.)
|
||||
- NO_PATCH = project dir exists, patch file not yet written
|
||||
- WARN = patch present but defective/fix pattern not in our lists yet
|
||||
|
||||
## UNDF Numbering System
|
||||
|
||||
Every defect in this repo gets a **UNDF-2026-XXXXXXXXX** identifier (9-digit, covers 999,999,999 entries).
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue