docs: index docs/crawler.md in CLAUDE.md docs index

This commit is contained in:
russell@unturf.com 2026-05-22 07:42:32 -04:00
parent 24c7596bc4
commit 7fedea3f5f
No known key found for this signature in database

View file

@ -592,6 +592,9 @@ Architecture / ongoing work:
- `docs/cti-architecture.md` — CTI Clause Tree Intelligence.
- `docs/mesh.md` — mesh wire format + group-key state machine.
- `docs/crawler.md` — web crawler: BFS discovery, robots/feed
handling, polite vs `--fast`, and the content-addressed
diagnostics (dedupe by `document_root`, orphan finding).
- `docs/benchmarks.md` — orientation: harnesses, fixtures,
signal floor, make targets, bench-row schema, addenda index.
Read first when running a bench.