Created /crawler.html page explaining:
- Our user agent (uncloseai.com/1.42)
- What we use the crawler for (research, Q&A, content aggregation)
- Ethical practices (robots.txt, crawl delays, caching, smart depth)
- How to block the crawler
- Link to Discord bot
Added crawler link to sidebar navigation in inference.html
This provides transparency for webmasters and explains our ethical
crawling practices in human-friendly terms.
