russell@unturf.com
8d433427d2
First make fetch-textbooks run revealed three URL issues that the license-validator catches as policy-correct but the underlying hosts return as 404: - Keller-Trotter applied-combinatorics.org used a stale /appcomb/sec_pre.html path; the canonical PreTeXt-generated book lives at /book/app-comb-2.html. - Open Data Structures uses /ods-cpp/ /ods-python/ /ods-java/ trailing-slash directory paths, not /ods-cpp.html (404). Three language editions of the same content; ingest all three so the multi-language structure stays intact. - Hilbert Foundations of Geometry on Wikisource resolves to a 1906 Popular Science Monthly article ABOUT Hilbert's work, not Hilbert's translated text. Project Gutenberg eBook #17384 is PDF/TeX only — no clean HTML edition exists. Same status as Boole #15114: kept the entry as a placeholder for license + provenance documentation, urls intentionally empty until PDF source support lands (out of scope per #000031). Final fetch result: 7/7 manifested URLs ingested (Bogart, Keller-Trotter front matter, Levin front matter, Aristotle Prior Analytics, ODS C++, ODS Python, ODS Java) — 7 docs / 7 chunks / 86 edges / 8/8 Merkle proofs verify (verify samples 8 because Aristotle yields multiple chunks). Two textbooks (Hilbert + Boole) carry zero URLs; the helper script's allow-list passes them through (license is PD; license_url is set) but emits no URLs since their `urls` array is empty. Documents this gap explicitly in each entry's notes field so future operators see what awaits PDF source support. Front-matter URLs alone yield minimal indexable content per textbook (PreTeXt single-chapter scaffolds). To grow real coverage: append more chapter URLs to a textbook's `urls` array, OR use `make crawl-ingest URL=<home>` per the README's "Going deeper" section. |
||
|---|---|---|
| .. | ||
| batteries | ||
| fixtures | ||
| results | ||
| scripts | ||
| emergent_log.jsonl | ||
| qa_questions.txt | ||
| qa_questions_metacog_subset.txt | ||
| qa_questions_quantifier_baseline.txt | ||
| qa_questions_quantifier_subset.txt | ||
| qa_questions_smoke.txt | ||
| qa_sweep.py | ||
| run.sh | ||