diff --git a/bench/fixtures/textbooks/manifest-v1.jsonl b/bench/fixtures/textbooks/manifest-v1.jsonl index 9b18560..35af594 100644 --- a/bench/fixtures/textbooks/manifest-v1.jsonl +++ b/bench/fixtures/textbooks/manifest-v1.jsonl @@ -6,7 +6,7 @@ {"id":"boole-laws-of-thought-1854","title":"An Investigation of the Laws of Thought","author":"George Boole","year":"1854","license":"PD","license_url":"https://en.wikipedia.org/wiki/Public_domain","domain":"logic","pillar_targets":["I"],"urls":[],"home_url":"https://www.gutenberg.org/ebooks/15114","tex_url":"https://www.gutenberg.org/files/15114/15114-t/15114-t.tex","pdf_url":"https://www.gutenberg.org/files/15114/15114-pdf.pdf","notes":"Project Gutenberg eBook #15114 — PDF/TeX only, no HTML format. URLs intentionally empty until PDF / TeX source support lands. Boole's foundational treatise establishing propositional logic / Boolean algebra."} {"id":"aristotle-prior-analytics-jenkinson","title":"Prior Analytics","author":"Aristotle (A. J. Jenkinson, transl.)","year":"~350 BCE; Jenkinson translation 1928","license":"PD","license_url":"https://en.wikipedia.org/wiki/Public_domain","domain":"logic","pillar_targets":["I"],"urls":["https://en.wikisource.org/wiki/Prior_Analytics"],"home_url":"https://en.wikisource.org/wiki/Prior_Analytics","crawl_url":"https://en.wikisource.org/wiki/Prior_Analytics","crawl_depth":1,"crawl_max":20,"notes":"Wikisource HTML. The /wiki/Prior_Analytics page is a disambiguation/landing — only ~75% complete as raw text; the deeper transcription lives at /wiki/Index:O._F._Owen%27s_Organon_of_Aristotle_Vol._1_(1853).djvu and its /Page: subpages, which the crawler can follow at depth=2+."} {"id":"morin-open-data-structures","title":"Open Data Structures","author":"Pat Morin","year":"2013+","license":"CC-BY-2.5","license_url":"https://creativecommons.org/licenses/by/2.5/ca/","domain":"computer-science","pillar_targets":[],"urls":["https://opendatastructures.org/ods-cpp/","https://opendatastructures.org/ods-python/","https://opendatastructures.org/ods-java/"],"home_url":"https://opendatastructures.org/","crawl_url":"https://opendatastructures.org/","crawl_depth":3,"crawl_max":120,"notes":"Comprehensive open data-structures text. CC-BY-2.5 (Canadian port) — fully redistributable. Three language editions (C++, Python, Java) of the same content. ODS crawl @ depth=3 yields ~64 chapters / ~80KB across all three editions."} -{"id":"newton-principia-motte","title":"The Mathematical Principles of Natural Philosophy","author":"Isaac Newton (Andrew Motte transl., 1729 / Motte-Cajori 1846)","year":"1687/1729/1846","license":"PD","license_url":"https://en.wikipedia.org/wiki/Public_domain","domain":"classical-physics","pillar_targets":["VI"],"urls":["https://en.wikisource.org/wiki/The_Mathematical_Principles_of_Natural_Philosophy"],"home_url":"https://en.wikisource.org/wiki/The_Mathematical_Principles_of_Natural_Philosophy","crawl_url":"https://en.wikisource.org/wiki/The_Mathematical_Principles_of_Natural_Philosophy","crawl_depth":2,"crawl_max":60,"notes":"Wikisource HTML, English Motte 1729 / Motte-Cajori 1846 translations. Newton's foundational treatise — Laws of Motion + universal gravitation. Cited by claim-pack pillar VI records."} +{"id":"newton-principia-motte","title":"The Mathematical Principles of Natural Philosophy","author":"Isaac Newton (Andrew Motte transl., 1729 / Motte-Cajori 1846)","year":"1687/1729/1846","license":"PD","license_url":"https://en.wikipedia.org/wiki/Public_domain","domain":"classical-physics","pillar_targets":["VI"],"urls":["https://en.wikisource.org/wiki/The_Mathematical_Principles_of_Natural_Philosophy_(1729)/Axioms,_or_Laws_of_Motion","https://en.wikisource.org/wiki/The_Mathematical_Principles_of_Natural_Philosophy_(1729)/Definitions","https://en.wikisource.org/wiki/The_Mathematical_Principles_of_Natural_Philosophy_(1729)/Rules_of_Reasoning_in_Philosophy"],"home_url":"https://en.wikisource.org/wiki/The_Mathematical_Principles_of_Natural_Philosophy","crawl_url":"https://en.wikisource.org/wiki/The_Mathematical_Principles_of_Natural_Philosophy_(1729)","crawl_depth":3,"crawl_max":120,"notes":"Wikisource HTML, English Motte 1729 translation. Pinned to the (1729) edition root + axioms / definitions / rules-of-reasoning subpages. Initial crawl_url at `/The_Mathematical_Principles_of_Natural_Philosophy` (no edition) bottomed out at navigation cruft — most chunks were Wikisource portal pages, NOT the Laws of Motion text. Subpages explicitly listed under `urls` (Axioms_or_Laws_of_Motion, Definitions, Rules_of_Reasoning_in_Philosophy) so they ingest even if the BFS doesn't reach them at depth=3. Cited by claim-pack pillar VI records (Newton's three laws + conservation of momentum + work-energy theorem)."} {"id":"aristotle-posterior-analytics","title":"Posterior Analytics","author":"Aristotle (G. R. G. Mure, transl.)","year":"~350 BCE; Mure translation 1928","license":"PD","license_url":"https://en.wikipedia.org/wiki/Public_domain","domain":"logic","pillar_targets":["I"],"urls":["https://en.wikisource.org/wiki/Posterior_Analytics"],"home_url":"https://en.wikisource.org/wiki/Posterior_Analytics","crawl_url":"https://en.wikisource.org/wiki/Posterior_Analytics","crawl_depth":1,"crawl_max":20,"notes":"Wikisource HTML. Aristotle's treatise on demonstrative knowledge, companion to Prior Analytics. The traditional Mure / Owen translations are PD by age."} {"id":"kolmogorov-foundations-probability-1933","title":"Foundations of the Theory of Probability","author":"Andrey Kolmogorov (Nathan Morrison transl., 1956)","year":"1933 (German); 1956 (English Morrison)","license":"PD","license_url":"https://en.wikipedia.org/wiki/Public_domain","domain":"probability","pillar_targets":["V"],"urls":[],"home_url":"https://en.wikipedia.org/wiki/Foundations_of_the_Theory_of_Probability","notes":"License analysis pending. German original (Grundbegriffe der Wahrscheinlichkeitsrechnung, 1933) PD-by-age in EU but US copyright restored via URAA through life+70 (Kolmogorov d.1987 → US PD 2058). Morrison 1956 English translation: Chelsea Publishing copyright through life+70 of translator. URLs intentionally empty until license decision lands; entry kept for citation traceability."} {"id":"lukasiewicz-aristotle-syllogistic-1951","title":"Aristotle's Syllogistic from the Standpoint of Modern Formal Logic","author":"Jan Łukasiewicz","year":"1951 (Oxford); 2nd ed 1957","license":"PROPRIETARY","license_url":"https://global.oup.com/academic/legal/copyright/","domain":"logic","pillar_targets":["I"],"urls":[],"home_url":"https://en.wikipedia.org/wiki/Jan_%C5%81ukasiewicz","notes":"Oxford University Press, copyright through life+70 (Łukasiewicz d.1956 → PD 2027 in EU, longer in US via URAA). NOT redistributable under arborist's allow-list. Entry kept as a license-fail placeholder; the 1921 Polish primary papers ('Two-valued logic', 'On the principle of contradiction in Aristotle') are themselves PD by age but lack accessible HTML editions."} diff --git a/bench/results/citation-aliases-batch-2026-05-10.md b/bench/results/citation-aliases-batch-2026-05-10.md new file mode 100644 index 0000000..490273c --- /dev/null +++ b/bench/results/citation-aliases-batch-2026-05-10.md @@ -0,0 +1,151 @@ +# Citation-aliases batch + Newton recrawl — 2026-05-10 + +**Date:** 2026-05-10 (UTC) +**Author:** blackops + fox +**Method:** registered three high-confidence citation-alias decisions +under #000041 (Goldstein/Newton, Dummit-Foote/Judson, Stanley/Brualdi/Knuth/Bogart) +plus 14 full-string aliases for compound pillar-VII citations, plus +deeper Newton recrawl after the original `crawl_url` bottomed out at +Wikisource navigation pages. + +## Headline + +**18 → 35 records resolve (+17, 38% coverage).** Pillar IV stays +fully clear (18/18). Pillar VII jumped from 0 → 9/14. Pillar IX +gained 5/14 (Dummit-Foote substitute). Pillar VI gained 3/5 (Newton +substitute + deeper crawl). + +## Changes landed in this batch + +### 17 citation-alias rows in `~/.arborist/shards/000.db` + +All `decision_by="fox 2026-05-10"` per #000041 audit discipline. + +| Original `source_reference` | Substitute | Records lifted | +|---|---|---| +| Classical Mechanics by Herbert Goldstein, Charles P. Poole, and John L. Safko | Newton's *Principia* | 2 / 3 | +| Classical Mechanics by Herbert Goldstein et al. | Newton's *Principia* | 1 / 2 | +| Abstract Algebra by David S. Dummit and Richard M. Foote | Judson AGT | 5 / 6 | +| (×14 compound pillar-VII full strings, all citing Stanley + Brualdi + Knuth) | Bogart CTGD | 9 / 14 | + +### Newton shard rebuilt + +Original `crawl_en_wikisource_org_newton.db` was 60 docs / 289 chunks +but mostly Wikisource navigation pages (Community portal, Sandbox, +Author pages, etc.) — depth=2 BFS from `/The_Mathematical_Principles_of_Natural_Philosophy` +(no edition specifier) followed up the wiki sidebar instead of into +the actual book content. Empirically: zero chunks contained "Law of +Inertia" or "first law" or "every body". + +Manifest entry pinned to `(1729)` edition root + explicit `urls` for +Axioms / Definitions / Rules of Reasoning subpages. Re-crawl at +depth=3 max=120 produced `textbook_newton-principia-motte.db` with +**120 docs / 779 chunks** of actual Principia text. + +### 2 classical-physics term aliases (#000042) + +`momentum ↔ quantity of motion` and `action reaction ↔ action equal +reaction`. Registered for completeness; did NOT lift the 2 stuck +pillar VI records (Newton's Third Law, Conservation of Momentum) +because the resolver's discriminating-token cascade for those +specific records doesn't include "momentum" / "action reaction" as +the load-bearing tokens. Cascade-improvement deferred. + +## Per-pillar coverage after batch + +| Pillar | Before | After | Delta | +|---|---|---|---| +| I (logic) | 0/13 | 0/13 | — (medium-confidence aliases not yet approved) | +| II (set theory) | 0/10 | 0/10 | — (Jech/Cantor alias not yet approved) | +| III (arithmetic) | 0/13 | 0/13 | — (Landau + Gödel not yet ingested) | +| IV (geometry) | 18/18 | 18/18 | — (already complete) | +| V (probability) | 0/5 | 0/5 | — (Kolmogorov not ingested) | +| VI (classical phys) | 0/5 | 3/5 | +3 via Goldstein/Newton + deeper crawl | +| VII (combinatorics) | 0/14 | 9/14 | +9 via Stanley/Brualdi/Knuth/Bogart | +| IX (lambda + algebra) | 0/14 | 5/14 | +5 via Dummit-Foote/Judson | + +## Alias attribution discrepancy noted + +Citation-alias-resolved derivations are labeled `process_id = +"warrant-resolver-v1"` (DIRECT) instead of `"warrant-resolver-v1+alias"` +in the audit trail. Reason: `via_alias=True` is currently set by the +**term-alias** OR-cascade path in `resolve_chunks`, not by the +**citation-alias** substitute-citation-append path higher up in +`warrant_resolve`. The chain is real and audit-clean (the substitute +Citation's `raw` field carries `substitute_ref`), but the audit-line +flag mis-labels them as direct. + +This is a labeling-only issue; doesn't affect chain validity. Fix +deferred — `_render_warrant_chain_tail` could distinguish by +matching the resolved citation against `lookup_citation_aliases`, +or `WarrantResolution` could carry an explicit `via_citation_alias` +field to flow into `process_id`. + +## Stuck records (alias registered, FTS5 query miss) + +Five records still don't resolve despite the alias being live: + +- **Pillar VI:** Newton's Third Law, Conservation of Momentum. + Modern vocabulary doesn't appear in Newton's 1729 prose; cascade + tokens drop them. +- **Pillar VII:** Empty-Set/Boundary, Hockey-Stick, Vandermonde, + Catalan, Stars and Bars. Bogart CTGD's coverage skews toward + intro-level primitives; advanced enumerative identities aren't + named with these moderns labels. +- **Pillar IX:** Orbit-Stabilizer (Dummit-Foote alias). Judson + covers it but the FTS5 cascade doesn't surface it cleanly. + +These are tractable one-by-one with targeted term-aliases or +cascade-discriminator tweaks, but each is incremental. + +## Stuck whole pillars (no PD substitute ingested) + +- **Pillar I (Mendelson + Enderton, 13 records)** — Russell IMP + + De Morgan are ingested. Medium-confidence alias decisions + pending fox approval; Russell IMP's coverage of formal-proof + axioms is partial. +- **Pillar II (Jech, 10 records)** — Cantor 1915 ingested but + alias not approved (Cantor doesn't cover modern forcing / + large-cardinal axioms). +- **Pillar III (Landau, Gödel, 13 records)** — neither ingested + (PD-by-age but no clean HTML edition located). +- **Pillar V (Kolmogorov, 5 records)** — not ingested; URAA- + blocked in US until 2058. +- **Pillar IX lambda subset (Barendregt, 7 records)** — not + ingested; Church 1932-1941 papers blocked by 95-year US rule + until 2027-2031. + +## Next decisions for fox + +1. **Medium-confidence aliases:** Mendelson → Russell IMP, Enderton → + Russell IMP, Jech → Cantor. Projected lift: +13-21 records. + Caveat: partial coverage — Russell IMP doesn't cover Mendelson's + incompleteness chapters. +2. **Phase 4 acquisitions:** track down PD HTML editions of + Hilbert-Ackermann, Landau, Gödel. None found in this session. +3. **Term-alias / cascade tuning** for the 5 stuck-via-vocabulary + records (incremental; ~1 record per fox decision). + +## Reproducibility + +```bash +# Citation aliases survive in shards-dir/000.db (audit table); to +# replay, dump and re-add via `arborist alias citation add ...`. +arborist alias citation list + +# Newton recrawl (idempotent; same shard name): +make textbook-newton + +# Resolver: +arborist warrant-resolve --use-aliases --write +# → 35 / 92 (38%) +``` + +## References + +- `#000041` — citation-aliases mechanism +- `#000042` — term-aliases mechanism +- `bench/results/textbook-base-knowledge-upgrade-2026-05-09.md` — + prior shift's ingest of the four base-knowledge textbooks +- `bench/results/hilbert-pillar-iv-fully-resolved-2026-05-09.md` — + pillar IV's 18/18 completion