ticket #000031 follow-ups B-1 + B-2: alias attribution + source-side title author

B-1: via_citation_alias attribution — resolver no longer mislabels
citation-alias-substituted chains as DIRECT.

  - New Citation.via_citation_alias field (default False, preserves
    parse-from-source-ref path).
  - warrant_status sets via_citation_alias=True on substitute
    Citations from #000041 lookup_citation_aliases.
  - resolve_chunks reads citation.via_citation_alias as a "floor" for
    via_alias on every match it produces (Pass 1 hits inherit it
    too, not just Pass 2 term-alias hits). Audit-honest: matches
    from a substitute Citation are alias-driven regardless of which
    cascade pass found the chunk.

  Live re-resolve under the new attribution: 18 direct + 74 +alias
  (was 75/17 mis-labeled). The 18 direct = exactly Hilbert pillar IV
  records resolving on the literally-cited Hilbert textbook. All 74
  records resolved via citation-alias substitution now carry
  process_id="warrant-resolver-v1+alias" in derivations.

  3 new unit tests (test_warrant_resolver.py): default-False on
  parsed Citations, explicit-True construction works, resolve_chunks
  propagates the floor onto every ResolutionMatch.

B-2: source-side title-from-author backfill — eliminates the
per-shard SQL UPDATE workaround.

  - HtmlPageSource accepts default_author kwarg; appends ', by
    <author>' to ingested document titles when the <title> tag
    doesn't already include the surname.
  - TextbookTexSource accepts default_author kwarg; appends ' by
    <author>' to titles when the LaTeX has no \author{} macro AND
    no PG-style 'Author:' boilerplate.
  - _CrawledHtmlSource (BFS-crawler bridge) accepts default_author
    kwarg; same append logic. ingest_crawled() and arborist crawl
    --ingest plumb it through.
  - arborist ingest --author + arborist crawl --author CLI flags.
  - bench/scripts/textbooks_manifest.py:cmd_lookup emits the
    manifest's `author` field as a 7th tab column.
  - make textbook target reads the author column and threads
    --author into both crawl-ingest and shallow-ingest paths.

  Idempotency preserved — surname-already-in-title detection prevents
  double-stamping on re-ingest. Shards previously SQL-backfilled
  (Cantor / Russell IMP / Bogart / Judson / Levin / KT / Peano /
  Grinstead-Snell) keep their existing titles; new ingests pick up
  the author signal at source time.

  Live smoke: arborist ingest --source html --author "Bertrand
  Russell" against PG #41654 yields title "Introduction to
  Mathematical Philosophy | Project Gutenberg, by Bertrand Russell"
  with no SQL UPDATE needed.

Total: 1655 tests pass (was 1652). Both follow-ups land additive,
fail-closed, idempotent. The two cleanup items from #000031
Phase 3's commit message are now closed.
This commit is contained in:
russell@unturf.com 2026-05-10 09:35:49 -04:00
parent 7e81425d49
commit 551c9695e0
No known key found for this signature in database
8 changed files with 304 additions and 15 deletions

View file

@ -216,3 +216,125 @@ def test_build_record_query_cascade_orders_correctly():
assert queries[0] == '"line incidence"'
# The content-token AND-join should appear in the cascade.
assert any(" AND " in q for q in queries[1:])
# --- B-1: via_citation_alias attribution -----------------------
def test_citation_via_citation_alias_default_false():
"""Citations parsed from the original source_reference start
with via_citation_alias=False backward-compatible."""
from arborist.qa.warrant_resolver import Citation, parse_citation
cs = parse_citation("Foundations of Geometry by David Hilbert")
assert all(c.via_citation_alias is False for c in cs)
def test_citation_explicit_via_citation_alias_flag():
"""Construct a substitute Citation as if from #000041 lookup —
via_citation_alias=True. Used by warrant_resolve to flag chains
that came through a citation-alias substitution rather than the
parsed source_reference."""
from arborist.qa.warrant_resolver import Citation
sub = Citation(
title="Russell IMP",
authors=("Bertrand Russell",),
raw="Russell IMP by Bertrand Russell",
via_citation_alias=True,
)
assert sub.via_citation_alias is True
def test_resolve_chunks_propagates_via_citation_alias_to_match(tmp_path, monkeypatch):
"""When resolve_chunks is called with a Citation whose
via_citation_alias=True is set, every ResolutionMatch it produces
inherits via_alias=True so process_id correctly attributes the
derivation as `warrant-resolver-v1+alias`."""
import sqlite3
from pathlib import Path
from arborist.qa.warrant_resolver import (
Citation,
ResolutionMatch,
resolve_chunks,
)
# Build a tiny fake shard cluster: shards/000.db (empty) +
# crawl/textbook_test.db with one document + one chunk that the
# _shard_matches_citation heuristic can find by author surname.
shards_dir = tmp_path / "shards"
crawl_dir = tmp_path / "crawl"
shards_dir.mkdir()
crawl_dir.mkdir()
from arborist.store import SCHEMA_SQL
main_db = shards_dir / "000.db"
sqlite3.connect(str(main_db)).executescript(SCHEMA_SQL).close()
sub_db = crawl_dir / "textbook_test.db"
conn = sqlite3.connect(str(sub_db))
conn.executescript(SCHEMA_SQL)
conn.execute(
"INSERT INTO documents "
"(document_root, document_uri, source_type, kind, "
" compression_depth, title, chunking_version, "
" canonicalization_version, schema_version, ingest_ts) "
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
("a" * 64, "https://test/", "html", "surface", 0,
"Test Book by Bertrand Russell", "tok-512-v1",
"norm-v1", "v9.8.0", 0),
)
conn.execute(
"INSERT INTO chunks (chunk_id, document_root, idx, leaf_hash, content) "
"VALUES (1, ?, 0, ?, ?)",
("a" * 64, "h" * 64, "philosophy mathematics test sample"),
)
conn.execute(
"INSERT INTO chunks_fts (rowid, content) VALUES (1, ?)",
("philosophy mathematics test sample",),
)
conn.commit()
conn.close()
# Citation with via_citation_alias=True (mimics what #000041
# citation-alias lookup appends).
sub_cit = Citation(
title="Test Book",
authors=("Bertrand Russell",),
raw="Test Book by Bertrand Russell",
via_citation_alias=True,
)
matches = resolve_chunks(
sub_cit,
shards_dir,
theorem_name="Axiom of Test",
record_content="philosophy mathematics test sample axiom",
limit=3,
)
assert matches, "expected a match for the substitute citation"
assert all(
isinstance(m, ResolutionMatch) and m.via_alias is True
for m in matches
), "every match from a via_citation_alias=True Citation must be flagged via_alias"
# Sanity check the inverse — Citation without the flag → matches
# are via_alias=False (the existing direct-cascade path).
parsed_cit = Citation(
title="Test Book",
authors=("Bertrand Russell",),
raw="Test Book by Bertrand Russell",
)
plain_matches = resolve_chunks(
parsed_cit,
shards_dir,
theorem_name="Axiom of Test",
record_content="philosophy mathematics test sample axiom",
limit=3,
)
assert plain_matches
assert all(m.via_alias is False for m in plain_matches), (
"Citation parsed from source_ref (no via_citation_alias) "
"must NOT inherit via_alias=True from the citation_alias_floor"
)