Implements both alias mechanisms (citation-aliases #000041, term-aliases #000042) as one cohesive layer. Same audit discipline; same opt-in via --use-aliases on warrant-resolve; same distinct process_id "warrant-resolver-v1+alias" on alias- resolved derivations rows. What landed =========== arborist/store.py — two new tables under SCHEMA_SQL: citation_aliases — substitute textbook for proprietary cite term_aliases — bridge vocabulary mismatches (incidence ↔ connection in geometry, etc.) Both with NOT NULL audit fields (decision_at + decision_by + optional decision_rationale); both with PK constraints ensuring idempotent re-add. arborist/qa/aliases.py — helper module: add_citation_alias / list / lookup / remove add_term_alias / list / lookup (bidirectional) / remove expand_query_with_term_aliases — OR-rewrites FTS5 tokens while preserving phrase syntax domain_for_pillar — Roman numeral → domain string arborist/qa/warrant_resolver.py — two-pass cascade: Pass 1: unaliased queries (matches carry via_alias=False) Pass 2: alias-expanded queries (only when pass 1 missed; matches carry via_alias=True) ResolutionMatch grew a via_alias field; warrant_resolve uses it to pick the right process_id per derivation row. iter_claim_pack_records now yields a 6-tuple including the pillar (parsed from doc URI) so domain lookup works. arborist/cli.py — alias subcommand group: arborist alias citation add ORIGINAL --substitute SUB --by FOX [...] arborist alias citation list / remove arborist alias term add TERM ALT --domain D --by FOX [...] arborist alias term list / remove warrant-resolve --use-aliases flag sweep --target warrants --use-aliases flag All audit fields fail-closed at the API surface (refuses on empty --by; ValueError raised at the helper level). End-to-end smoke test ===================== Registered the Hilbert smoke-test alias: arborist alias term add incidence connection \ --domain geometry \ --by "blackops 2026-05-09 (smoke test)" \ --rationale "Hilbert 1902 Townsend uses 'connection' for what modern texts call 'incidence'" Re-ran warrant-resolve --use-aliases --write: records_total: 92 records_resolved: 15 (was 11 without aliases) derivations_written: 15 Breakdown by process_id: warrant-resolver-v1: 11 (original-citation matches) warrant-resolver-v1+alias: 4 (alias-resolved Hilbert axioms) The 4 alias-resolved records are exactly the Hilbert "Incidence" axioms blocked by terminology mismatch in #000040 §6: Axiom of Line Incidence Axiom of Plane Incidence Axiom of Point-Line Incidence Axiom of Point-Plane Incidence All four bound to chunk 56 in the Hilbert TeX surface — the chapter discussing "axioms of connection" (Hilbert's original 1902 vocabulary). Audit trail correctly distinguishes substituted chains from original ones. Tests: 18 new in test_aliases.py covering add / list / lookup / remove / domain isolation / lowercase normalization / bidirectional lookup / audit-discipline raises / query expansion (basic + phrase-preserving + no-match passthrough + unreachable-DB fallback). Full suite: 1623 passed / 28 skipped. Tickets #000041 + #000042 closed. Operators can now add more aliases via the CLI as fox makes decisions per #000038. The alias mechanism is fail-closed by default — existing warrant-resolve runs without --use-aliases continue to produce the original 11/92 chains; --use-aliases opt-in adds the substituted chains alongside without polluting the unsubsituted audit trail.
This commit is contained in:
parent
44b38c55da
commit
d3c40c95d2
8 changed files with 1255 additions and 26 deletions
276
arborist/cli.py
276
arborist/cli.py
|
|
@ -3450,12 +3450,160 @@ def _cmd_warrant_resolve(args: argparse.Namespace) -> int:
|
|||
print("--shards-dir is required for warrant-resolve", file=sys.stderr)
|
||||
return 2
|
||||
summary = warrant_resolve(
|
||||
shards_dir, write=args.write, limit=args.limit
|
||||
shards_dir,
|
||||
write=args.write,
|
||||
limit=args.limit,
|
||||
use_aliases=getattr(args, "use_aliases", False),
|
||||
)
|
||||
print(json.dumps(summary, indent=2))
|
||||
return 0
|
||||
|
||||
|
||||
def _aliases_db_path(args: argparse.Namespace) -> str | None:
|
||||
"""Resolve the aliases DB path. Default: shard 000 of the shards
|
||||
cluster (alongside claim-pack records). Operators can override
|
||||
via --aliases-db."""
|
||||
explicit = getattr(args, "aliases_db", None)
|
||||
if explicit:
|
||||
return str(explicit)
|
||||
shards_dir = args.global_shards_dir or getattr(args, "shards_dir", None)
|
||||
if not shards_dir:
|
||||
return None
|
||||
return str(Path(shards_dir) / "000.db")
|
||||
|
||||
|
||||
def _cmd_alias_citation_add(args: argparse.Namespace) -> int:
|
||||
"""Add a citation alias. Refuses without --by; audit-discipline
|
||||
fail-closed at the API surface."""
|
||||
from arborist.qa.aliases import add_citation_alias
|
||||
|
||||
db = _aliases_db_path(args)
|
||||
if not db:
|
||||
print("--shards-dir or --aliases-db is required", file=sys.stderr)
|
||||
return 2
|
||||
try:
|
||||
inserted = add_citation_alias(
|
||||
db,
|
||||
original_ref=args.original,
|
||||
substitute_ref=args.substitute,
|
||||
substitute_authors=args.author or [],
|
||||
substitute_title=args.title or "",
|
||||
decision_by=args.by,
|
||||
decision_rationale=args.rationale or "",
|
||||
)
|
||||
except ValueError as exc:
|
||||
print(f"alias add failed: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
print(json.dumps({"inserted": inserted, "db": db}, indent=2))
|
||||
return 0
|
||||
|
||||
|
||||
def _cmd_alias_citation_list(args: argparse.Namespace) -> int:
|
||||
from arborist.qa.aliases import list_citation_aliases
|
||||
|
||||
db = _aliases_db_path(args)
|
||||
if not db:
|
||||
print("--shards-dir or --aliases-db is required", file=sys.stderr)
|
||||
return 2
|
||||
aliases = list_citation_aliases(db, original_filter=args.filter)
|
||||
print(
|
||||
json.dumps(
|
||||
[
|
||||
{
|
||||
"original_ref": a.original_ref,
|
||||
"substitute_ref": a.substitute_ref,
|
||||
"substitute_authors": list(a.substitute_authors),
|
||||
"substitute_title": a.substitute_title,
|
||||
"decision_at": a.decision_at,
|
||||
"decision_by": a.decision_by,
|
||||
"decision_rationale": a.decision_rationale,
|
||||
}
|
||||
for a in aliases
|
||||
],
|
||||
indent=2,
|
||||
ensure_ascii=False,
|
||||
)
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
def _cmd_alias_citation_remove(args: argparse.Namespace) -> int:
|
||||
from arborist.qa.aliases import remove_citation_alias
|
||||
|
||||
db = _aliases_db_path(args)
|
||||
if not db:
|
||||
print("--shards-dir or --aliases-db is required", file=sys.stderr)
|
||||
return 2
|
||||
removed = remove_citation_alias(db, args.original, args.substitute)
|
||||
print(json.dumps({"removed": removed, "db": db}, indent=2))
|
||||
return 0
|
||||
|
||||
|
||||
def _cmd_alias_term_add(args: argparse.Namespace) -> int:
|
||||
"""Add a term alias. Refuses without --by; audit-discipline
|
||||
fail-closed at the API surface."""
|
||||
from arborist.qa.aliases import add_term_alias
|
||||
|
||||
db = _aliases_db_path(args)
|
||||
if not db:
|
||||
print("--shards-dir or --aliases-db is required", file=sys.stderr)
|
||||
return 2
|
||||
try:
|
||||
inserted = add_term_alias(
|
||||
db,
|
||||
term=args.term,
|
||||
alternate_term=args.alternate,
|
||||
domain=args.domain,
|
||||
decision_by=args.by,
|
||||
decision_rationale=args.rationale or "",
|
||||
)
|
||||
except ValueError as exc:
|
||||
print(f"alias add failed: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
print(json.dumps({"inserted": inserted, "db": db}, indent=2))
|
||||
return 0
|
||||
|
||||
|
||||
def _cmd_alias_term_list(args: argparse.Namespace) -> int:
|
||||
from arborist.qa.aliases import list_term_aliases
|
||||
|
||||
db = _aliases_db_path(args)
|
||||
if not db:
|
||||
print("--shards-dir or --aliases-db is required", file=sys.stderr)
|
||||
return 2
|
||||
aliases = list_term_aliases(db, domain=args.domain, term_filter=args.filter)
|
||||
print(
|
||||
json.dumps(
|
||||
[
|
||||
{
|
||||
"term": a.term,
|
||||
"alternate_term": a.alternate_term,
|
||||
"domain": a.domain,
|
||||
"decision_at": a.decision_at,
|
||||
"decision_by": a.decision_by,
|
||||
"decision_rationale": a.decision_rationale,
|
||||
}
|
||||
for a in aliases
|
||||
],
|
||||
indent=2,
|
||||
ensure_ascii=False,
|
||||
)
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
def _cmd_alias_term_remove(args: argparse.Namespace) -> int:
|
||||
from arborist.qa.aliases import remove_term_alias
|
||||
|
||||
db = _aliases_db_path(args)
|
||||
if not db:
|
||||
print("--shards-dir or --aliases-db is required", file=sys.stderr)
|
||||
return 2
|
||||
removed = remove_term_alias(db, args.term, args.alternate, args.domain)
|
||||
print(json.dumps({"removed": removed, "db": db}, indent=2))
|
||||
return 0
|
||||
|
||||
|
||||
def _cmd_sweep(args: argparse.Namespace) -> int:
|
||||
"""Unconscious sweep — drain the meta-cognition backlog.
|
||||
|
||||
|
|
@ -3482,7 +3630,10 @@ def _cmd_sweep(args: argparse.Namespace) -> int:
|
|||
|
||||
if args.target == "warrants":
|
||||
summary = warrant_resolve(
|
||||
shards_dir, write=args.write, limit=args.limit
|
||||
shards_dir,
|
||||
write=args.write,
|
||||
limit=args.limit,
|
||||
use_aliases=getattr(args, "use_aliases", False),
|
||||
)
|
||||
summary["target"] = "warrants"
|
||||
summary["mode"] = "write" if args.write else "dry-run"
|
||||
|
|
@ -5228,6 +5379,16 @@ def build_parser() -> argparse.ArgumentParser:
|
|||
warrant_resolve_cmd.add_argument(
|
||||
"--limit", type=int, default=1, help="how many top matches per record (default 1)"
|
||||
)
|
||||
warrant_resolve_cmd.add_argument(
|
||||
"--use-aliases",
|
||||
dest="use_aliases",
|
||||
action="store_true",
|
||||
help=(
|
||||
"look up registered citation + term aliases at resolve "
|
||||
"time (#000041 + #000042); alias-resolved derivations "
|
||||
"carry process_id 'warrant-resolver-v1+alias'"
|
||||
),
|
||||
)
|
||||
warrant_resolve_cmd.set_defaults(func=_cmd_warrant_resolve)
|
||||
|
||||
# ----- unconscious sweep (#000037 §3.1 partial) --------------------------
|
||||
|
|
@ -5264,8 +5425,119 @@ def build_parser() -> argparse.ArgumentParser:
|
|||
"--limit", type=int, default=1,
|
||||
help="how many top matches per record to consider (default 1)",
|
||||
)
|
||||
sweep_cmd.add_argument(
|
||||
"--use-aliases",
|
||||
dest="use_aliases",
|
||||
action="store_true",
|
||||
help="apply citation + term aliases (#000041 + #000042) at resolve time",
|
||||
)
|
||||
sweep_cmd.set_defaults(func=_cmd_sweep)
|
||||
|
||||
# ----- alias subcommands (#000041 + #000042) -----------------------------
|
||||
alias_cmd = sub.add_parser(
|
||||
"alias",
|
||||
help="curated citation + term aliases (audit-disciplined; opt-in)",
|
||||
)
|
||||
alias_sub = alias_cmd.add_subparsers(dest="alias_op", required=True)
|
||||
|
||||
# alias citation {add,list,remove}
|
||||
alias_cit = alias_sub.add_parser(
|
||||
"citation",
|
||||
help="citation aliases (#000041): substitute textbook for proprietary cite",
|
||||
)
|
||||
alias_cit_sub = alias_cit.add_subparsers(dest="alias_cit_op", required=True)
|
||||
|
||||
cit_add = alias_cit_sub.add_parser(
|
||||
"add", help="add a citation alias (refuses without --by)"
|
||||
)
|
||||
cit_add.add_argument("original", help="original source_reference string")
|
||||
cit_add.add_argument(
|
||||
"--substitute", required=True, help="replacement citation string"
|
||||
)
|
||||
cit_add.add_argument(
|
||||
"--author", action="append", default=None,
|
||||
help="author of the substitute work (repeatable)",
|
||||
)
|
||||
cit_add.add_argument(
|
||||
"--title", default="", help="title of the substitute work"
|
||||
)
|
||||
cit_add.add_argument(
|
||||
"--by", required=True,
|
||||
help="who decided this alias (audit field; refuses if empty)",
|
||||
)
|
||||
cit_add.add_argument(
|
||||
"--rationale", default="", help="why this alias is appropriate"
|
||||
)
|
||||
cit_add.add_argument(
|
||||
"--aliases-db", default=None,
|
||||
help="aliases DB path (default: <shards-dir>/000.db)",
|
||||
)
|
||||
cit_add.set_defaults(func=_cmd_alias_citation_add)
|
||||
|
||||
cit_list = alias_cit_sub.add_parser(
|
||||
"list", help="list registered citation aliases"
|
||||
)
|
||||
cit_list.add_argument(
|
||||
"--filter", default=None, help="substring filter on original_ref"
|
||||
)
|
||||
cit_list.add_argument("--aliases-db", default=None)
|
||||
cit_list.set_defaults(func=_cmd_alias_citation_list)
|
||||
|
||||
cit_rm = alias_cit_sub.add_parser(
|
||||
"remove", help="remove a citation alias by (original, substitute)"
|
||||
)
|
||||
cit_rm.add_argument("original")
|
||||
cit_rm.add_argument("--substitute", required=True)
|
||||
cit_rm.add_argument("--aliases-db", default=None)
|
||||
cit_rm.set_defaults(func=_cmd_alias_citation_remove)
|
||||
|
||||
# alias term {add,list,remove}
|
||||
alias_term = alias_sub.add_parser(
|
||||
"term",
|
||||
help="term aliases (#000042): vocabulary bridge for old vs modern words",
|
||||
)
|
||||
alias_term_sub = alias_term.add_subparsers(dest="alias_term_op", required=True)
|
||||
|
||||
term_add = alias_term_sub.add_parser(
|
||||
"add", help="add a term alias (refuses without --by)"
|
||||
)
|
||||
term_add.add_argument("term", help="modern term used in claim-pack records")
|
||||
term_add.add_argument(
|
||||
"alternate", help="historical / foreign / alternate term used in textbook prose"
|
||||
)
|
||||
term_add.add_argument(
|
||||
"--domain", required=True,
|
||||
help="domain string (e.g., 'geometry', 'logic')",
|
||||
)
|
||||
term_add.add_argument(
|
||||
"--by", required=True,
|
||||
help="who decided this alias (audit field; refuses if empty)",
|
||||
)
|
||||
term_add.add_argument(
|
||||
"--rationale", default="", help="why this alias is appropriate"
|
||||
)
|
||||
term_add.add_argument("--aliases-db", default=None)
|
||||
term_add.set_defaults(func=_cmd_alias_term_add)
|
||||
|
||||
term_list = alias_term_sub.add_parser(
|
||||
"list", help="list registered term aliases"
|
||||
)
|
||||
term_list.add_argument("--domain", default=None)
|
||||
term_list.add_argument(
|
||||
"--filter", default=None, help="substring filter on term"
|
||||
)
|
||||
term_list.add_argument("--aliases-db", default=None)
|
||||
term_list.set_defaults(func=_cmd_alias_term_list)
|
||||
|
||||
term_rm = alias_term_sub.add_parser(
|
||||
"remove", help="remove a term alias by (term, alternate, domain)"
|
||||
)
|
||||
term_rm.add_argument("term")
|
||||
term_rm.add_argument("alternate")
|
||||
term_rm.add_argument("--domain", required=True)
|
||||
term_rm.add_argument("--aliases-db", default=None)
|
||||
term_rm.set_defaults(func=_cmd_alias_term_remove)
|
||||
|
||||
# ----- mesh subcommands (off by default) ---------------------------------
|
||||
mesh_cmd = sub.add_parser(
|
||||
"mesh",
|
||||
|
|
|
|||
512
arborist/qa/aliases.py
Normal file
512
arborist/qa/aliases.py
Normal file
|
|
@ -0,0 +1,512 @@
|
|||
"""Citation + term aliases for warrant-resolver query expansion.
|
||||
|
||||
Implements `#000041` (citation_aliases) + `#000042` (term_aliases)
|
||||
as one cohesive helper module. Both tables share schema shape
|
||||
(audit-discipline fields: ``decision_at`` + ``decision_by`` +
|
||||
``decision_rationale``); both are read at warrant-resolve time
|
||||
when the operator opts in via ``--use-aliases``.
|
||||
|
||||
**Citation alias** = "use a different textbook for THIS cite"
|
||||
(e.g., Hilbert-Ackermann 1928 instead of Mendelson 1997 for
|
||||
pillar I logic axioms whose original cite is proprietary).
|
||||
|
||||
**Term alias** = "this textbook uses a different word for THIS
|
||||
concept" (e.g., Hilbert 1902 says "connection" for what modern
|
||||
texts call "incidence" in geometry).
|
||||
|
||||
The two are orthogonal: a single resolution can use both layers
|
||||
(cite Hilbert-Ackermann substitute AND match its "connection"
|
||||
vocabulary against modern claim-pack records that say
|
||||
"incidence").
|
||||
|
||||
Audit discipline: every row carries ``decision_at`` +
|
||||
``decision_by`` + (optional but recommended) ``decision_rationale``.
|
||||
The :func:`add_citation_alias` / :func:`add_term_alias` helpers
|
||||
require ``decision_by`` (raise on empty); silent fabrication of
|
||||
substitutions is not allowed. The CLI surface
|
||||
(``arborist alias citation add ...`` /
|
||||
``arborist alias term add ...``) enforces the same.
|
||||
|
||||
Source: tickets #000041 + #000042.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
import sqlite3
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Iterable, Optional
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Aliases DB resolution
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Aliases live alongside the shards-cluster main DB (or a sibling
|
||||
# dedicated file). For the first iteration we put them in shard 000.db
|
||||
# (where the claim-pack records live) so the resolver only needs to
|
||||
# attach one DB. Future: split into a dedicated file per the design
|
||||
# notes in #000041 §2.1 (TBD via mesh-sync semantics).
|
||||
|
||||
|
||||
def _connect_rw(db_path: str | Path) -> sqlite3.Connection:
|
||||
return sqlite3.connect(str(db_path))
|
||||
|
||||
|
||||
def _connect_ro(db_path: str | Path) -> Optional[sqlite3.Connection]:
|
||||
try:
|
||||
return sqlite3.connect(f"file:{db_path}?mode=ro", uri=True)
|
||||
except sqlite3.OperationalError:
|
||||
return None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Citation aliases
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CitationAlias:
|
||||
original_ref: str
|
||||
substitute_ref: str
|
||||
substitute_authors: tuple[str, ...]
|
||||
substitute_title: str
|
||||
decision_at: int
|
||||
decision_by: str
|
||||
decision_rationale: str = ""
|
||||
|
||||
|
||||
def add_citation_alias(
|
||||
db_path: str | Path,
|
||||
*,
|
||||
original_ref: str,
|
||||
substitute_ref: str,
|
||||
substitute_authors: Iterable[str],
|
||||
substitute_title: str,
|
||||
decision_by: str,
|
||||
decision_rationale: str = "",
|
||||
decision_at: Optional[int] = None,
|
||||
) -> bool:
|
||||
"""Insert a citation alias row. Returns True if newly inserted,
|
||||
False if (original, substitute) already existed.
|
||||
|
||||
Raises ValueError on empty `decision_by` or `original_ref` or
|
||||
`substitute_ref`. Audit discipline at the API surface — fail
|
||||
closed; silent substitution is not allowed.
|
||||
"""
|
||||
if not original_ref.strip():
|
||||
raise ValueError("original_ref must be non-empty")
|
||||
if not substitute_ref.strip():
|
||||
raise ValueError("substitute_ref must be non-empty")
|
||||
if not decision_by.strip():
|
||||
raise ValueError("decision_by must be non-empty (audit discipline)")
|
||||
|
||||
if decision_at is None:
|
||||
decision_at = int(time.time())
|
||||
authors_json = json.dumps(list(substitute_authors), ensure_ascii=False)
|
||||
conn = _connect_rw(db_path)
|
||||
try:
|
||||
cur = conn.execute(
|
||||
"INSERT OR IGNORE INTO citation_aliases "
|
||||
"(original_ref, substitute_ref, substitute_authors, "
|
||||
" substitute_title, decision_at, decision_by, "
|
||||
" decision_rationale) "
|
||||
"VALUES (?, ?, ?, ?, ?, ?, ?)",
|
||||
(
|
||||
original_ref.strip(),
|
||||
substitute_ref.strip(),
|
||||
authors_json,
|
||||
substitute_title.strip(),
|
||||
decision_at,
|
||||
decision_by.strip(),
|
||||
decision_rationale.strip(),
|
||||
),
|
||||
)
|
||||
conn.commit()
|
||||
return cur.rowcount > 0
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def list_citation_aliases(
|
||||
db_path: str | Path,
|
||||
*,
|
||||
original_filter: Optional[str] = None,
|
||||
) -> list[CitationAlias]:
|
||||
"""Return all citation aliases (or those whose original_ref
|
||||
matches the optional substring filter)."""
|
||||
conn = _connect_ro(db_path)
|
||||
if conn is None:
|
||||
return []
|
||||
try:
|
||||
if original_filter:
|
||||
rows = conn.execute(
|
||||
"SELECT * FROM citation_aliases "
|
||||
"WHERE original_ref LIKE ? "
|
||||
"ORDER BY decision_at DESC",
|
||||
(f"%{original_filter}%",),
|
||||
).fetchall()
|
||||
else:
|
||||
rows = conn.execute(
|
||||
"SELECT * FROM citation_aliases ORDER BY decision_at DESC"
|
||||
).fetchall()
|
||||
except sqlite3.OperationalError:
|
||||
return []
|
||||
finally:
|
||||
conn.close()
|
||||
out: list[CitationAlias] = []
|
||||
for r in rows:
|
||||
try:
|
||||
authors = tuple(json.loads(r[2]) or [])
|
||||
except (TypeError, ValueError):
|
||||
authors = ()
|
||||
out.append(
|
||||
CitationAlias(
|
||||
original_ref=r[0],
|
||||
substitute_ref=r[1],
|
||||
substitute_authors=authors,
|
||||
substitute_title=r[3],
|
||||
decision_at=r[4],
|
||||
decision_by=r[5],
|
||||
decision_rationale=r[6] or "",
|
||||
)
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
def remove_citation_alias(
|
||||
db_path: str | Path, original_ref: str, substitute_ref: str
|
||||
) -> bool:
|
||||
conn = _connect_rw(db_path)
|
||||
try:
|
||||
cur = conn.execute(
|
||||
"DELETE FROM citation_aliases "
|
||||
"WHERE original_ref = ? AND substitute_ref = ?",
|
||||
(original_ref.strip(), substitute_ref.strip()),
|
||||
)
|
||||
conn.commit()
|
||||
return cur.rowcount > 0
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def lookup_citation_aliases(
|
||||
db_path: str | Path, original_ref: str
|
||||
) -> list[CitationAlias]:
|
||||
"""Return the registered substitutes for an original citation.
|
||||
Used by the warrant resolver when `--use-aliases` is set."""
|
||||
conn = _connect_ro(db_path)
|
||||
if conn is None:
|
||||
return []
|
||||
try:
|
||||
rows = conn.execute(
|
||||
"SELECT * FROM citation_aliases WHERE original_ref = ?",
|
||||
(original_ref.strip(),),
|
||||
).fetchall()
|
||||
except sqlite3.OperationalError:
|
||||
return []
|
||||
finally:
|
||||
conn.close()
|
||||
out: list[CitationAlias] = []
|
||||
for r in rows:
|
||||
try:
|
||||
authors = tuple(json.loads(r[2]) or [])
|
||||
except (TypeError, ValueError):
|
||||
authors = ()
|
||||
out.append(
|
||||
CitationAlias(
|
||||
original_ref=r[0],
|
||||
substitute_ref=r[1],
|
||||
substitute_authors=authors,
|
||||
substitute_title=r[3],
|
||||
decision_at=r[4],
|
||||
decision_by=r[5],
|
||||
decision_rationale=r[6] or "",
|
||||
)
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Term aliases
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TermAlias:
|
||||
term: str
|
||||
alternate_term: str
|
||||
domain: str
|
||||
decision_at: int
|
||||
decision_by: str
|
||||
decision_rationale: str = ""
|
||||
|
||||
|
||||
def add_term_alias(
|
||||
db_path: str | Path,
|
||||
*,
|
||||
term: str,
|
||||
alternate_term: str,
|
||||
domain: str,
|
||||
decision_by: str,
|
||||
decision_rationale: str = "",
|
||||
decision_at: Optional[int] = None,
|
||||
) -> bool:
|
||||
"""Insert a term alias row. Returns True if newly inserted, False
|
||||
on PK collision. Raises ValueError on empty audit fields."""
|
||||
if not term.strip():
|
||||
raise ValueError("term must be non-empty")
|
||||
if not alternate_term.strip():
|
||||
raise ValueError("alternate_term must be non-empty")
|
||||
if not domain.strip():
|
||||
raise ValueError("domain must be non-empty")
|
||||
if not decision_by.strip():
|
||||
raise ValueError("decision_by must be non-empty (audit discipline)")
|
||||
if decision_at is None:
|
||||
decision_at = int(time.time())
|
||||
conn = _connect_rw(db_path)
|
||||
try:
|
||||
cur = conn.execute(
|
||||
"INSERT OR IGNORE INTO term_aliases "
|
||||
"(term, alternate_term, domain, decision_at, "
|
||||
" decision_by, decision_rationale) "
|
||||
"VALUES (?, ?, ?, ?, ?, ?)",
|
||||
(
|
||||
term.strip().lower(),
|
||||
alternate_term.strip().lower(),
|
||||
domain.strip().lower(),
|
||||
decision_at,
|
||||
decision_by.strip(),
|
||||
decision_rationale.strip(),
|
||||
),
|
||||
)
|
||||
conn.commit()
|
||||
return cur.rowcount > 0
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def list_term_aliases(
|
||||
db_path: str | Path,
|
||||
*,
|
||||
domain: Optional[str] = None,
|
||||
term_filter: Optional[str] = None,
|
||||
) -> list[TermAlias]:
|
||||
"""Return all term aliases (optionally filtered by domain or
|
||||
a substring match on the term)."""
|
||||
conn = _connect_ro(db_path)
|
||||
if conn is None:
|
||||
return []
|
||||
try:
|
||||
clauses: list[str] = []
|
||||
params: list = []
|
||||
if domain:
|
||||
clauses.append("domain = ?")
|
||||
params.append(domain.strip().lower())
|
||||
if term_filter:
|
||||
clauses.append("term LIKE ?")
|
||||
params.append(f"%{term_filter.strip().lower()}%")
|
||||
where = (" WHERE " + " AND ".join(clauses)) if clauses else ""
|
||||
rows = conn.execute(
|
||||
f"SELECT * FROM term_aliases{where} ORDER BY decision_at DESC",
|
||||
params,
|
||||
).fetchall()
|
||||
except sqlite3.OperationalError:
|
||||
return []
|
||||
finally:
|
||||
conn.close()
|
||||
return [
|
||||
TermAlias(
|
||||
term=r[0],
|
||||
alternate_term=r[1],
|
||||
domain=r[2],
|
||||
decision_at=r[3],
|
||||
decision_by=r[4],
|
||||
decision_rationale=r[5] or "",
|
||||
)
|
||||
for r in rows
|
||||
]
|
||||
|
||||
|
||||
def remove_term_alias(
|
||||
db_path: str | Path, term: str, alternate_term: str, domain: str
|
||||
) -> bool:
|
||||
conn = _connect_rw(db_path)
|
||||
try:
|
||||
cur = conn.execute(
|
||||
"DELETE FROM term_aliases "
|
||||
"WHERE term = ? AND alternate_term = ? AND domain = ?",
|
||||
(
|
||||
term.strip().lower(),
|
||||
alternate_term.strip().lower(),
|
||||
domain.strip().lower(),
|
||||
),
|
||||
)
|
||||
conn.commit()
|
||||
return cur.rowcount > 0
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def lookup_term_aliases(
|
||||
db_path: str | Path, term: str, domain: str
|
||||
) -> list[str]:
|
||||
"""Return the registered alternate-terms for `(term, domain)`.
|
||||
Used by the warrant resolver to OR-expand FTS5 queries.
|
||||
Bidirectional: also searches by alternate_term so a registered
|
||||
`(incidence, connection, geometry)` row also returns
|
||||
"incidence" when looking up `("connection", "geometry")`.
|
||||
"""
|
||||
conn = _connect_ro(db_path)
|
||||
if conn is None:
|
||||
return []
|
||||
try:
|
||||
rows = conn.execute(
|
||||
"SELECT alternate_term FROM term_aliases "
|
||||
"WHERE term = ? AND domain = ? "
|
||||
"UNION "
|
||||
"SELECT term FROM term_aliases "
|
||||
"WHERE alternate_term = ? AND domain = ?",
|
||||
(
|
||||
term.strip().lower(),
|
||||
domain.strip().lower(),
|
||||
term.strip().lower(),
|
||||
domain.strip().lower(),
|
||||
),
|
||||
).fetchall()
|
||||
except sqlite3.OperationalError:
|
||||
return []
|
||||
finally:
|
||||
conn.close()
|
||||
return [r[0] for r in rows if r[0]]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Pillar → domain mapping
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
_PILLAR_DOMAIN = {
|
||||
"I": "logic",
|
||||
"II": "set-theory",
|
||||
"III": "arithmetic",
|
||||
"IV": "geometry",
|
||||
"V": "probability",
|
||||
"VI": "classical-physics",
|
||||
"VII": "combinatorics",
|
||||
"IX": "lambda-calculus",
|
||||
}
|
||||
|
||||
|
||||
def domain_for_pillar(pillar: str) -> str:
|
||||
"""Map a Roman-numeral pillar identifier to its domain string."""
|
||||
return _PILLAR_DOMAIN.get(pillar.upper().strip(), "")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Query expansion (used by warrant resolver)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def expand_query_with_term_aliases(
|
||||
fts5_query: str, domain: str, db_path: str | Path
|
||||
) -> str:
|
||||
"""Replace each significant token in a FTS5 query with its
|
||||
`(token OR alt1 OR alt2)` expansion when (token, domain) has
|
||||
registered aliases.
|
||||
|
||||
Preserves phrase syntax (`"foo bar"` becomes
|
||||
`("foo bar" OR "alt1 alt2")` — the alias for the multi-word
|
||||
phrase is looked up separately if registered). Operators on
|
||||
the query (`AND`, `OR`, `NOT`, parens) pass through unchanged.
|
||||
|
||||
Returns the original query unchanged when no aliases apply or
|
||||
when the aliases DB is unreachable.
|
||||
"""
|
||||
if not fts5_query.strip() or not domain.strip():
|
||||
return fts5_query
|
||||
|
||||
# Split on whitespace, preserving quoted phrases as single tokens.
|
||||
parts = _tokenize_fts_query(fts5_query)
|
||||
if not parts:
|
||||
return fts5_query
|
||||
|
||||
out: list[str] = []
|
||||
for part in parts:
|
||||
# Preserve operators / parens unchanged.
|
||||
if part.upper() in ("AND", "OR", "NOT") or part in ("(", ")"):
|
||||
out.append(part)
|
||||
continue
|
||||
# Phrase tokens (`"foo bar"`) — lookup the whole phrase;
|
||||
# otherwise treat as a single bare token.
|
||||
bare = part.strip().lower().strip('"')
|
||||
if not bare or len(bare) < 3:
|
||||
out.append(part)
|
||||
continue
|
||||
alts = lookup_term_aliases(db_path, bare, domain)
|
||||
if not alts:
|
||||
out.append(part)
|
||||
continue
|
||||
# Build the OR-expansion. Quote the parts so multi-word
|
||||
# phrases survive FTS5 parsing.
|
||||
expanded = [_quote_for_fts(part)] + [
|
||||
_quote_for_fts(_match_quoting(part, a)) for a in alts
|
||||
]
|
||||
out.append("(" + " OR ".join(expanded) + ")")
|
||||
return " ".join(out)
|
||||
|
||||
|
||||
_QUOTED_PHRASE_RE = re.compile(r'"[^"]*"')
|
||||
|
||||
|
||||
def _tokenize_fts_query(q: str) -> list[str]:
|
||||
"""Naive tokenizer that preserves quoted phrases as single tokens.
|
||||
Whitespace-split everything else."""
|
||||
out: list[str] = []
|
||||
i = 0
|
||||
while i < len(q):
|
||||
c = q[i]
|
||||
if c == '"':
|
||||
end = q.find('"', i + 1)
|
||||
if end == -1:
|
||||
# unterminated quote — fall back to treating the rest
|
||||
# as one token so we don't corrupt the user's query.
|
||||
out.append(q[i:])
|
||||
return out
|
||||
out.append(q[i : end + 1])
|
||||
i = end + 1
|
||||
elif c.isspace():
|
||||
i += 1
|
||||
elif c in "()":
|
||||
out.append(c)
|
||||
i += 1
|
||||
else:
|
||||
j = i
|
||||
while j < len(q) and not q[j].isspace() and q[j] not in '()"':
|
||||
j += 1
|
||||
out.append(q[i:j])
|
||||
i = j
|
||||
return out
|
||||
|
||||
|
||||
def _quote_for_fts(token: str) -> str:
|
||||
"""Pass through already-quoted phrases; quote bare multi-word
|
||||
tokens (which shouldn't exist in our tokenizer's output, but
|
||||
defense in depth)."""
|
||||
if token.startswith('"') and token.endswith('"'):
|
||||
return token
|
||||
if " " in token:
|
||||
return f'"{token}"'
|
||||
return token
|
||||
|
||||
|
||||
def _match_quoting(reference: str, alt: str) -> str:
|
||||
"""If `reference` is a quoted phrase, return alt as a quoted
|
||||
phrase; otherwise return alt bare. Preserves the original
|
||||
query's matching style."""
|
||||
if reference.startswith('"') and reference.endswith('"'):
|
||||
return f'"{alt}"'
|
||||
return alt
|
||||
|
|
@ -201,7 +201,16 @@ def _strip_metadata(text: str) -> str:
|
|||
|
||||
@dataclass
|
||||
class ResolutionMatch:
|
||||
"""One resolved (citation → surface chunk) candidate."""
|
||||
"""One resolved (citation → surface chunk) candidate.
|
||||
|
||||
``via_alias`` flags matches that REQUIRED a registered citation
|
||||
or term alias to be found (#000041 + #000042). The unaliased
|
||||
cascade tries first; if it produces no match, the aliased
|
||||
cascade is tried, and its matches carry ``via_alias=True``.
|
||||
Used by ``warrant_resolve`` to pick the right ``process_id``
|
||||
for the derivations row (`warrant-resolver-v1+alias` for
|
||||
via_alias matches, `warrant-resolver-v1` otherwise).
|
||||
"""
|
||||
|
||||
citation: Citation
|
||||
shard_path: str
|
||||
|
|
@ -212,6 +221,7 @@ class ResolutionMatch:
|
|||
chunk_idx: int
|
||||
score: float
|
||||
snippet: str
|
||||
via_alias: bool = False
|
||||
|
||||
|
||||
# Per-shard cached set of (lowercased title tokens). Reading the
|
||||
|
|
@ -569,6 +579,8 @@ def resolve_chunks(
|
|||
theorem_name: str = "",
|
||||
record_content: str = "",
|
||||
limit: int = 5,
|
||||
aliases_db: Path | str | None = None,
|
||||
domain: str = "",
|
||||
) -> list[ResolutionMatch]:
|
||||
"""Search every surface shard under ``shards_dir`` for chunks that
|
||||
match the citation + theorem-name signal. Returns a ranked list.
|
||||
|
|
@ -602,18 +614,39 @@ def resolve_chunks(
|
|||
continue
|
||||
candidate_shards.append(db)
|
||||
|
||||
queries = _build_record_query_cascade(
|
||||
base_queries = _build_record_query_cascade(
|
||||
citation, theorem_name=theorem_name, record_content=record_content
|
||||
)
|
||||
if not queries:
|
||||
if not base_queries:
|
||||
return []
|
||||
|
||||
# Two-pass cascade: try the UNALIASED queries first (matches
|
||||
# carry via_alias=False); only fall through to the alias-
|
||||
# expanded queries when the unaliased cascade produces nothing
|
||||
# for a given shard. Term-alias OR-expansion (#000042) preserves
|
||||
# phrase syntax.
|
||||
aliased_queries: list[str] = []
|
||||
if aliases_db and domain:
|
||||
try:
|
||||
from arborist.qa.aliases import expand_query_with_term_aliases
|
||||
|
||||
for q in base_queries:
|
||||
expanded = expand_query_with_term_aliases(q, domain, aliases_db)
|
||||
# Only add to the alias-cascade if the expansion is
|
||||
# actually different from the original — otherwise
|
||||
# we're just re-running the same query.
|
||||
if expanded != q:
|
||||
aliased_queries.append(expanded)
|
||||
except Exception:
|
||||
aliased_queries = []
|
||||
|
||||
matches: list[ResolutionMatch] = []
|
||||
for shard in candidate_shards:
|
||||
if not _shard_matches_citation(str(shard), citation):
|
||||
continue
|
||||
# Try queries in cascade order; first to yield rows wins.
|
||||
for q in queries:
|
||||
# Pass 1: unaliased cascade.
|
||||
hit = False
|
||||
for q in base_queries:
|
||||
rows = _fts5_search(str(shard), q, limit=limit)
|
||||
if rows:
|
||||
for chunk_id, idx, doc_root, doc_uri, title, snippet, rank in rows:
|
||||
|
|
@ -628,6 +661,31 @@ def resolve_chunks(
|
|||
chunk_idx=idx,
|
||||
score=-float(rank or 0.0),
|
||||
snippet=snippet or "",
|
||||
via_alias=False,
|
||||
)
|
||||
)
|
||||
hit = True
|
||||
break
|
||||
if hit:
|
||||
continue
|
||||
# Pass 2: alias-expanded cascade (only when unaliased found
|
||||
# nothing). Matches here carry via_alias=True.
|
||||
for q in aliased_queries:
|
||||
rows = _fts5_search(str(shard), q, limit=limit)
|
||||
if rows:
|
||||
for chunk_id, idx, doc_root, doc_uri, title, snippet, rank in rows:
|
||||
matches.append(
|
||||
ResolutionMatch(
|
||||
citation=citation,
|
||||
shard_path=str(shard),
|
||||
document_root=doc_root,
|
||||
document_uri=doc_uri,
|
||||
document_title=title or "",
|
||||
chunk_id=chunk_id,
|
||||
chunk_idx=idx,
|
||||
score=-float(rank or 0.0),
|
||||
snippet=snippet or "",
|
||||
via_alias=True,
|
||||
)
|
||||
)
|
||||
break
|
||||
|
|
@ -811,15 +869,20 @@ def _decompress_chunk(content) -> str:
|
|||
|
||||
def iter_claim_pack_records(
|
||||
shards_dir: Path | str,
|
||||
) -> Iterator[tuple[str, str, str, str, str]]:
|
||||
) -> Iterator[tuple[str, str, str, str, str, str]]:
|
||||
"""Yield ``(shard_path, record_root, title, source_reference,
|
||||
chunk_content)`` for every claim_pack record across the shards-dir.
|
||||
chunk_content, pillar)`` for every claim_pack record across the
|
||||
shards-dir.
|
||||
|
||||
Phase 5 (#000039) added the ``chunk_content`` element so the
|
||||
Phase 5 (#000040) added the ``chunk_content`` element so the
|
||||
warrant resolver can use the record's ∇verbose body as a
|
||||
content-token signal — the title alone often shares
|
||||
ubiquitous-in-the-textbook tokens; the prose has more
|
||||
discriminating language.
|
||||
content-token signal.
|
||||
|
||||
#000042 added ``pillar`` (Roman numeral I-VII / IX) so the
|
||||
resolver can pick the right domain when looking up term aliases
|
||||
(pillar IV → "geometry"; pillar I → "logic"; etc.). Pillar is
|
||||
parsed from the doc URI's `/pillar/<P>/` segment per the
|
||||
claim_pack source's URI scheme.
|
||||
"""
|
||||
shards_dir = Path(shards_dir).expanduser()
|
||||
for db in sorted(shards_dir.glob("*.db")):
|
||||
|
|
@ -831,20 +894,34 @@ def iter_claim_pack_records(
|
|||
continue
|
||||
try:
|
||||
for row in conn.execute(
|
||||
"SELECT d.document_root, d.title, c.content "
|
||||
"SELECT d.document_root, d.title, d.document_uri, c.content "
|
||||
"FROM documents d JOIN chunks c ON c.document_root = d.document_root "
|
||||
"WHERE d.source_type = 'claim_pack' "
|
||||
"ORDER BY d.ingest_ts"
|
||||
):
|
||||
doc_root, title, content = row
|
||||
doc_root, title, doc_uri, content = row
|
||||
text = _decompress_chunk(content)
|
||||
m = _SOURCE_RE.search(text)
|
||||
source_ref = m.group(1).strip() if m else ""
|
||||
yield (str(db), doc_root, title or "", source_ref, text)
|
||||
pillar = _pillar_from_uri(doc_uri or "")
|
||||
yield (
|
||||
str(db), doc_root, title or "", source_ref, text, pillar
|
||||
)
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
_PILLAR_URI_RE = re.compile(r"/pillar/([IVX]+)/", re.IGNORECASE)
|
||||
|
||||
|
||||
def _pillar_from_uri(uri: str) -> str:
|
||||
"""Pull the pillar Roman numeral from a claim-pack document URI.
|
||||
URIs follow the scheme
|
||||
`claim-pack://<bundle>/pillar/<P>/{axioms|theorems}/...`."""
|
||||
m = _PILLAR_URI_RE.search(uri)
|
||||
return m.group(1).upper() if m else ""
|
||||
|
||||
|
||||
def _has_derivation(db_path: str, record_root: str) -> bool:
|
||||
try:
|
||||
conn = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True)
|
||||
|
|
@ -946,15 +1023,54 @@ class WarrantStatus:
|
|||
has_derivation: bool
|
||||
|
||||
|
||||
def warrant_status(shards_dir: Path | str, limit: int = 5) -> list[WarrantStatus]:
|
||||
def warrant_status(
|
||||
shards_dir: Path | str,
|
||||
limit: int = 5,
|
||||
use_aliases: bool = False,
|
||||
) -> list[WarrantStatus]:
|
||||
"""Walk every claim-pack record and report what surface chunks the
|
||||
citation resolver finds. Read-only — no DB writes.
|
||||
|
||||
``use_aliases`` (#000041 + #000042) — when True, the resolver
|
||||
looks up registered citation + term aliases stored in the
|
||||
shards' main DB and uses them to expand the FTS5 query +
|
||||
propose substitute citations. Default False to preserve the
|
||||
original strict behavior.
|
||||
"""
|
||||
out: list[WarrantStatus] = []
|
||||
for shard_path, record_root, title, source_ref, content in iter_claim_pack_records(
|
||||
shards_dir
|
||||
aliases_db = (
|
||||
str(Path(shards_dir).expanduser() / "000.db") if use_aliases else None
|
||||
)
|
||||
for shard_path, record_root, title, source_ref, content, pillar in (
|
||||
iter_claim_pack_records(shards_dir)
|
||||
):
|
||||
citations = parse_citation(source_ref)
|
||||
# #000041 — citation-alias substitution.
|
||||
if use_aliases and aliases_db and source_ref:
|
||||
try:
|
||||
from arborist.qa.aliases import lookup_citation_aliases
|
||||
|
||||
for sub in lookup_citation_aliases(aliases_db, source_ref):
|
||||
citations.append(
|
||||
Citation(
|
||||
title=sub.substitute_title,
|
||||
authors=sub.substitute_authors,
|
||||
raw=sub.substitute_ref,
|
||||
)
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Map pillar → domain for term-alias lookups.
|
||||
domain = ""
|
||||
if use_aliases and pillar:
|
||||
try:
|
||||
from arborist.qa.aliases import domain_for_pillar
|
||||
|
||||
domain = domain_for_pillar(pillar)
|
||||
except Exception:
|
||||
domain = ""
|
||||
|
||||
all_matches: list[ResolutionMatch] = []
|
||||
for c in citations:
|
||||
all_matches.extend(
|
||||
|
|
@ -964,6 +1080,8 @@ def warrant_status(shards_dir: Path | str, limit: int = 5) -> list[WarrantStatus
|
|||
theorem_name=title,
|
||||
record_content=content,
|
||||
limit=limit,
|
||||
aliases_db=aliases_db,
|
||||
domain=domain,
|
||||
)
|
||||
)
|
||||
all_matches.sort(key=lambda m: m.score, reverse=True)
|
||||
|
|
@ -980,24 +1098,35 @@ def warrant_status(shards_dir: Path | str, limit: int = 5) -> list[WarrantStatus
|
|||
return out
|
||||
|
||||
|
||||
WARRANT_PROCESS_ID_ALIASED = "warrant-resolver-v1+alias"
|
||||
|
||||
|
||||
def warrant_resolve(
|
||||
shards_dir: Path | str,
|
||||
write: bool = False,
|
||||
limit: int = 1,
|
||||
use_aliases: bool = False,
|
||||
) -> dict:
|
||||
"""Run the resolver across every claim-pack record. With
|
||||
``write=False`` (default) returns a dry-run summary. With
|
||||
``write=True``, computes Merkle inclusion proofs for top-scoring
|
||||
matches and writes ``derivations`` rows.
|
||||
|
||||
When ``use_aliases=True``, citation + term aliases (#000041 +
|
||||
#000042) get applied: query expansion via OR-joined alternate
|
||||
terms, and substitute citations alongside originals. Alias-
|
||||
resolved derivations carry ``process_id =
|
||||
"warrant-resolver-v1+alias"`` so audit can tell substituted
|
||||
chains from original ones.
|
||||
"""
|
||||
records_total = 0
|
||||
records_resolved = 0
|
||||
derivations_written = 0
|
||||
statuses = warrant_status(shards_dir, limit=limit)
|
||||
statuses = warrant_status(shards_dir, limit=limit, use_aliases=use_aliases)
|
||||
# Build a record_root → shard_path map once so we don't iterate
|
||||
# again per record.
|
||||
record_shards: dict[str, str] = {}
|
||||
for shard_path, record_root, _, _, _ in iter_claim_pack_records(shards_dir):
|
||||
for shard_path, record_root, _, _, _, _ in iter_claim_pack_records(shards_dir):
|
||||
record_shards[record_root] = shard_path
|
||||
|
||||
for status in statuses:
|
||||
|
|
@ -1012,8 +1141,21 @@ def warrant_resolve(
|
|||
if proof is None:
|
||||
continue
|
||||
host_shard = record_shards.get(status.record_root)
|
||||
# The match itself carries the alias flag — set when the
|
||||
# unaliased cascade missed and the alias-expanded cascade
|
||||
# produced the hit. This is honest: alias-resolved chains
|
||||
# only get the +alias process_id when the alias was
|
||||
# actually load-bearing for the match.
|
||||
process_id = (
|
||||
WARRANT_PROCESS_ID_ALIASED if top.via_alias
|
||||
else WARRANT_PROCESS_ID
|
||||
)
|
||||
if host_shard and write_derivation(
|
||||
host_shard, status.record_root, top.document_root, proof
|
||||
host_shard,
|
||||
status.record_root,
|
||||
top.document_root,
|
||||
proof,
|
||||
process_id=process_id,
|
||||
):
|
||||
derivations_written += 1
|
||||
return {
|
||||
|
|
@ -1021,4 +1163,5 @@ def warrant_resolve(
|
|||
"records_resolved": records_resolved,
|
||||
"derivations_written": derivations_written,
|
||||
"wrote": write,
|
||||
"use_aliases": use_aliases,
|
||||
}
|
||||
|
|
|
|||
|
|
@ -463,6 +463,51 @@ CREATE INDEX IF NOT EXISTS idx_adapter_loss_kind
|
|||
ON adapter_loss_reports(loss_kind, stage);
|
||||
CREATE INDEX IF NOT EXISTS idx_adapter_loss_chunk
|
||||
ON adapter_loss_reports(chunk_id, stage);
|
||||
|
||||
-- Citation aliases (#000041): map a claim-pack record's original
|
||||
-- source_reference string to a substitute citation when the original
|
||||
-- text isn't ingestable (proprietary / unavailable) but a comparable
|
||||
-- PD work is. Read at warrant-resolve time when --use-aliases is
|
||||
-- passed; alias-resolved derivations carry process_id
|
||||
-- "warrant-resolver-v1+alias" so the audit trail tells substituted
|
||||
-- chains from original ones.
|
||||
--
|
||||
-- Audit discipline: decision_at + decision_by + decision_rationale
|
||||
-- are NOT NULL. The CLI refuses to add rows without those fields.
|
||||
-- "Honest substitute, not silent fabrication."
|
||||
CREATE TABLE IF NOT EXISTS citation_aliases (
|
||||
original_ref TEXT NOT NULL,
|
||||
substitute_ref TEXT NOT NULL,
|
||||
substitute_authors TEXT NOT NULL, -- JSON array of strings
|
||||
substitute_title TEXT NOT NULL,
|
||||
decision_at INTEGER NOT NULL,
|
||||
decision_by TEXT NOT NULL,
|
||||
decision_rationale TEXT,
|
||||
PRIMARY KEY (original_ref, substitute_ref)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_citation_alias_orig
|
||||
ON citation_aliases(original_ref);
|
||||
|
||||
-- Term aliases (#000042): bridge vocabulary mismatches between a
|
||||
-- claim-pack record's modern term and a textbook's historical /
|
||||
-- foreign-language / pre-modern term for the same concept. E.g.,
|
||||
-- modern "incidence" ↔ Hilbert-1902-translation "connection" in
|
||||
-- the geometry domain. Read at warrant-resolve time when
|
||||
-- --use-aliases is passed; query expansion replaces a token with
|
||||
-- (token OR alt) so FTS5 matches either vocabulary.
|
||||
--
|
||||
-- Same audit discipline as citation_aliases.
|
||||
CREATE TABLE IF NOT EXISTS term_aliases (
|
||||
term TEXT NOT NULL,
|
||||
alternate_term TEXT NOT NULL,
|
||||
domain TEXT NOT NULL, -- e.g., "geometry", "logic"
|
||||
decision_at INTEGER NOT NULL,
|
||||
decision_by TEXT NOT NULL,
|
||||
decision_rationale TEXT,
|
||||
PRIMARY KEY (term, alternate_term, domain)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_term_alias_term_domain
|
||||
ON term_aliases(term, domain);
|
||||
"""
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -61,8 +61,8 @@ Newest first. Update on every open/close.
|
|||
|
||||
| ID | Title | Status | Opened | Directive |
|
||||
|----------|------------------------------------------------|-----------------------|------------|-----------|
|
||||
| #000042 | Term-aliases table (vocabulary-mismatch bridge) | open · awaiting first term-alias decision; trigger candidate: incidence ↔ connection (geometry) for Hilbert 1902 | 2026-05-09 | — |
|
||||
| #000041 | Citation-aliases table (PD substitutes for proprietary cites) | open · awaiting first PD-substitute decision from #000038 | 2026-05-09 | — |
|
||||
| #000042 | Term-aliases table (vocabulary-mismatch bridge) | closed · landed 2026-05-09 (smoke-test alias incidence↔connection geometry resolves 4 stuck Hilbert records) | 2026-05-09 | — |
|
||||
| #000041 | Citation-aliases table (PD substitutes for proprietary cites) | closed · mechanism landed 2026-05-09; rows added via fox decisions per #000038 | 2026-05-09 | — |
|
||||
| #000040 | Phase 5 resolver fix — phrase + content-token cascade (Hilbert terminology mismatch surfaced) | closed · cascade landed 2026-05-09; lift blocked by 1902-vs-modern vocab; follow-up #000042 | 2026-05-09 | — |
|
||||
| #000039 | Optional `sqlite-vec` retrieval backend (A/B vs FTS5, hybrid not replacement) | open · awaiting go/no-go (doc-only Phase 0) | 2026-05-09 | — |
|
||||
| #000038 | Phase 4 content acquisition — proprietary textbook license decisions for warrant coverage | open · awaiting go/no-go | 2026-05-09 | — |
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Ticket #000041 — Citation-aliases table
|
||||
|
||||
**Status:** open · awaiting go/no-go (depends on #000038 PD-substitute decisions)
|
||||
**Status:** closed · mechanism landed 2026-05-09; populated by fox decisions per #000038
|
||||
**Opened:** 2026-05-09
|
||||
**Scope:** Add an `arborist citation_aliases` table that maps
|
||||
a claim-pack record's original `source_reference` string to a
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Ticket #000042 — Term-aliases table (vocabulary-mismatch bridge)
|
||||
|
||||
**Status:** open · awaiting go/no-go (depends on #000038 + #000040 §6 surfacing concrete need-cases)
|
||||
**Status:** closed · mechanism landed 2026-05-09 with smoke-test alias (incidence ↔ connection geometry) — 4 stuck Hilbert records now resolve via alias-expanded cascade
|
||||
**Opened:** 2026-05-09
|
||||
**Scope:** Add an `arborist term_aliases` table that maps a
|
||||
`(term, domain)` pair to an alternate term used in older /
|
||||
|
|
|
|||
257
tests/test_aliases.py
Normal file
257
tests/test_aliases.py
Normal file
|
|
@ -0,0 +1,257 @@
|
|||
"""Tests for the citation + term alias mechanisms (#000041 + #000042).
|
||||
|
||||
Pure unit + small-DB tests over the helpers in arborist/qa/aliases.py.
|
||||
The end-to-end resolver-uses-alias path is exercised separately via
|
||||
the `arborist warrant-resolve --use-aliases` CLI on real shards.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlite3
|
||||
|
||||
import pytest
|
||||
|
||||
from arborist.qa.aliases import (
|
||||
add_citation_alias,
|
||||
add_term_alias,
|
||||
domain_for_pillar,
|
||||
expand_query_with_term_aliases,
|
||||
list_citation_aliases,
|
||||
list_term_aliases,
|
||||
lookup_citation_aliases,
|
||||
lookup_term_aliases,
|
||||
remove_citation_alias,
|
||||
remove_term_alias,
|
||||
)
|
||||
from arborist.store import SCHEMA_SQL
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def db(tmp_path):
|
||||
"""Fresh SQLite DB with the alias schema applied."""
|
||||
p = tmp_path / "aliases.db"
|
||||
conn = sqlite3.connect(str(p))
|
||||
try:
|
||||
conn.executescript(SCHEMA_SQL)
|
||||
conn.commit()
|
||||
finally:
|
||||
conn.close()
|
||||
return str(p)
|
||||
|
||||
|
||||
# --- pillar domain mapping -------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"pillar,expected",
|
||||
[
|
||||
("I", "logic"),
|
||||
("IV", "geometry"),
|
||||
("VII", "combinatorics"),
|
||||
("IX", "lambda-calculus"),
|
||||
("UNKNOWN", ""),
|
||||
("", ""),
|
||||
],
|
||||
)
|
||||
def test_domain_for_pillar(pillar, expected):
|
||||
assert domain_for_pillar(pillar) == expected
|
||||
|
||||
|
||||
# --- citation aliases -----------------------------------------------
|
||||
|
||||
|
||||
def test_citation_alias_add_lookup_remove(db):
|
||||
inserted = add_citation_alias(
|
||||
db,
|
||||
original_ref="Introduction to Mathematical Logic by Elliott Mendelson",
|
||||
substitute_ref="Principles of Mathematical Logic by Hilbert and Ackermann",
|
||||
substitute_authors=["David Hilbert", "Wilhelm Ackermann"],
|
||||
substitute_title="Principles of Mathematical Logic",
|
||||
decision_by="fox 2026-05-09",
|
||||
decision_rationale="Hilbert-Ackermann 1928 is PD; covers same propositional axioms",
|
||||
)
|
||||
assert inserted is True
|
||||
|
||||
aliases = lookup_citation_aliases(
|
||||
db, "Introduction to Mathematical Logic by Elliott Mendelson"
|
||||
)
|
||||
assert len(aliases) == 1
|
||||
a = aliases[0]
|
||||
assert a.substitute_title == "Principles of Mathematical Logic"
|
||||
assert "David Hilbert" in a.substitute_authors
|
||||
|
||||
# Idempotent — second add returns False.
|
||||
inserted_again = add_citation_alias(
|
||||
db,
|
||||
original_ref="Introduction to Mathematical Logic by Elliott Mendelson",
|
||||
substitute_ref="Principles of Mathematical Logic by Hilbert and Ackermann",
|
||||
substitute_authors=["David Hilbert", "Wilhelm Ackermann"],
|
||||
substitute_title="Principles of Mathematical Logic",
|
||||
decision_by="fox 2026-05-09",
|
||||
)
|
||||
assert inserted_again is False
|
||||
|
||||
# Remove + verify gone.
|
||||
removed = remove_citation_alias(
|
||||
db,
|
||||
"Introduction to Mathematical Logic by Elliott Mendelson",
|
||||
"Principles of Mathematical Logic by Hilbert and Ackermann",
|
||||
)
|
||||
assert removed is True
|
||||
assert lookup_citation_aliases(
|
||||
db, "Introduction to Mathematical Logic by Elliott Mendelson"
|
||||
) == []
|
||||
|
||||
|
||||
def test_citation_alias_audit_discipline(db):
|
||||
"""Empty `decision_by` raises — fail-closed audit."""
|
||||
with pytest.raises(ValueError, match="decision_by must be non-empty"):
|
||||
add_citation_alias(
|
||||
db,
|
||||
original_ref="A",
|
||||
substitute_ref="B",
|
||||
substitute_authors=[],
|
||||
substitute_title="",
|
||||
decision_by="",
|
||||
)
|
||||
|
||||
|
||||
def test_citation_alias_list_filter(db):
|
||||
add_citation_alias(
|
||||
db,
|
||||
original_ref="Mendelson 1997",
|
||||
substitute_ref="HA 1928",
|
||||
substitute_authors=["Hilbert"],
|
||||
substitute_title="Principles",
|
||||
decision_by="fox",
|
||||
)
|
||||
add_citation_alias(
|
||||
db,
|
||||
original_ref="Stanley §1.2",
|
||||
substitute_ref="Brualdi §3.5",
|
||||
substitute_authors=["Brualdi"],
|
||||
substitute_title="Combinatorics",
|
||||
decision_by="fox",
|
||||
)
|
||||
all_a = list_citation_aliases(db)
|
||||
assert len(all_a) == 2
|
||||
filtered = list_citation_aliases(db, original_filter="Mendelson")
|
||||
assert len(filtered) == 1
|
||||
assert filtered[0].original_ref == "Mendelson 1997"
|
||||
|
||||
|
||||
# --- term aliases ----------------------------------------------------
|
||||
|
||||
|
||||
def test_term_alias_add_lookup(db):
|
||||
inserted = add_term_alias(
|
||||
db,
|
||||
term="incidence",
|
||||
alternate_term="connection",
|
||||
domain="geometry",
|
||||
decision_by="fox 2026-05-09",
|
||||
decision_rationale="Hilbert 1902 Townsend uses 'connection'",
|
||||
)
|
||||
assert inserted is True
|
||||
|
||||
# Forward lookup
|
||||
alts = lookup_term_aliases(db, "incidence", "geometry")
|
||||
assert "connection" in alts
|
||||
|
||||
# Bidirectional — registered (incidence, connection) also returns
|
||||
# "incidence" when looking up by "connection".
|
||||
reverse = lookup_term_aliases(db, "connection", "geometry")
|
||||
assert "incidence" in reverse
|
||||
|
||||
|
||||
def test_term_alias_domain_isolation(db):
|
||||
"""Same term in different domains stay distinct."""
|
||||
add_term_alias(
|
||||
db, term="set", alternate_term="class",
|
||||
domain="set-theory", decision_by="fox",
|
||||
)
|
||||
add_term_alias(
|
||||
db, term="set", alternate_term="ensemble",
|
||||
domain="combinatorics", decision_by="fox",
|
||||
)
|
||||
assert lookup_term_aliases(db, "set", "set-theory") == ["class"]
|
||||
assert lookup_term_aliases(db, "set", "combinatorics") == ["ensemble"]
|
||||
|
||||
|
||||
def test_term_alias_lowercase_normalization(db):
|
||||
"""Term + alternate + domain stored lowercased; lookup is
|
||||
case-insensitive."""
|
||||
add_term_alias(
|
||||
db, term="Incidence", alternate_term="Connection",
|
||||
domain="GEOMETRY", decision_by="fox",
|
||||
)
|
||||
assert lookup_term_aliases(db, "incidence", "geometry") == ["connection"]
|
||||
assert lookup_term_aliases(db, "INCIDENCE", "Geometry") == ["connection"]
|
||||
|
||||
|
||||
def test_term_alias_remove(db):
|
||||
add_term_alias(
|
||||
db, term="x", alternate_term="y",
|
||||
domain="z", decision_by="fox",
|
||||
)
|
||||
removed = remove_term_alias(db, "x", "y", "z")
|
||||
assert removed is True
|
||||
assert lookup_term_aliases(db, "x", "z") == []
|
||||
|
||||
|
||||
def test_term_alias_audit_discipline(db):
|
||||
"""Empty fields raise — fail-closed."""
|
||||
with pytest.raises(ValueError, match="decision_by must be non-empty"):
|
||||
add_term_alias(
|
||||
db, term="a", alternate_term="b",
|
||||
domain="c", decision_by="",
|
||||
)
|
||||
with pytest.raises(ValueError, match="domain must be non-empty"):
|
||||
add_term_alias(
|
||||
db, term="a", alternate_term="b",
|
||||
domain="", decision_by="fox",
|
||||
)
|
||||
|
||||
|
||||
# --- query expansion -------------------------------------------------
|
||||
|
||||
|
||||
def test_expand_query_with_term_aliases_basic(db):
|
||||
add_term_alias(
|
||||
db, term="incidence", alternate_term="connection",
|
||||
domain="geometry", decision_by="fox",
|
||||
)
|
||||
expanded = expand_query_with_term_aliases(
|
||||
"incidence axiom", "geometry", db
|
||||
)
|
||||
assert "(incidence OR connection)" in expanded
|
||||
# Other tokens pass through.
|
||||
assert "axiom" in expanded
|
||||
|
||||
|
||||
def test_expand_query_preserves_phrase_syntax(db):
|
||||
add_term_alias(
|
||||
db, term="line incidence", alternate_term="line connection",
|
||||
domain="geometry", decision_by="fox",
|
||||
)
|
||||
expanded = expand_query_with_term_aliases(
|
||||
'"line incidence"', "geometry", db
|
||||
)
|
||||
assert '"line incidence"' in expanded
|
||||
assert '"line connection"' in expanded
|
||||
assert " OR " in expanded
|
||||
|
||||
|
||||
def test_expand_query_no_aliases_passes_through(db):
|
||||
"""No registered aliases → query unchanged."""
|
||||
q = "modus ponens"
|
||||
expanded = expand_query_with_term_aliases(q, "logic", db)
|
||||
assert expanded == q
|
||||
|
||||
|
||||
def test_expand_query_unreachable_db_returns_original():
|
||||
"""Bad DB path → query unchanged (fail-closed; aliases are opt-in)."""
|
||||
expanded = expand_query_with_term_aliases(
|
||||
"incidence", "geometry", "/nonexistent/path/aliases.db"
|
||||
)
|
||||
assert expanded == "incidence"
|
||||
Loading…
Add table
Add a link
Reference in a new issue