russell@unturf.com
2c98fc964e
Three workstreams, full suite 2482 passed, experimental paths default-OFF. #000055 — Windows quickstart without make tasks.py (pure-stdlib runner) + make.bat shim + .gitattributes; README Windows section rewritten. Quickstart needs only Python 3.10+ (no make/bzip2/curl/bash). Mirrors the Makefile quickstart subset; drift-pinned by tests/test_tasks_runner.py. #000001 §7 Phase 0 — deterministic cross-language guard arborist/qa/crosslang.py: non-English signal (¿/¡/non-ASCII) + an es function-word stoppack. Fail-closed to UNGROUNDED before retrieval/LLM (mirrors the quantifier reject-DAG) when no content token survives, else strips es stopwords from the retrieval query only. English path byte-identical by construction. Default OFF (crosslang_guard_enabled). Measured: the anarcocapitalismo field case 10.4s -> 1.6s. #000056 — Operation Sandwich (cross-language grounding) arborist/qa/mt/: opus-mt es/fr/ru<->en, lazy per-pair memoised singleton (fixes the 88%-engine-error concurrency defect), manifest-pinned, [mt] extra; entity_mask wrapper. Sandwich = translate query in (retrieval + LLM prompt) -> English answer -> UNTOUCHED verifier grounds English-vs-English -> translate the verified answer out as display-only (banner-labelled, zero grounding). question_hash + verifier_policy_hash invariant; MT engine identity binds into RetrievalPlan, not governance. CLI --crosslang-translate / make XLANG_MT=1. Default OFF; entity_mask default OFF (measured net-negative at bench scale). Fan-out bench (bench/*.py): Spanish ~0% -> 71% grounded vs the real no-support baseline; the round-trip predictor was tried and refuted; the entity-mask lever failed at scale (corpus-title anchoring untried). CLAUDE.md: cross-language bright-line convention + module map. Pre-existing modified diagram files are intentionally excluded.
62 lines
2 KiB
Python
62 lines
2 KiB
Python
#!/usr/bin/env python3
|
|
"""Generate the Spanish bench set from the English one via opus-mt-en-es.
|
|
|
|
Reproducible round-trip fixture for #000056: the SAME engine the
|
|
sandwich uses on its edges generates the Spanish questions, so the
|
|
bench measures `es(question) → [sandwich es→en] → English grounding`
|
|
end-to-end with no hand-translation, no Hermes-for-translation, no
|
|
egress. Comment / blank lines are preserved verbatim so qa_sweep.py
|
|
skips them exactly as in the English file.
|
|
|
|
Writes:
|
|
bench/qa_questions_es.txt one es question per line
|
|
bench/qa_questions_es_map.json [{en, es}, ...] in file order
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
ROOT = Path(__file__).resolve().parents[1]
|
|
sys.path.insert(0, str(ROOT))
|
|
|
|
from arborist.qa.mt import OpusMTTranslator # noqa: E402
|
|
|
|
SRC = ROOT / "bench" / "qa_questions.txt"
|
|
OUT = ROOT / "bench" / "qa_questions_es.txt"
|
|
MAP = ROOT / "bench" / "qa_questions_es_map.json"
|
|
|
|
|
|
def main() -> int:
|
|
tr = OpusMTTranslator()
|
|
lines = SRC.read_text().splitlines()
|
|
out_lines: list[str] = [
|
|
"# AUTO-GENERATED from bench/qa_questions.txt via "
|
|
"Helsinki-NLP/opus-mt-en-es (#000056 Operation Sandwich).",
|
|
"# Do not hand-edit — regenerate: python3 bench/make_es_questions.py",
|
|
"",
|
|
]
|
|
pairs: list[dict] = []
|
|
n = 0
|
|
for ln in lines:
|
|
s = ln.strip()
|
|
if not s or s.startswith("#"):
|
|
out_lines.append(ln)
|
|
continue
|
|
es = tr.translate(s, "en", "es")
|
|
if not tr.available:
|
|
print("opus-mt unavailable — install 'arborist[mt]'", file=sys.stderr)
|
|
return 1
|
|
out_lines.append(es)
|
|
pairs.append({"en": s, "es": es})
|
|
n += 1
|
|
print(f" [{n}] {s} -> {es}", flush=True)
|
|
OUT.write_text("\n".join(out_lines) + "\n")
|
|
MAP.write_text(json.dumps(pairs, ensure_ascii=False, indent=2) + "\n")
|
|
print(f"\nwrote {OUT} ({n} questions)\nwrote {MAP}", flush=True)
|
|
return 0
|
|
|
|
|
|
if __name__ == "__main__":
|
|
raise SystemExit(main())
|