russell@unturf.com
2c98fc964e
Three workstreams, full suite 2482 passed, experimental paths default-OFF. #000055 — Windows quickstart without make tasks.py (pure-stdlib runner) + make.bat shim + .gitattributes; README Windows section rewritten. Quickstart needs only Python 3.10+ (no make/bzip2/curl/bash). Mirrors the Makefile quickstart subset; drift-pinned by tests/test_tasks_runner.py. #000001 §7 Phase 0 — deterministic cross-language guard arborist/qa/crosslang.py: non-English signal (¿/¡/non-ASCII) + an es function-word stoppack. Fail-closed to UNGROUNDED before retrieval/LLM (mirrors the quantifier reject-DAG) when no content token survives, else strips es stopwords from the retrieval query only. English path byte-identical by construction. Default OFF (crosslang_guard_enabled). Measured: the anarcocapitalismo field case 10.4s -> 1.6s. #000056 — Operation Sandwich (cross-language grounding) arborist/qa/mt/: opus-mt es/fr/ru<->en, lazy per-pair memoised singleton (fixes the 88%-engine-error concurrency defect), manifest-pinned, [mt] extra; entity_mask wrapper. Sandwich = translate query in (retrieval + LLM prompt) -> English answer -> UNTOUCHED verifier grounds English-vs-English -> translate the verified answer out as display-only (banner-labelled, zero grounding). question_hash + verifier_policy_hash invariant; MT engine identity binds into RetrievalPlan, not governance. CLI --crosslang-translate / make XLANG_MT=1. Default OFF; entity_mask default OFF (measured net-negative at bench scale). Fan-out bench (bench/*.py): Spanish ~0% -> 71% grounded vs the real no-support baseline; the round-trip predictor was tried and refuted; the entity-mask lever failed at scale (corpus-title anchoring untried). CLAUDE.md: cross-language bright-line convention + module map. Pre-existing modified diagram files are intentionally excluded.
37 lines
1.4 KiB
Python
37 lines
1.4 KiB
Python
#!/usr/bin/env python3
|
|
"""Generate bench/qa_questions_<lang>.txt from the English set via
|
|
opus-mt-en-<lang> (#000056 §9 multi-bread). usage: make_lang_questions.py <lang>"""
|
|
from __future__ import annotations
|
|
import json, sys
|
|
from pathlib import Path
|
|
ROOT = Path(__file__).resolve().parents[1]
|
|
sys.path.insert(0, str(ROOT))
|
|
from arborist.qa.mt import OpusMTTranslator
|
|
|
|
LANG = sys.argv[1] if len(sys.argv) > 1 else "es"
|
|
SRC = ROOT / "bench" / "qa_questions.txt"
|
|
OUT = ROOT / "bench" / f"qa_questions_{LANG}.txt"
|
|
MAP = ROOT / "bench" / f"qa_questions_{LANG}_map.json"
|
|
|
|
def main() -> int:
|
|
tr = OpusMTTranslator()
|
|
out = [f"# AUTO-GENERATED from qa_questions.txt via opus-mt-en-{LANG} (#000056).",
|
|
"# Regenerate: python3 bench/make_lang_questions.py " + LANG, ""]
|
|
pairs = []
|
|
n = 0
|
|
for ln in SRC.read_text().splitlines():
|
|
s = ln.strip()
|
|
if not s or s.startswith("#"):
|
|
out.append(ln); continue
|
|
t = tr.translate(s, "en", LANG)
|
|
if not tr.available:
|
|
print("opus-mt unavailable", file=sys.stderr); return 1
|
|
out.append(t); pairs.append({"en": s, LANG: t}); n += 1
|
|
print(f" [{n}] {s} -> {t}", flush=True)
|
|
OUT.write_text("\n".join(out) + "\n")
|
|
MAP.write_text(json.dumps(pairs, ensure_ascii=False, indent=2) + "\n")
|
|
print(f"wrote {OUT} ({n}) + {MAP}", flush=True)
|
|
return 0
|
|
|
|
if __name__ == "__main__":
|
|
raise SystemExit(main())
|