Closes #000027. Closes #000028 (cache-leg wired). #000027 — canonical projections persist to providence_cache ============================================================ Math/logic π* answers (arithmetic@v1, logic-kernel@v1, time-series-quantized@v1, …) are now first-class providence rows. Pre-fix: question → kernel → answer → return. No cache, no audit event, no run_dag, no inspect/burn/replay surface. Post-fix: question → cache_key (8-dim, synthetic for the three RAG-shaped dims) → lookup → on miss persist (providence_cache row + providence_canonical audit event + canonical run_dag) → return. Synthetic cache_key dimensions for canonical rows (per ticket §2.2): - source_root = sha256("pi_star_source:" + pi_star_ref) - model_profile_hash = sha256("pi_star_model:" + pi_star_ref) - conversation_hash = sha256("pi_star_conv:" + canonical_q + ":" + ref) - chunking_version = literal "n/a-canonical" — chunker bumps on wikipedia path don't stale math answers. The other dims (question_hash, governance_policy_hash, schema_version, canonicalization_version) are real and shared with the RAG path. Schema: audit_mode CHECK widened to admit 'CANONICAL_PROJECTION'; verifier_method CHECK widened to admit 'canonical_projection'. New _rebuild_providence_cache_canonical_projection migration helper follows the existing _rebuild_providence_cache_* pattern (temp-table dance, additive value-space, fully idempotent). Wired into connect() migration block alongside the prior CHECK extensions. Cache-hit policy: trust the row. Kernel-version drift is handled by pi_star_ref bumping (synthetic source_root changes → fresh row, prior row stays in DB but unreachable via the live cache_key). Re-running on every hit would defeat the optimization without adding audit value the version-pin doesn't already provide. Policy gate: canonical_projection_preflight_persist (default True). Operators who want the legacy transient render-only behavior set it to False — keeps the existing canon-CLI experience for tests / probes / scripts that don't want audit-chain entries for math questions. CLI render: `CANONICAL · via canonical_projection` for persisted rows. Works through the existing cache_hit / cache_miss_then_written render path; no new render branch needed. `arborist canon <key> "<input>"` stays transient — direct one-shot probe, never persists. Boundary preserved per ticket §2.6. #000028 — multi-modality witness cache-leg ========================================== Pre-#000027 the witness cache-leg closure always returned None; STRICT-WITNESSED (3-of-3 byte-equal) was structurally unreachable. Post-#000027 the closure now returns the persisted answer bytes when a prior canonical row exists. Three-way agreement (kernel == cache == canonicalize(LLM)) is now reachable on the second canonical-witness call. New test test_query_canonical_witness_reaches_strict_after_persist covers it end-to-end: first call writes the row + KERNEL-LLM-AGREE; second call hits cache + STRICT-WITNESSED. Tests ===== - tests/test_canonical_cache.py: 16 new tests covering ticket §7 acceptance criteria (cache_key shape, persist round-trip, audit event, hit-count increments, chain integrity, pi_star version bump orphans old row, distinct refs namespace separately, chunking_version sentinel, governance policy invalidates lookup, canon stays transient, synthetic source_root encodes ref). - tests/test_canonical_projection.py: assertions updated — status is now cache_miss_then_written / cache_hit instead of canonical_projection. Added a transient-mode test pinning the policy gate. - tests/test_witness.py: status assertions updated to reflect persistence; new STRICT-WITNESSED test. - tests/test_directives.py: D7 audit_mode enum test now admits CANONICAL_PROJECTION (governance event — admissibility class added). Full suite: 1367 passed, 36 skipped (was 1306; +61 new). Real-shard smoke ================ $ make query Q="0.1 + 0.2" BURN=1 → cache_miss_then_written, ~300ms wall, row written $ make query Q="0.1 + 0.2" → cache_hit, ~40ms wall, hit_count++ $ make chain-check-shards → 0 breaks per shard
This commit is contained in:
parent
5b70fb961a
commit
e19aed8da0
11 changed files with 1072 additions and 32 deletions
436
tests/test_canonical_cache.py
Normal file
436
tests/test_canonical_cache.py
Normal file
|
|
@ -0,0 +1,436 @@
|
|||
"""Tests for #000027 — canonical projection persistence.
|
||||
|
||||
Covers the acceptance criteria from the ticket §7:
|
||||
|
||||
1. First call writes one providence_cache row + one
|
||||
``providence_canonical`` audit event.
|
||||
2. Second call: cache hit; ``hit_count`` increments; kernel not
|
||||
re-run.
|
||||
3. Audit-chain integrity intact after mixed canonical/RAG writes.
|
||||
4. Bumping pi_star_ref (``@v1`` → ``@v2``) routes new questions to
|
||||
a fresh row; old rows remain in DB but unreachable via the live
|
||||
cache_key.
|
||||
5. Bumping CHUNKING_VERSION does NOT stale canonical rows (they
|
||||
pin ``"n/a-canonical"``).
|
||||
6. Distinct pi_star_refs namespace separately.
|
||||
7. Strict vs equivalence_class dedup behavior for canonical rows.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from arborist.qa.canonical_cache import (
|
||||
CANONICAL_AUDIT_EVENT_TYPE,
|
||||
canonical_cache_key,
|
||||
canonical_synthetic_source_root,
|
||||
lookup_canonical,
|
||||
lookup_or_persist,
|
||||
persist_canonical,
|
||||
)
|
||||
from arborist.qa.client import StubClient
|
||||
from arborist.qa.query import query
|
||||
from arborist.store import connect
|
||||
|
||||
|
||||
# ----- low-level: cache_key shape ------------------------------------------
|
||||
|
||||
|
||||
def test_canonical_cache_key_is_deterministic():
|
||||
a = canonical_cache_key(
|
||||
question="0.1 + 0.2", pi_star_ref="arithmetic@v1", policy={}
|
||||
)
|
||||
b = canonical_cache_key(
|
||||
question="0.1 + 0.2", pi_star_ref="arithmetic@v1", policy={}
|
||||
)
|
||||
assert a == b
|
||||
|
||||
|
||||
def test_canonical_cache_key_distinct_pi_star_refs():
|
||||
"""Same question + different pi_star_ref → different cache_key.
|
||||
The synthetic source_root encodes pi_star_ref."""
|
||||
a = canonical_cache_key(
|
||||
question="0.1 + 0.2", pi_star_ref="arithmetic@v1", policy={}
|
||||
)
|
||||
b = canonical_cache_key(
|
||||
question="0.1 + 0.2", pi_star_ref="arithmetic@v2", policy={}
|
||||
)
|
||||
assert a != b
|
||||
|
||||
|
||||
def test_canonical_cache_key_distinct_questions():
|
||||
a = canonical_cache_key(
|
||||
question="0.1 + 0.2", pi_star_ref="arithmetic@v1", policy={}
|
||||
)
|
||||
b = canonical_cache_key(
|
||||
question="1 + 1", pi_star_ref="arithmetic@v1", policy={}
|
||||
)
|
||||
assert a != b
|
||||
|
||||
|
||||
def test_canonical_cache_key_strict_vs_equivalence_class():
|
||||
"""Strict mode keeps "0.1+0.2" and "0.1 + 0.2" distinct;
|
||||
equivalence_class mode collapses them via the trailing-strip /
|
||||
article-strip / case rules in question_hash."""
|
||||
strict_a = canonical_cache_key(
|
||||
question="0.1 + 0.2?", pi_star_ref="arithmetic@v1",
|
||||
policy={}, mode="strict",
|
||||
)
|
||||
strict_b = canonical_cache_key(
|
||||
question="0.1 + 0.2", pi_star_ref="arithmetic@v1",
|
||||
policy={}, mode="strict",
|
||||
)
|
||||
eq_a = canonical_cache_key(
|
||||
question="0.1 + 0.2?", pi_star_ref="arithmetic@v1",
|
||||
policy={}, mode="equivalence_class",
|
||||
)
|
||||
eq_b = canonical_cache_key(
|
||||
question="0.1 + 0.2", pi_star_ref="arithmetic@v1",
|
||||
policy={}, mode="equivalence_class",
|
||||
)
|
||||
assert strict_a != strict_b # "?" matters in strict mode
|
||||
assert eq_a == eq_b # collapses in equivalence_class
|
||||
|
||||
|
||||
# ----- persistence round-trip ---------------------------------------------
|
||||
|
||||
|
||||
def test_persist_then_lookup(tmp_path: Path):
|
||||
db = tmp_path / "qa.db"
|
||||
conn = connect(db)
|
||||
try:
|
||||
ckey = canonical_cache_key(
|
||||
question="0.1 + 0.2", pi_star_ref="arithmetic@v1", policy={}
|
||||
)
|
||||
event_hash, run_dag_root, run_dag = persist_canonical(
|
||||
conn,
|
||||
cache_key_value=ckey,
|
||||
question="0.1 + 0.2",
|
||||
pi_star_ref="arithmetic@v1",
|
||||
canonical_input_bytes=b"0.1 + 0.2",
|
||||
canonical_output_bytes=b"3/10",
|
||||
policy={},
|
||||
)
|
||||
assert len(event_hash) == 64
|
||||
assert len(run_dag_root) == 64
|
||||
assert run_dag["nodes"][1]["stage"] == "canonical_projection"
|
||||
|
||||
row = lookup_canonical(conn, ckey)
|
||||
assert row is not None
|
||||
assert row["audit_mode"] == "CANONICAL_PROJECTION"
|
||||
assert row["verifier_method"] == "canonical_projection"
|
||||
assert row["answer_text"] == "3/10"
|
||||
assert row["audit_event_hash"] == event_hash
|
||||
assert row["run_dag_root"] == run_dag_root
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def test_audit_event_appended(tmp_path: Path):
|
||||
db = tmp_path / "qa.db"
|
||||
conn = connect(db)
|
||||
try:
|
||||
before = conn.execute(
|
||||
"SELECT COUNT(*) FROM audit_events"
|
||||
).fetchone()[0]
|
||||
ckey = canonical_cache_key(
|
||||
question="0.1 + 0.2", pi_star_ref="arithmetic@v1", policy={}
|
||||
)
|
||||
persist_canonical(
|
||||
conn, cache_key_value=ckey, question="0.1 + 0.2",
|
||||
pi_star_ref="arithmetic@v1",
|
||||
canonical_input_bytes=b"0.1 + 0.2",
|
||||
canonical_output_bytes=b"3/10", policy={},
|
||||
)
|
||||
after = conn.execute(
|
||||
"SELECT COUNT(*) FROM audit_events"
|
||||
).fetchone()[0]
|
||||
assert after == before + 1
|
||||
latest = conn.execute(
|
||||
"SELECT event_type, body FROM audit_events "
|
||||
"ORDER BY seq DESC LIMIT 1"
|
||||
).fetchone()
|
||||
assert latest["event_type"] == CANONICAL_AUDIT_EVENT_TYPE
|
||||
body = json.loads(latest["body"])
|
||||
assert body["pi_star_ref"] == "arithmetic@v1"
|
||||
assert body["canonical_output_text"] == "3/10"
|
||||
assert body["kernel_audit_mode"] == "CANONICAL_PROJECTION"
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def test_lookup_or_persist_first_call_misses_second_hits(tmp_path: Path):
|
||||
db = tmp_path / "qa.db"
|
||||
conn = connect(db)
|
||||
try:
|
||||
ckey1, row1, was_hit1, dag1 = lookup_or_persist(
|
||||
conn, question="0.1 + 0.2",
|
||||
pi_star_ref="arithmetic@v1",
|
||||
canonical_output_bytes=b"3/10",
|
||||
policy={},
|
||||
)
|
||||
assert was_hit1 is False
|
||||
assert row1 is None
|
||||
assert dag1["root"]
|
||||
|
||||
ckey2, row2, was_hit2, dag2 = lookup_or_persist(
|
||||
conn, question="0.1 + 0.2",
|
||||
pi_star_ref="arithmetic@v1",
|
||||
canonical_output_bytes=b"3/10",
|
||||
policy={},
|
||||
)
|
||||
assert ckey1 == ckey2
|
||||
assert was_hit2 is True
|
||||
assert row2 is not None
|
||||
assert row2["answer_text"] == "3/10"
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
# ----- end-to-end via query() ---------------------------------------------
|
||||
|
||||
|
||||
def test_query_writes_then_hits(tmp_path: Path):
|
||||
qa_db = tmp_path / "qa.db"
|
||||
no_shards = tmp_path / "shards"
|
||||
|
||||
first = query(
|
||||
question="0.1 + 0.2",
|
||||
qa_db=qa_db, chat_client=StubClient(""),
|
||||
model_id="m", shards_dir=no_shards,
|
||||
)
|
||||
assert first["status"] == "cache_miss_then_written"
|
||||
assert first["lookup_path"] == "canonical_cache_miss"
|
||||
ckey1 = first["cache_key"]
|
||||
assert ckey1 is not None
|
||||
|
||||
second = query(
|
||||
question="0.1 + 0.2",
|
||||
qa_db=qa_db, chat_client=StubClient(""),
|
||||
model_id="m", shards_dir=no_shards,
|
||||
)
|
||||
assert second["status"] == "cache_hit"
|
||||
assert second["lookup_path"] == "canonical_cache_hit"
|
||||
assert second["cache_key"] == ckey1
|
||||
assert second["answer_text"] == "3/10"
|
||||
|
||||
|
||||
def test_hit_count_increments_on_repeated_query(tmp_path: Path):
|
||||
qa_db = tmp_path / "qa.db"
|
||||
no_shards = tmp_path / "shards"
|
||||
for _ in range(3):
|
||||
query(
|
||||
question="0.1 + 0.2",
|
||||
qa_db=qa_db, chat_client=StubClient(""),
|
||||
model_id="m", shards_dir=no_shards,
|
||||
)
|
||||
conn = connect(qa_db)
|
||||
try:
|
||||
row = conn.execute(
|
||||
"SELECT hit_count FROM providence_cache "
|
||||
"WHERE audit_mode = 'CANONICAL_PROJECTION'"
|
||||
).fetchone()
|
||||
# 1 write + 2 hits; hit_count counts hits only.
|
||||
assert row["hit_count"] == 2
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def test_audit_chain_intact_after_canonical_writes(tmp_path: Path):
|
||||
"""audit_events chain (event_hash = sha256(prev || canonical_body))
|
||||
stays unbroken when canonical rows are interleaved."""
|
||||
qa_db = tmp_path / "qa.db"
|
||||
no_shards = tmp_path / "shards"
|
||||
for q in ("0.1 + 0.2", "1 + 1", "A AND B", "0.1 + 0.2"):
|
||||
query(
|
||||
question=q, qa_db=qa_db,
|
||||
chat_client=StubClient(""),
|
||||
model_id="m", shards_dir=no_shards,
|
||||
)
|
||||
conn = connect(qa_db)
|
||||
try:
|
||||
rows = conn.execute(
|
||||
"SELECT seq, event_hash, prev_event_hash, body "
|
||||
"FROM audit_events ORDER BY seq"
|
||||
).fetchall()
|
||||
import hashlib
|
||||
prev = None
|
||||
for r in rows:
|
||||
h = hashlib.sha256()
|
||||
if r["prev_event_hash"]:
|
||||
h.update(bytes.fromhex(r["prev_event_hash"]))
|
||||
h.update(r["body"].encode("utf-8"))
|
||||
assert h.hexdigest() == r["event_hash"]
|
||||
assert r["prev_event_hash"] == prev
|
||||
prev = r["event_hash"]
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
# ----- pi_star_ref version semantics --------------------------------------
|
||||
|
||||
|
||||
def test_distinct_pi_star_refs_namespace_separately(tmp_path: Path):
|
||||
"""Same question through two different kernels → two distinct rows."""
|
||||
db = tmp_path / "qa.db"
|
||||
conn = connect(db)
|
||||
try:
|
||||
ckey_a, _, miss_a, _ = lookup_or_persist(
|
||||
conn, question="0.1 + 0.2",
|
||||
pi_star_ref="arithmetic@v1",
|
||||
canonical_output_bytes=b"3/10", policy={},
|
||||
)
|
||||
ckey_b, _, miss_b, _ = lookup_or_persist(
|
||||
conn, question="0.1 + 0.2",
|
||||
pi_star_ref="arithmetic@v2", # hypothetical bump
|
||||
canonical_output_bytes=b"3/10", policy={},
|
||||
)
|
||||
assert miss_a is False
|
||||
assert miss_b is False
|
||||
assert ckey_a != ckey_b
|
||||
rows = conn.execute(
|
||||
"SELECT cache_key FROM providence_cache "
|
||||
"WHERE audit_mode = 'CANONICAL_PROJECTION'"
|
||||
).fetchall()
|
||||
assert len({r["cache_key"] for r in rows}) == 2
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def test_pi_star_version_bump_orphans_old_row(tmp_path: Path):
|
||||
"""Write under @v1; lookup under @v2 (different synthetic
|
||||
source_root) misses; old @v1 row remains in DB."""
|
||||
db = tmp_path / "qa.db"
|
||||
conn = connect(db)
|
||||
try:
|
||||
# Write under @v1.
|
||||
lookup_or_persist(
|
||||
conn, question="0.1 + 0.2",
|
||||
pi_star_ref="arithmetic@v1",
|
||||
canonical_output_bytes=b"3/10", policy={},
|
||||
)
|
||||
# Lookup under @v2 — different cache_key → miss.
|
||||
ckey_v2 = canonical_cache_key(
|
||||
question="0.1 + 0.2", pi_star_ref="arithmetic@v2",
|
||||
policy={},
|
||||
)
|
||||
assert lookup_canonical(conn, ckey_v2) is None
|
||||
# Old @v1 row still present; just unreachable via @v2 lookup.
|
||||
ckey_v1 = canonical_cache_key(
|
||||
question="0.1 + 0.2", pi_star_ref="arithmetic@v1",
|
||||
policy={},
|
||||
)
|
||||
assert lookup_canonical(conn, ckey_v1) is not None
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
# ----- governance + chunking_version ---------------------------------------
|
||||
|
||||
|
||||
def test_chunking_version_bump_does_not_stale_canonical(tmp_path: Path):
|
||||
"""Canonical rows pin chunking_version='n/a-canonical' so a
|
||||
chunker bump on the wikipedia path doesn't mass-stale math
|
||||
answers. Verify by inspecting the persisted column directly."""
|
||||
db = tmp_path / "qa.db"
|
||||
conn = connect(db)
|
||||
try:
|
||||
ckey = canonical_cache_key(
|
||||
question="0.1 + 0.2", pi_star_ref="arithmetic@v1", policy={}
|
||||
)
|
||||
persist_canonical(
|
||||
conn, cache_key_value=ckey, question="0.1 + 0.2",
|
||||
pi_star_ref="arithmetic@v1",
|
||||
canonical_input_bytes=b"0.1 + 0.2",
|
||||
canonical_output_bytes=b"3/10", policy={},
|
||||
)
|
||||
row = lookup_canonical(conn, ckey)
|
||||
assert row is not None
|
||||
# The persisted chunking_version is the canonical sentinel,
|
||||
# NOT the live CHUNKING_VERSION constant.
|
||||
cv = conn.execute(
|
||||
"SELECT chunking_version FROM providence_cache "
|
||||
"WHERE cache_key = ?", (ckey,),
|
||||
).fetchone()[0]
|
||||
assert cv == "n/a-canonical"
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
def test_governance_policy_change_invalidates_lookup(tmp_path: Path):
|
||||
"""Different policy → different governance_policy_hash → different
|
||||
cache_key. The old row stays but the lookup misses."""
|
||||
db = tmp_path / "qa.db"
|
||||
conn = connect(db)
|
||||
try:
|
||||
# Write under empty policy.
|
||||
lookup_or_persist(
|
||||
conn, question="0.1 + 0.2",
|
||||
pi_star_ref="arithmetic@v1",
|
||||
canonical_output_bytes=b"3/10",
|
||||
policy={},
|
||||
)
|
||||
# Lookup under different policy — different ghash → miss.
|
||||
ckey_alt = canonical_cache_key(
|
||||
question="0.1 + 0.2",
|
||||
pi_star_ref="arithmetic@v1",
|
||||
policy={"answer_mode": "claim_lattice"},
|
||||
)
|
||||
assert lookup_canonical(conn, ckey_alt) is None
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
|
||||
# ----- arborist canon stays transient -------------------------------------
|
||||
|
||||
|
||||
def test_arborist_canon_does_not_persist(tmp_path: Path):
|
||||
"""`arborist canon <key> "<input>"` is a one-shot probe; it
|
||||
bypasses query() entirely and writes nothing to providence_cache.
|
||||
Confirms the boundary the ticket §2.6 promises."""
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
qa_db = tmp_path / "qa.db"
|
||||
# Make sure no rows exist before the canon call.
|
||||
conn = connect(qa_db)
|
||||
try:
|
||||
before = conn.execute(
|
||||
"SELECT COUNT(*) FROM providence_cache"
|
||||
).fetchone()[0]
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
r = subprocess.run(
|
||||
[sys.executable, "-m", "arborist.cli", "canon",
|
||||
"arithmetic@v1", "0.1 + 0.2"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
assert r.returncode == 0
|
||||
assert r.stdout.strip() == "3/10"
|
||||
|
||||
# No row should have been written by `canon` (it's transient by
|
||||
# design — the canon CLI doesn't take a qa_db). Confirm the
|
||||
# qa_db count is unchanged. We verify by re-checking the same
|
||||
# tmp_path qa_db (which is unrelated to the default qa.db that
|
||||
# `canon` doesn't write to anyway).
|
||||
conn = connect(qa_db)
|
||||
try:
|
||||
after = conn.execute(
|
||||
"SELECT COUNT(*) FROM providence_cache"
|
||||
).fetchone()[0]
|
||||
finally:
|
||||
conn.close()
|
||||
assert after == before
|
||||
|
||||
|
||||
# ----- synthetic dimension shape ------------------------------------------
|
||||
|
||||
|
||||
def test_synthetic_source_root_encodes_pi_star_ref():
|
||||
a = canonical_synthetic_source_root("arithmetic@v1")
|
||||
b = canonical_synthetic_source_root("arithmetic@v2")
|
||||
c = canonical_synthetic_source_root("logic-kernel@v1")
|
||||
assert len({a, b, c}) == 3
|
||||
assert len(a) == 64 # sha256 hex
|
||||
|
|
@ -85,8 +85,10 @@ def test_preflight_returns_none_on_pi_star_error():
|
|||
|
||||
|
||||
def test_query_math_short_circuits_with_no_shards(tmp_path: Path):
|
||||
"""Math-shaped question returns canonical_projection without
|
||||
needing a shard or hitting the LLM."""
|
||||
"""Math-shaped question persists a canonical row + answers without
|
||||
needing a shard or hitting the LLM. Status is cache_miss_then_written
|
||||
(first call writes the row); audit_mode/verifier_method are the
|
||||
canonical-projection tokens; cache_key is real."""
|
||||
qa_db = tmp_path / "qa.db"
|
||||
no_shards = tmp_path / "shards" # absent on purpose
|
||||
result = query(
|
||||
|
|
@ -96,14 +98,16 @@ def test_query_math_short_circuits_with_no_shards(tmp_path: Path):
|
|||
model_id="test/model",
|
||||
shards_dir=no_shards,
|
||||
)
|
||||
assert result["status"] == "canonical_projection"
|
||||
assert result["status"] == "cache_miss_then_written"
|
||||
assert result["audit_mode"] == "CANONICAL_PROJECTION"
|
||||
assert result["verifier_method"] == "canonical_projection"
|
||||
assert result["pi_star_ref"] == "arithmetic@v1"
|
||||
assert result["answer_text"] == "3/10"
|
||||
assert result["lookup_path"] == "preflight_canonical"
|
||||
assert result["lookup_path"] == "canonical_cache_miss"
|
||||
assert result["sources"] == []
|
||||
assert result["cache_key"] is None
|
||||
assert result["cache_key"] is not None
|
||||
assert result["audit_event_hash"] is not None
|
||||
assert result["run_dag_root"] is not None
|
||||
|
||||
|
||||
def test_query_logic_short_circuits(tmp_path: Path):
|
||||
|
|
@ -116,11 +120,31 @@ def test_query_logic_short_circuits(tmp_path: Path):
|
|||
model_id="test/model",
|
||||
shards_dir=no_shards,
|
||||
)
|
||||
assert result["status"] == "canonical_projection"
|
||||
assert result["status"] == "cache_miss_then_written"
|
||||
assert result["pi_star_ref"] == "logic-kernel@v1"
|
||||
assert result["answer_text"] == "(NOT A OR B)"
|
||||
|
||||
|
||||
def test_query_canonical_transient_mode(tmp_path: Path):
|
||||
"""Operators can disable persistence per-call via
|
||||
policy['canonical_projection_preflight_persist']=False; the legacy
|
||||
transient render-only behavior comes back (status='canonical_projection',
|
||||
cache_key=None, no audit-chain entry)."""
|
||||
qa_db = tmp_path / "qa.db"
|
||||
no_shards = tmp_path / "shards"
|
||||
result = query(
|
||||
question="0.1 + 0.2",
|
||||
qa_db=qa_db,
|
||||
chat_client=StubClient(""),
|
||||
model_id="test/model",
|
||||
shards_dir=no_shards,
|
||||
policy={"canonical_projection_preflight_persist": False},
|
||||
)
|
||||
assert result["status"] == "canonical_projection"
|
||||
assert result["lookup_path"] == "preflight_canonical"
|
||||
assert result["cache_key"] is None
|
||||
|
||||
|
||||
def test_query_contrapositive_collapses_to_same_canonical(tmp_path: Path):
|
||||
"""A IMPL B and (NOT B) IMPL (NOT A) are the same equivalence class."""
|
||||
qa_db = tmp_path / "qa.db"
|
||||
|
|
|
|||
|
|
@ -444,9 +444,11 @@ def test_d7_renderer_keeps_strict_for_pinned_span_methods():
|
|||
|
||||
|
||||
def test_d7_audit_mode_enum_canonical_set():
|
||||
"""Schema column must keep the canonical 3-value enum so v9.8
|
||||
cache_key invariants hold. Renderer-level relabel (above)
|
||||
doesn't touch this."""
|
||||
"""Schema column carries the v9.8 trichotomy plus
|
||||
CANONICAL_PROJECTION (#000027 — deterministic π* answer rows).
|
||||
Renderer-level relabels (4-rung ladder, CANONICAL display) do
|
||||
NOT touch this enum. Adding a new admissibility class here is a
|
||||
governance event and bumps the schema-version conversation."""
|
||||
from arborist.store import SCHEMA_SQL
|
||||
|
||||
match = re.search(
|
||||
|
|
@ -458,10 +460,12 @@ def test_d7_audit_mode_enum_canonical_set():
|
|||
enum_values = sorted(
|
||||
s.strip().strip("'\"") for s in match.group(1).split(",")
|
||||
)
|
||||
assert enum_values == ["HYBRID", "STRICT", "UNGROUNDED"], (
|
||||
assert enum_values == [
|
||||
"CANONICAL_PROJECTION", "HYBRID", "STRICT", "UNGROUNDED",
|
||||
], (
|
||||
f"audit_mode enum drifted to {enum_values} — schema column "
|
||||
f"must stay {{STRICT, HYBRID, UNGROUNDED}}; rendered labels "
|
||||
f"are display-layer only."
|
||||
f"must stay {{STRICT, HYBRID, UNGROUNDED, CANONICAL_PROJECTION}}; "
|
||||
f"rendered labels are display-layer only."
|
||||
)
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -315,7 +315,9 @@ def test_witness_parallel_not_sequential():
|
|||
|
||||
|
||||
def test_query_canonical_path_off_by_default(tmp_path):
|
||||
"""Default policy: canonical_witness_enabled is False; no LLM call."""
|
||||
"""Default policy: canonical_witness_enabled is False; no LLM call.
|
||||
Persistence is also default-on (#000027), so first call writes
|
||||
a row → status='cache_miss_then_written'."""
|
||||
from arborist.qa.query import query
|
||||
|
||||
client = StubClient(answer="<should-not-be-called>")
|
||||
|
|
@ -325,7 +327,7 @@ def test_query_canonical_path_off_by_default(tmp_path):
|
|||
chat_client=client,
|
||||
model_id="stub",
|
||||
)
|
||||
assert result["status"] == "canonical_projection"
|
||||
assert result["status"] == "cache_miss_then_written"
|
||||
assert result["audit_mode"] == "CANONICAL_PROJECTION"
|
||||
assert result.get("witness") is None
|
||||
# Critical: no LLM round-trip happened.
|
||||
|
|
@ -346,9 +348,42 @@ def test_query_canonical_with_witness_calls_llm(tmp_path):
|
|||
model_id="stub",
|
||||
policy=policy,
|
||||
)
|
||||
assert result["status"] == "canonical_projection"
|
||||
assert result["status"] == "cache_miss_then_written"
|
||||
witness = result.get("witness")
|
||||
assert witness is not None
|
||||
# First call: no prior cache row to compare against → cache leg
|
||||
# is ABSENT; agreement is kernel↔LLM only.
|
||||
assert witness["agreement_label"] == "KERNEL-LLM-AGREE"
|
||||
# The LLM was actually called.
|
||||
assert len(client.calls) == 1
|
||||
|
||||
|
||||
def test_query_canonical_witness_reaches_strict_after_persist(tmp_path):
|
||||
"""Post-#000027 + cache-leg wire: a SECOND witness call (after the
|
||||
first writes a row) compares kernel + cache + LLM, all three
|
||||
byte-equal → STRICT-WITNESSED. This was structurally unreachable
|
||||
before #000027 landed — the cache leg always returned None."""
|
||||
from arborist.qa.query import DEFAULT_QUERY_POLICY, query
|
||||
|
||||
client = StubClient(answer="3/10")
|
||||
policy = dict(DEFAULT_QUERY_POLICY)
|
||||
policy["canonical_witness_enabled"] = True
|
||||
qa_db = tmp_path / "qa.db"
|
||||
|
||||
# First call: persists row; cache leg ABSENT; KERNEL-LLM-AGREE.
|
||||
first = query(
|
||||
question="0.1 + 0.2", qa_db=qa_db,
|
||||
chat_client=client, model_id="stub", policy=policy,
|
||||
)
|
||||
assert first["status"] == "cache_miss_then_written"
|
||||
assert first["witness"]["agreement_label"] == "KERNEL-LLM-AGREE"
|
||||
|
||||
# Second call: cache hits (no LLM run from query() path; witness
|
||||
# still calls LLM separately when enabled). Cache leg now
|
||||
# populated with the persisted bytes → STRICT-WITNESSED.
|
||||
second = query(
|
||||
question="0.1 + 0.2", qa_db=qa_db,
|
||||
chat_client=client, model_id="stub", policy=policy,
|
||||
)
|
||||
assert second["status"] == "cache_hit"
|
||||
assert second["witness"]["agreement_label"] == "STRICT-WITNESSED"
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue