fan-out: warrant ladder wiring · witness follow-ups · 5F Phase 1d

Three small streams in one commit; each closes / expands a
recently-landed ticket without changing its hard contract.

#000026 Phase 3 wiring — authorship warrant ladder visible
============================================================

Phase 3 sidecar (arborist/qa/warrant_authorship.py landed in 60b5748)
exposed the classifier but didn't surface it. Two wirings:

- arborist/qa/inspect.py — diagnose_authorship_warrant runs against
  the cached row's question + answer + per-source raw chunks +
  URIs + titles; result lands as `authorship` field alongside the
  other sidecars.
- arborist/cli.py _render_warrant_tail — appends ` · warrant:
  <readable-tier>` when result['authorship'] is populated with a
  non-quiet tier. AUTHOR_COPYRIGHT_FOOTER → "copyright-footer", etc.
  NO_AUTHORSHIP_SIGNAL stays silent. Backward-compat: results
  without an `authorship` key render unchanged.

Tests: 3 inspect-path tests (no-signal, copyright-footer,
repository-owner) + 4 render-tail tests (presence, no-signal
silence, missing-key silence, all-six-tiers readable mapping).

#000028 follow-ups — capital ledger + sample-rate
==================================================

Two policy fields layered on top of canonical_witness_enabled:

- canonical_witness_sample_rate (0.0..1.0; default 1.0). Operators
  wanting passive calibration set 0.05 to fire witness on 5% of
  canonical questions while paying 5% of LLM cost. 0.0 effectively
  off; 1.0 = current always-on behavior. Gating uses random.random()
  so distribution is uniform; clamped to [0, 1].
- Capital ledger row written for each FIRED witness (not skipped
  ones). op_type='canonical_witness'; estimator inputs include
  prompt_chars + answer_chars + llm_seconds + agreement_label +
  pi_star_ref. Best-effort: ledger-write failure must never fail
  the query (sidecar discipline).

Tests: 4 new — sample_rate=0.0 skips (no LLM call, no ledger row);
sample_rate=1.0 always fires; capital_ledger row written under
op_type='canonical_witness' with full input blob; sampled-out
witness records zero ledger rows.

Both fields fold into governance_policy_hash naturally via the
existing policy-hash machinery — flipping witness mode invalidates
prior records as expected.

#000025 Phase 1d — 5F fixture catalog 30 → 50
==============================================

Both synthetic and live sides of all 5 sub-batteries expanded
30 → 50 (+200 fixtures total: 5 × 20 synthetic, 5 × 20 live).

  function       — claim_count cycles 2..7 across new fixtures
  falsification  — 10-violation palette across new ids
  feedback-loop  — fact-N learning chains
  finetuning     — capability transitions across canonical π*
                   (math/logic/algebra/calculus pool)
  formulate      — multi-pointer claim shapes

500/500 pass through respective runners. test_session_integration
total bumped 562 → 662. Pinned test_5f_*_runs counts updated 30 →
50 (synthetic main + embedded + live).

Tests
=====

Full suite: 1467 passed, 36 skipped (was 1388; +79 across warrant
render + witness sample/ledger + 5F implicit coverage).
This commit is contained in:
russell@unturf.com 2026-05-09 12:42:56 -04:00
parent 04f3f5d2a8
commit 708aa450cb
No known key found for this signature in database
19 changed files with 619 additions and 62 deletions

View file

@ -228,31 +228,31 @@ def test_5f_function_runs():
assert res.battery == "5f"
assert res.sub_battery == "function"
# Phase 1c (2026-05-09) — fixture catalog expanded 10 → 30.
assert res.pass_count == 30
assert res.pass_count == 50 # Phase 1d
assert res.metrics["function_pass_rate"] == 1.0
def test_5f_finetuning_runs():
res = b_5f.run_finetuning(F5F / "finetuning-v1.jsonl")
assert res.pass_count == 30
assert res.pass_count == 50 # Phase 1d
assert res.metrics["adaptation_improvement_rate"] == 1.0
def test_5f_falsification_runs():
res = b_5f.run_falsification(F5F / "falsification-v1.jsonl")
assert res.pass_count == 30
assert res.pass_count == 50 # Phase 1d
assert res.metrics["error_detection_rate"] == 1.0
def test_5f_formulate_runs():
res = b_5f.run_formulate(F5F / "formulate-v1.jsonl")
assert res.pass_count == 30
assert res.pass_count == 50 # Phase 1d
assert res.metrics["structural_match_rate"] == 1.0
def test_5f_feedback_loop_runs():
res = b_5f.run_feedback_loop(F5F / "feedback-loop-v1.jsonl")
assert res.pass_count == 30
assert res.pass_count == 50 # Phase 1d
assert res.metrics["integration_coverage_rate"] == 1.0
@ -422,7 +422,7 @@ def test_5f_formulate_live_path_routes_through_parse_claims():
runner derives produced_lattice via parse_pointer_claims and
matches against expected_lattice."""
res = b_5f.run_formulate(F5F / "formulate-live-v1.jsonl")
assert res.pass_count == 30
assert res.pass_count == 50 # Phase 1d
assert res.metrics["structural_match_rate"] == 1.0
# Every task ran through the live path.
for t in res.per_task:
@ -434,7 +434,7 @@ def test_5f_formulate_embedded_path_still_works():
after the Phase 1b.2 wire-up + Phase 1c expansion. Backward
compat invariant."""
res = b_5f.run_formulate(F5F / "formulate-v1.jsonl")
assert res.pass_count == 30 # Phase 1c — expanded 10 → 30
assert res.pass_count == 50 # Phase 1d — expanded 10 → 30
for t in res.per_task:
assert t.detail["source"] == "embedded"
@ -478,7 +478,7 @@ def test_5f_feedback_loop_live_path_writes_real_audit_events():
+ memory.snapshot; expected_delta predicates verified against
the resulting audit_events / memory_branch_summaries."""
res = b_5f.run_feedback_loop(F5F / "feedback-loop-live-v1.jsonl")
assert res.pass_count == 30
assert res.pass_count == 50 # Phase 1d
# Every passing task must report source=live.
for t in res.per_task:
if t.passed:
@ -487,7 +487,7 @@ def test_5f_feedback_loop_live_path_writes_real_audit_events():
def test_5f_feedback_loop_embedded_path_still_works():
res = b_5f.run_feedback_loop(F5F / "feedback-loop-v1.jsonl")
assert res.pass_count == 30 # Phase 1c
assert res.pass_count == 50 # Phase 1d
for t in res.per_task:
assert t.detail["source"] == "embedded"
@ -552,7 +552,7 @@ def test_5f_live_feedback_chain_audit_chain_intact():
def test_5f_function_live_path_routes_through_parse_claims():
res = b_5f.run_function(F5F / "function-live-v1.jsonl")
assert res.pass_count == 30
assert res.pass_count == 50 # Phase 1d
for t in res.per_task:
if t.passed:
assert t.detail["source"] == "live"
@ -560,7 +560,7 @@ def test_5f_function_live_path_routes_through_parse_claims():
def test_5f_function_embedded_path_still_works():
res = b_5f.run_function(F5F / "function-v1.jsonl")
assert res.pass_count == 30 # Phase 1c
assert res.pass_count == 50 # Phase 1d
for t in res.per_task:
assert t.detail["source"] == "embedded"
@ -576,7 +576,7 @@ def test_5f_function_live_helper_uses_real_parser():
def test_5f_finetuning_live_path_round_trips_selfmodel():
res = b_5f.run_finetuning(F5F / "finetuning-live-v1.jsonl")
assert res.pass_count == 30
assert res.pass_count == 50 # Phase 1d
for t in res.per_task:
if t.passed:
assert t.detail["source"] == "live"
@ -584,7 +584,7 @@ def test_5f_finetuning_live_path_round_trips_selfmodel():
def test_5f_finetuning_embedded_path_still_works():
res = b_5f.run_finetuning(F5F / "finetuning-v1.jsonl")
assert res.pass_count == 30 # Phase 1c
assert res.pass_count == 50 # Phase 1d
for t in res.per_task:
assert t.detail["source"] == "embedded"
@ -610,7 +610,7 @@ def test_5f_finetuning_live_helper_persists_real_selfmodel():
def test_5f_falsification_live_path_routes_through_verify_quotes():
res = b_5f.run_falsification(F5F / "falsification-live-v1.jsonl")
assert res.pass_count == 30
assert res.pass_count == 50 # Phase 1d
for t in res.per_task:
if t.passed:
assert t.detail["source"] == "live"
@ -618,7 +618,7 @@ def test_5f_falsification_live_path_routes_through_verify_quotes():
def test_5f_falsification_embedded_path_still_works():
res = b_5f.run_falsification(F5F / "falsification-v1.jsonl")
assert res.pass_count == 30 # Phase 1c
assert res.pass_count == 50 # Phase 1d
for t in res.per_task:
assert t.detail["source"] == "embedded"

View file

@ -249,10 +249,14 @@ def test_classify_no_overlap_full_invention():
def _seed_record(qa_db: Path, *, cache_key: str, sources: list[dict],
unverified: list[str]) -> None:
unverified: list[str],
question_text: str = "what is foo?",
answer_text: str = "answer here") -> None:
"""Insert a minimal providence_cache row with a merkle_proof that
points at the given sources. Caller has already populated each
shard with the actual document + chunks."""
shard with the actual document + chunks. ``question_text`` /
``answer_text`` default to the canonical pre-warrant fixture
pair; tests that exercise warrant-shape sidecars override them."""
conn = connect(qa_db)
try:
with transaction(conn):
@ -278,8 +282,8 @@ def _seed_record(qa_db: Path, *, cache_key: str, sources: list[dict],
cache_key,
"00" * 32,
"qh",
"what is foo?",
"answer here",
question_text,
answer_text,
json.dumps({"sources": sources}, ensure_ascii=False),
"mh",
"ch",
@ -764,3 +768,87 @@ def test_register_metaphor_dictionary_idempotent(tmp_path):
finally:
m._extra_dict_paths = saved_extra
m._english_wordlist_cache = saved_cache
# ---------------------------------------------------------------------
# Authorship warrant ladder (#000026 Phase 3) — wired into inspect()
# ---------------------------------------------------------------------
def test_inspect_includes_authorship_field(tmp_path):
"""Every inspect() result carries an `authorship` sidecar dict.
For non-authorship questions, tier is NO_AUTHORSHIP_SIGNAL."""
qa_db = tmp_path / "qa.db"
shard = tmp_path / "001.db"
DOC = "ab" * 32
_seed_doc(
shard,
document_root=DOC,
document_uri="https://example.com/x",
chunk_text="Pikachu can store electricity in its cheeks.",
)
_seed_record(
qa_db, cache_key="01" * 32,
sources=[{
"document_root": DOC, "document_uri": "https://example.com/x",
"title": "X", "shard": shard.name, "chunk_idx": 0,
}],
unverified=[],
)
result = inspect_cache_key("01" * 32, qa_db=qa_db, shards_dir=tmp_path)
assert "authorship" in result
assert result["authorship"]["tier"] == "NO_AUTHORSHIP_SIGNAL"
def test_inspect_authorship_copyright_footer_tier(tmp_path):
"""Authorship-shaped question + copyright-footer chunk text →
AUTHOR_COPYRIGHT_FOOTER tier (the canonical virt-back case)."""
qa_db = tmp_path / "qa.db"
shard = tmp_path / "001.db"
DOC = "cd" * 32
_seed_doc(
shard,
document_root=DOC,
document_uri="https://example.com/virt-back",
chunk_text="virt-back is a backup utility.\n\n© Russell Ballestrini",
)
_seed_record(
qa_db, cache_key="02" * 32,
sources=[{
"document_root": DOC,
"document_uri": "https://example.com/virt-back",
"title": "virt-back", "shard": shard.name, "chunk_idx": 0,
}],
unverified=[],
question_text="who wrote virt-back?",
answer_text="Russell Ballestrini wrote virt-back.",
)
result = inspect_cache_key("02" * 32, qa_db=qa_db, shards_dir=tmp_path)
assert result["authorship"]["tier"] == "AUTHOR_COPYRIGHT_FOOTER"
assert "Russell Ballestrini" in result["authorship"]["candidate_names"]
def test_inspect_authorship_repo_owner_tier(tmp_path):
"""Repo-URL chunks fire tier 2 (REPOSITORY_OWNER)."""
qa_db = tmp_path / "qa.db"
shard = tmp_path / "001.db"
DOC = "ef" * 32
_seed_doc(
shard,
document_root=DOC,
document_uri="https://example.com/foo",
chunk_text="See https://github.com/russellballestrini/virt-back for source.",
)
_seed_record(
qa_db, cache_key="03" * 32,
sources=[{
"document_root": DOC,
"document_uri": "https://example.com/foo",
"title": "foo", "shard": shard.name, "chunk_idx": 0,
}],
unverified=[],
question_text="who maintains virt-back?",
answer_text="russellballestrini.",
)
result = inspect_cache_key("03" * 32, qa_db=qa_db, shards_dir=tmp_path)
assert result["authorship"]["tier"] == "AUTHOR_REPOSITORY_OWNER"

View file

@ -239,13 +239,17 @@ def test_full_dav1dprometheus_suite_runs_end_to_end(tmp_path, capsys):
def test_full_suite_total_fixture_count():
"""Sanity check: the complete Dav1DPrometheus suite executes 562
"""Sanity check: the complete Dav1DPrometheus suite executes 662
deterministic tasks across 21 sub-batteries (5S+5T+5F+5R).
History:
- Phase 1a baseline: 462 tasks.
- Phase 1c (#000025, 2026-05-09): 5F synthetic side expanded
10 30 across all 5 sub-batteries; +100 562.
10 30; +100 562.
- Phase 1d (#000025, 2026-05-09): 5F synthetic 30 → 50; +100 → 662.
(Live side ALSO went 30 50 but lives in *-live-v1 files
that the default-fixture-set doesn't load — those run via
the dedicated Makefile targets.)
"""
from bench.batteries.runner import _DEFAULT_FIXTURES, _run_one
@ -253,7 +257,7 @@ def test_full_suite_total_fixture_count():
for (battery, sub), fx in _DEFAULT_FIXTURES.items():
result = _run_one(battery, sub, Path(fx))
total += result.pass_count + result.fail_count
assert total == 562
assert total == 662
def test_5s_phase1a_digests_unchanged_after_phase1b():

View file

@ -271,3 +271,65 @@ def test_noise_capitalized_words_not_classified_as_names():
# to secondary or no-signal.
assert "Reserved" not in out.get("candidate_names", [])
assert "All" not in out.get("candidate_names", [])
# ---------- Render-tail integration (#000026 Phase 3 wiring) ---------------
def test_render_tail_emits_warrant_when_authorship_set():
"""`_render_warrant_tail` adds ` · warrant: <readable-tier>` when
result['authorship'] is populated with a non-quiet tier."""
from arborist.cli import _render_warrant_tail
result = {
"violations": [],
"authorship": {
"tier": "AUTHOR_COPYRIGHT_FOOTER",
"tier_rank": 5,
"signals": [],
"candidate_names": ["Russell Ballestrini"],
},
}
tail = _render_warrant_tail(result)
assert "warrant: copyright-footer" in tail
def test_render_tail_silent_for_no_authorship_signal():
"""Sidecar's quiet verdict (NO_AUTHORSHIP_SIGNAL) → no tail."""
from arborist.cli import _render_warrant_tail
result = {
"violations": [],
"authorship": {
"tier": "NO_AUTHORSHIP_SIGNAL",
"tier_rank": 99,
},
}
tail = _render_warrant_tail(result)
assert "warrant" not in tail
def test_render_tail_silent_when_no_authorship_field():
"""Backward-compat: results without an `authorship` key render
unchanged (the tail-builder must not raise / must not emit)."""
from arborist.cli import _render_warrant_tail
result = {"violations": []}
tail = _render_warrant_tail(result)
assert "warrant" not in tail
def test_render_tail_warrant_uses_readable_token_per_tier():
"""Each tier renders with hyphen-lowercase and AUTHOR_ stripped."""
from arborist.cli import _render_warrant_tail
expectations = {
"AUTHOR_PACKAGE_METADATA": "warrant: package-metadata",
"AUTHOR_REPOSITORY_OWNER": "warrant: repository-owner",
"AUTHOR_PAGE_BYLINE": "warrant: page-byline",
"AUTHOR_PRIMARY_PAGE_TITLE": "warrant: primary-page-title",
"AUTHOR_COPYRIGHT_FOOTER": "warrant: copyright-footer",
"AUTHOR_SECONDARY_SOURCE": "warrant: secondary-source",
}
for tier, expected in expectations.items():
tail = _render_warrant_tail({
"violations": [],
"authorship": {"tier": tier, "tier_rank": 1},
})
assert expected in tail, f"{tier} → tail={tail!r}"

View file

@ -387,3 +387,101 @@ def test_query_canonical_witness_reaches_strict_after_persist(tmp_path):
)
assert second["status"] == "cache_hit"
assert second["witness"]["agreement_label"] == "STRICT-WITNESSED"
# ---------- #000028 follow-ups: capital ledger + sample-rate -------------
def test_query_canonical_witness_sample_rate_zero_skips(tmp_path):
"""sample_rate=0.0 → witness never fires (passive-calibration
config); LLM is not called even when canonical_witness_enabled=True."""
from arborist.qa.query import DEFAULT_QUERY_POLICY, query
client = StubClient(answer="3/10")
policy = dict(DEFAULT_QUERY_POLICY)
policy["canonical_witness_enabled"] = True
policy["canonical_witness_sample_rate"] = 0.0
qa_db = tmp_path / "qa.db"
result = query(
question="0.1 + 0.2", qa_db=qa_db,
chat_client=client, model_id="stub", policy=policy,
)
assert result["status"] == "cache_miss_then_written"
assert result.get("witness") is None # skipped — sampled out
assert client.calls == [] # critical: no LLM round-trip
def test_query_canonical_witness_sample_rate_one_always_fires(tmp_path):
"""sample_rate=1.0 (default when enabled) → witness fires every call."""
from arborist.qa.query import DEFAULT_QUERY_POLICY, query
client = StubClient(answer="3/10")
policy = dict(DEFAULT_QUERY_POLICY)
policy["canonical_witness_enabled"] = True
policy["canonical_witness_sample_rate"] = 1.0
qa_db = tmp_path / "qa.db"
result = query(
question="0.1 + 0.2", qa_db=qa_db,
chat_client=client, model_id="stub", policy=policy,
)
assert result.get("witness") is not None
assert len(client.calls) == 1
def test_query_canonical_witness_capital_ledger_records_cost(tmp_path):
"""When witness fires + persistence writes a row, the witness LLM
cost lands in capital_ledger as op_type='canonical_witness'.
ForkScore can then compare witness-on vs witness-off forks
honestly via the ledger summary."""
from arborist.qa.query import DEFAULT_QUERY_POLICY, query
from arborist.store import connect
client = StubClient(answer="3/10")
policy = dict(DEFAULT_QUERY_POLICY)
policy["canonical_witness_enabled"] = True
qa_db = tmp_path / "qa.db"
query(
question="0.1 + 0.2", qa_db=qa_db,
chat_client=client, model_id="stub", policy=policy,
)
conn = connect(qa_db)
try:
rows = conn.execute(
"SELECT op_type, estimator_inputs_blob FROM capital_ledger "
"WHERE op_type = 'canonical_witness'"
).fetchall()
finally:
conn.close()
assert len(rows) == 1
import json as _json
inputs = _json.loads(rows[0]["estimator_inputs_blob"])
assert inputs["pi_star_ref"] == "arithmetic@v1"
assert inputs["agreement_label"] in (
"STRICT-WITNESSED", "KERNEL-LLM-AGREE",
"CACHE-DRIFT", "LLM-DIVERGED", "KERNEL-LLM-DIVERGED",
)
def test_query_canonical_witness_no_ledger_when_sampled_out(tmp_path):
"""Sampled-out witness must not record capital cost (no work happened)."""
from arborist.qa.query import DEFAULT_QUERY_POLICY, query
from arborist.store import connect
client = StubClient(answer="3/10")
policy = dict(DEFAULT_QUERY_POLICY)
policy["canonical_witness_enabled"] = True
policy["canonical_witness_sample_rate"] = 0.0
qa_db = tmp_path / "qa.db"
query(
question="0.1 + 0.2", qa_db=qa_db,
chat_client=client, model_id="stub", policy=policy,
)
conn = connect(qa_db)
try:
n = conn.execute(
"SELECT COUNT(*) FROM capital_ledger "
"WHERE op_type = 'canonical_witness'"
).fetchone()[0]
finally:
conn.close()
assert n == 0