arborist/docs/ticket-000005-label-ladder-migration.md
russell@unturf.com 3796c238cc
qa(label): #000005 land — four-rung ladder migration (D7 sharpened)
Replaces the two-rung EVIDENCE-LINKED / EVIDENCE-LINKED-PARTIAL
display label for claim-lattice methods with a four-rung ladder
that names a strictly stronger property at each rung:

  POINTER-LINKED       pointer/source/chunk verified;
                       warrant either didn't apply or failed
  ANCHOR-WARRANTED     pointer-linked + warrant passed where it ran;
                       other soft demotes may apply
  EVIDENCE-WARRANTED   anchor-warranted + no soft demotes
  UNGROUNDED           no verified pairs

HYBRID gets a -PARTIAL suffix on whichever rung applies.

Implementation: _render_audit_label gains a violations parameter
(defaults to None for backward-compat). _ladder_rung_for_lattice
discriminates rungs from the existing violations list:
  - WARRANT_MISSING in violations → POINTER-LINKED
  - any of {LAZY_ANCHOR_DEMOTED, POINTER_OVERFLOW_TRIMMED,
    TOO_MANY_CLAIMS, BARE_NAME_CLAIM} → ANCHOR-WARRANTED
  - else → EVIDENCE-WARRANTED

Design simplification vs the ticket's §3 sketch: the proposed
verifier_steps_ran field on the verdict dict was NOT needed. The
existing violations list carries enough signal to discriminate
all rungs. Per the five-step algorithm step 2: don't add fields
you don't need.

Quote / span / entity / paraphrase methods stay unchanged (their
STRICT verifies pinned spans, not synthesis).

Schema column audit_mode enum stays {STRICT, HYBRID, UNGROUNDED}
— pure renderer transformation, no governance_policy_hash bump,
no cache invalidation, no mesh-wire-format change. Existing
providence records render under the new ladder on next read.

5 new renderer tests in tests/test_cli_render.py covering each
rung mapping. D7 anti-regression test in tests/test_directives.py
updated to gate on the ladder labels. Bench helper docstring
follows. Full suite: 711 passed.

Directive D7 stays at ✓; ticket #000005 closed.
2026-05-01 19:20:47 -04:00

8.4 KiB

Ticket #000005 — Label ladder migration: POINTER-LINKED / ANCHOR-WARRANTED / EVIDENCE-WARRANTED

Status: closed · landed 2026-05-02 Opened: 2026-05-01 Closed: 2026-05-02 Directive: D7 — Rename labels honestly. Scope: Replace today's two-label rendered surface (EVIDENCE-LINKED / EVIDENCE-LINKED-PARTIAL) with a four-rung ladder that names exactly what each verifier path proved. Schema column stays STRICT / HYBRID / UNGROUNDED so the v9.8 cache_key invariant holds; only the rendered surface changes. Audience: fox + future blackops shifts. Hard constraint: schema column stays unchanged. Renderer-level mapping only. The audit chain, mesh wire format, providence_cache, and all programmatic callers continue to see the original audit_mode enum.


1. Problem statement

Today's renderer relabels claim-lattice modes:

STRICT     → EVIDENCE-LINKED · via claim_lattice
HYBRID     → EVIDENCE-LINKED-PARTIAL · via claim_lattice_pointer
UNGROUNDED → UNGROUNDED · via claim_lattice

EVIDENCE-LINKED improves on STRICT (which overclaims semantic truth) but still reads as "the evidence supports the claim." A careful operator parses it as "pointer + role + coverage checks passed." A casual one reads it as "the model got this right."

The four-rung ladder fox proposed (2026-05-01) carries the distinction explicitly:

POINTER-LINKED       pointer/source/chunk relation verified
ANCHOR-WARRANTED     pointer-linked AND cited evidence contains
                     required claim anchors (Rule 7 + future
                     entity/count/cause classes from #000003)
EVIDENCE-WARRANTED   anchor-warranted AND no hard violations
ENTAILMENT-VERIFIED  reserved for a future committed entailment
                     engine (NLI substrate). Not reachable today.

Each rung names a strictly stronger property. An operator reading the rung knows what's been proved without inferring from the verifier_method tail.

2. Mapping from current state to ladder

Today's verifier output gets re-mapped at the renderer layer:

Verifier method Today's render New render
claim_lattice EVIDENCE-LINKED · via … POINTER-LINKED (no warrant ran), ANCHOR-WARRANTED (warrant fired & passed), EVIDENCE-WARRANTED (anchor-warranted + zero soft-demote violations)
claim_lattice (HYBRID) EVIDENCE-LINKED-PARTIAL POINTER-LINKED-PARTIAL or ANCHOR-WARRANTED-PARTIAL depending on which checks demoted
claim_lattice (UNGROUNDED) UNGROUNDED UNGROUNDED (unchanged)
quote / span / entity / paraphrase STRICT/HYBRID/UNGROUNDED unchanged (these verify against pinned spans, not synthesis)

Mapping rule (claim-lattice modes only):

def _ladder_label(audit_mode, violations) -> str:
    if audit_mode == "UNGROUNDED":
        return "UNGROUNDED"
    has_warrant_check = any(v["kind"] == "WARRANT_RAN" for v in violations)
    has_warrant_miss  = any(v["kind"] == "WARRANT_MISSING" for v in violations)
    has_other_demote  = any(v["kind"] in DEMOTE_KINDS for v in violations)
    if not has_warrant_check:
        rung = "POINTER-LINKED"
    elif has_warrant_miss:
        rung = "POINTER-LINKED"          # warrant ran AND failed
    elif has_other_demote:
        rung = "ANCHOR-WARRANTED"        # warrant ok but other demotes
    else:
        rung = "EVIDENCE-WARRANTED"      # warrant ok AND no demotes
    if audit_mode == "HYBRID":
        rung = rung + "-PARTIAL"
    return rung

3. New verifier signal: WARRANT_RAN

For the renderer to know whether the warrant fired, the verifier needs to surface that signal. Today's warrant_check returns (ok, missing); only WARRANT_MISSING enters the violations list when ok=False. Add a benign WARRANT_RAN violation entry (or a separate verifier_steps_ran: list[str] field) so the renderer can distinguish "warrant didn't apply" from "warrant applied and passed."

This is a renderer-input enhancement, not a verifier semantic change. The hard checks stay binary; we just thread the "which checks ran" provenance up to the display layer.

4. Cache implications

None. Schema audit_mode enum is unchanged; the renderer mapping runs on read, not on write. Existing providence records continue to render under the new ladder without re-classification or cache invalidation.

governance_policy_hash does NOT include the ladder mapping — it's a pure presentation transformation, not a behavioral change. Two records with the same cache_key written before and after the ladder migration render under the new ladder identically.

5. Implementation sketch

  1. Verifier enhancement (aborist/qa/verify.py): both verify_claim_lattice and verify_claim_lattice_json add a verifier_steps_ran: list[str] field on the verdict dict listing which verifier paths fired (pointer_resolve, source_role, coverage, pointer_cap, warrant, lazy_anchor_demote).
  2. Renderer mapping (aborist/cli.py:_render_audit_label): add the four-rung mapping function gated on verifier_method.startswith("claim_lattice").
  3. Bench update (bench/qa_sweep.py): per-row ladder_rung field; markdown summary gets a "ladder coverage" table alongside the existing strict-rate breakdown.
  4. Live fixtures (tests/test_qa_quality_live.py): the existing test_mona_lisa_strict_* and siblings get their assertion strings updated to the new rung names.

6. Tests required

  • [ ] Renderer maps verifier output to POINTER-LINKED when no warrant ran.
  • [ ] Renderer maps to ANCHOR-WARRANTED when warrant ran & passed but other demotes fired.
  • [ ] Renderer maps to EVIDENCE-WARRANTED when warrant passed & no demotes.
  • [ ] Renderer suffix -PARTIAL appears for HYBRID + each rung.
  • [ ] Renderer keeps STRICT/HYBRID/UNGROUNDED for quote-mode verifier paths (no ladder rename).
  • [ ] Verifier verdict dict carries verifier_steps_ran list.
  • [ ] Live fixtures updated to gate on rung names.
  • [ ] Cache lookup returns identical results before & after migration (governance_policy_hash unchanged).

7. Out of scope

  • ENTAILMENT-VERIFIED rung. Reserved for a future committed entailment engine. Not reachable today; no code path produces it. Naming the rung today preserves the design space without forcing premature implementation.
  • Schema column rename. Stays audit_mode ∈ {STRICT, HYBRID, UNGROUNDED}. v9.8 cache_key invariants depend on this; never break.
  • Mesh wire format change. Mesh peers exchange audit_mode, not the rendered rung. No wire-protocol bump required.

8. Status

Closed 2026-05-02. Landed via:

  • _render_audit_label(audit_mode, verifier_method, violations) in aborist/cli.py — three-arg signature; violations defaults to None for backward-compat with callers that don't have the list yet.
  • _ladder_rung_for_lattice helper computes the rung from (audit_mode, violations) using the existing violation kinds (no new verifier output field needed):
    • WARRANT_MISSING in violations → POINTER-LINKED
    • any of LAZY_ANCHOR_DEMOTED / POINTER_OVERFLOW_TRIMMED / TOO_MANY_CLAIMS / BARE_NAME_CLAIMANCHOR-WARRANTED
    • else → EVIDENCE-WARRANTED
    • HYBRID adds -PARTIAL suffix.
  • Quote / span / entity / paraphrase methods unchanged (their STRICT verifies pinned spans, not synthesis).
  • Schema column audit_mode enum stays {STRICT, HYBRID, UNGROUNDED} — pure renderer transformation.

Design simplification vs §3: the proposed verifier_steps_ran: list[str] field on the verdict dict was NOT needed. The existing violations list carries enough signal to discriminate the rungs (presence of WARRANT_MISSING discriminates POINTER-LINKED from ANCHOR-WARRANTED; presence of any soft-demote kind discriminates ANCHOR-WARRANTED from EVIDENCE-WARRANTED). Per the five-step algorithm step 2: don't add fields you don't need.

5 new renderer tests in tests/test_cli_render.py covering each rung mapping. D7 directive test in tests/test_directives.py updated to gate on the ladder labels. Full suite: 711 passed.

Cache implications: none. Schema stays unchanged; existing records render under the new ladder without re-classification. governance_policy_hash does NOT include the rendered label — pure presentation transformation.

Mesh wire format: unchanged. Peers exchange audit_mode not the rendered rung.