Commit graph

6 commits

Author SHA1 Message Date
9ec9469c4d
qa(frame): #000002 land — reference-frame polarity contract (D3 → ✓)
The phrase-pattern retrieval route (commit 1b8677d) closed the
RETRIEVAL side of reference-frame failure; this ticket closes
the ANSWER side.

New module aborist/qa/frame.py:
  FrameDetection dataclass (frame_kind, reference_title,
    reference_uri, confidence). Sidecar — never enters cache_key
    or governance_policy_hash.
  detect_frame(question, sources, phrase_match_roots) — heuristic
    detector. Reference-frame classification fires when:
    (a) phrase route surfaced at least one source, AND
    (b) at least one phrase-matched source is a reference work,
        determined by:
          - title parenthetical disambig (`(novel)` / `(film)` /
            `(play)` / `(franchise)` / etc.), OR
          - body sample contains ≥3 DISTINCT fiction markers
            (novel / published / protagonist / plot / ...).
    Distinct-marker count keeps the heuristic robust against a
    history article saying "novel approach" twice.

aborist/qa/query.py:
  Calls detect_frame for lattice modes only. Body sample uses the
  ARTICLE LEAD (chunk_idx=0, post-wikitext-strip) — fiction
  markers cluster in the lead on Wikipedia, not in plot chunks
  that may have been query-relevance-ranked higher.
  New policy field claim_lattice_polarity_preamble injected as a
  user-role message before the grounding_reminder when
  frame_kind == "reference". Format-string with
  {reference_title} placeholder.
  Result dict carries frame_detection (kind / title / uri /
  confidence) for renderer + bench consumption.

aborist/cli.py:
  Renderer adds a `reference frame: <title>` line when
  frame_detection.kind == "reference". Skipped for literal /
  no-phrase-route / ambiguous rows.

Live verification — Orwell case:
  PRE  : "The text does not directly state that Oceania has always
          been at war with East Asia."
  POST : "In George Orwell's dystopian novel Nineteen Eighty-Four,
          the nation of Oceania is always at war with Eastasia,
          but this is a result of propaganda and doublethink, not
          actual historical continuity. The war with Eastasia is
          a fabricated conflict to maintain control..."

Multi-frame compilation: distinguishes propaganda claim from
fictional-actual continuity, exactly the polarity contract from
the ticket §2 abstraction.

Literal queries (capital of france) unchanged — polarity preamble
only injects when frame_kind classifies as "reference".

22 new tests (19 in test_frame.py + 3 in test_cli_render.py).
Full suite: 734 passed (was 712, +22).

Directive D3 status: ½ → ✓. Ticket #000002 closed.

All seven structural directives now ✓:
  D1, D2, D5 (were ✓);
  D3 #000002, D4 #000001, D6 #000003, D7 #000005 (closed in this
  series).
2026-05-01 20:18:13 -04:00
1dfa4636c3
qa(provenance): #000001 land — retrieval_plan_hash binds D4
New module aborist/qa/retrieval_plan.py:
  RetrievalPlan dataclass — frozen, captures the operator-
    influenceable retrieval inputs that determine source
    selection (retrieval_keywords, top_k, over_fetch,
    max_context_chars, shard_ids).
  retrieval_plan_hash() — SHA-256 over canonical-JSON.
    Deterministic per call; folds into the run-DAG retrieval
    stage as a bound input alongside the existing
    sources_summary output.

build_run_dag (aborist/qa/dag.py) accepts retrieval_plan_hash:
  When provided, the retrieval stage hash binds BOTH plan
    (input) and result (output): H({"retrieval_plan_hash":...,
    "sources_summary_hash":...}).
  When omitted (e.g. legacy / quote-mode callers that haven't
    plumbed it yet), the retrieval stage falls back to the
    historical sources-summary-only hash so pre-#000001
    run_dag_root values stay stable.

query.py constructs the plan per call and passes it through.
Question text intentionally NOT in the plan — already covered
by question_hash. Shard ids included so audit can reproduce
which shards the search ran against.

Two runs with identical sources but different retrieval keywords
now produce different run_dag_root values — the provenance gap
on operator hints (auditor recovers "these were the keywords
that pulled in those sources") closes for the run-DAG path.

Marker test in tests/test_directives.py flipped from "absent"
assertion to "present": test_d4_retrieval_plan_binding_landed.
Plus test_d4_retrieval_plan_hash_module_exists pinning the
module shape. Full suite: 712 passed.

Deferred per ticket §6:
  - audit events (retrieval_plan_built / retrieval_result_selected)
  - providence_cache.retrieval_keywords SQL column
  - optional strict cache_key mode (retrieval_plan_affects_cache_key)

These are ergonomic affordances atop the run-DAG binding; earn
their own tickets when bench evidence shows the blob path is
too friction-heavy for real workflows.

Directive D4 status: ½ → ✓. Ticket #000001 closed.
2026-05-01 19:26:01 -04:00
3796c238cc
qa(label): #000005 land — four-rung ladder migration (D7 sharpened)
Replaces the two-rung EVIDENCE-LINKED / EVIDENCE-LINKED-PARTIAL
display label for claim-lattice methods with a four-rung ladder
that names a strictly stronger property at each rung:

  POINTER-LINKED       pointer/source/chunk verified;
                       warrant either didn't apply or failed
  ANCHOR-WARRANTED     pointer-linked + warrant passed where it ran;
                       other soft demotes may apply
  EVIDENCE-WARRANTED   anchor-warranted + no soft demotes
  UNGROUNDED           no verified pairs

HYBRID gets a -PARTIAL suffix on whichever rung applies.

Implementation: _render_audit_label gains a violations parameter
(defaults to None for backward-compat). _ladder_rung_for_lattice
discriminates rungs from the existing violations list:
  - WARRANT_MISSING in violations → POINTER-LINKED
  - any of {LAZY_ANCHOR_DEMOTED, POINTER_OVERFLOW_TRIMMED,
    TOO_MANY_CLAIMS, BARE_NAME_CLAIM} → ANCHOR-WARRANTED
  - else → EVIDENCE-WARRANTED

Design simplification vs the ticket's §3 sketch: the proposed
verifier_steps_ran field on the verdict dict was NOT needed. The
existing violations list carries enough signal to discriminate
all rungs. Per the five-step algorithm step 2: don't add fields
you don't need.

Quote / span / entity / paraphrase methods stay unchanged (their
STRICT verifies pinned spans, not synthesis).

Schema column audit_mode enum stays {STRICT, HYBRID, UNGROUNDED}
— pure renderer transformation, no governance_policy_hash bump,
no cache invalidation, no mesh-wire-format change. Existing
providence records render under the new ladder on next read.

5 new renderer tests in tests/test_cli_render.py covering each
rung mapping. D7 anti-regression test in tests/test_directives.py
updated to gate on the ladder labels. Bench helper docstring
follows. Full suite: 711 passed.

Directive D7 stays at ✓; ticket #000005 closed.
2026-05-01 19:20:47 -04:00
f419d76292
qa(warrant): #000003 land — anchor-class generalization (D6 → ✓)
Three new question-shape classes dispatched through warrant_check
alongside the existing relation + date anchors:

(1) Entity-list shape — `name X`, `list X`, `who are the members
    of X`. List-aware extractor `extract_entity_list_anchors`
    (multi-word phrases ∪ solo-cap individual names) so comma-
    separated entities each contribute. ANY-match semantics:
    demote-don't-reject when an extra entity from training-prior
    appears alongside grounded ones.

(2) Count shape — `how many X`, `how much X`. Digit ↔ word
    equivalence (claim says "six", span says "6", or vice versa)
    with ordinal collapse (`sixth → 6`). Year-shaped digits
    filter out (those belong to the existing date anchor class).
    ALL-match semantics: every count token in the claim must
    appear in some cited span as digit or word.

(3) Why-cause shape — `why X`. Cause-anchor pool widens to
    ≥5-char lowercase common nouns (post a generic stopword set
    that filters quantifier-adjective fillers like "various",
    "factors", "situation") PLUS proper-noun anchors from the
    existing extractor. Gated on why-shape only: lowercase
    common-noun extraction has higher false-positive risk
    elsewhere.

Per-class policy gate (proposed `claim_lattice_warrant_classes`
dict) deferred per the five-step algorithm step 2: single
`warrant_check_enabled: bool` is the minimum viable gate; per-
class flags earn their slot when bench evidence shows over-firing
on a specific class.

17 new warrant tests (detector + extractor + integration).
Marker test in test_directives.py flipped from "absent" to
"present" assertion: test_d6_warrant_generalization_landed.

Full suite: 709 passed (was 692, +17).

Directive D6 status flipped to ✓ in seven-point-program.md.
Ticket #000003 closed.
2026-05-01 19:16:06 -04:00
6db5d1dc64
docs: open #000004 (closed/retroactive) + #000005; update program status
#000004 — Directive coverage in bench summary. Filed as closed
(landed in commit acd1f9c) for design-log traceability. Captures
the v1 substrate: per-row directive_compliance helper, _summarize
aggregation, markdown directive-coverage section, 5 unit tests.

#000005 — Label ladder migration (POINTER-LINKED / ANCHOR-WARRANTED
/ EVIDENCE-WARRANTED / ENTAILMENT-VERIFIED). Status open. Migration
from today's two-rung EVIDENCE-LINKED to a four-rung ladder where
each rung names a strictly stronger property. Schema column stays
{STRICT, HYBRID, UNGROUNDED} so v9.8 cache_key invariants hold;
renderer-level mapping only. Reserves ENTAILMENT-VERIFIED for a
future committed entailment engine without forcing premature
implementation.

TICKETS.md index gains both rows. Next ID bumps to 000006.

seven-point-program.md status snapshot updated:
  - D7 row links #000005 (ladder migration proposal).
  - D8 row links #000004 (closed substrate).
  - New "Anti-regression test layer" section maps each directive
    to its structural pin in tests/test_directives.py.

Note on fox's roadmap item #000006 (Internal CTI Expansion / frame
lattice / multi-frame answers): scope overlaps with #000002
(Reference-Frame Polarity Contract / Module L). NOT opening as
a separate ticket — Module L already covers it.
2026-05-01 16:31:44 -04:00
c1dcd3142e
docs: seven-point-program north-star — directive matrix + status snapshot
The 2026-05-01 distilled architectural directive becomes the
audit lens applied before every new ticket / feature / prompt
edit. Each of the seven directives gets:

  - status (✓ / ½ / ✗)
  - code anchors (verifier paths, renderer hooks)
  - pinning tests (per-rule coverage in tests/)
  - tickets covering remaining work
  - bench signal (which row column reports it)

Status snapshot at landing:
  D1 (no LLM in verifier)              ✓
  D2 (pointer clauses)                 ✓
  D3 (CTI internally)                  ½  → ticket #000002
  D4 (retrieval map AND evidence map)  ½  → ticket #000001
  D5 (deterministic pointer verify)    ✓
  D6 (anchor-class warrant)            ½  → ticket #000003 (this commit pair)
  D7 (honest labels)                   ✓
  D8 (test-pinning before automation)  discipline

CLAUDE.md docs index gains a "North-star" section pointing at the
program; ticket entries gain a "Directive" column threading them
back to the program.
2026-05-01 16:19:50 -04:00