arborist/docs/seven-point-program.md

# Seven-point program — the north-star directive

**Distilled 2026-05-01, Asia/Kuala_Lumpur**:

> Stop making Hermes prove things. Make Hermes emit pointer clauses;
> build CTI internally; bind the retrieval map and evidence map;
> verify pointers deterministically; add general anchor-class warrant
> before semantic NLI; rename labels honestly; and automate only after
> these invariants are test-pinned.

This is the architecture's north-star. Every new feature, prompt edit,
schema change, and ticket walks past these seven directives before
landing. The discipline frames how `bench/qa_sweep.py` and
`tests/test_qa_quality_live.py` cover the substrate: the bench is the
scoreboard, the live fixtures are the gates, and **this doc names what
the scoreboard and gates measure for**.

## The seven directives

### D1 — Stop making Hermes prove things

The verifier never calls the LLM. Faithfulness classification is a
deterministic property of (answer text, evidence map, source corpus),
not a model self-grade.

- **Status**: ✓ enforced.
- **Code anchors**:
  `aborist/qa/verify.py:verify_quotes`,
  `verify_claim_lattice`, `verify_claim_lattice_json` — pure functions
  with no chat-client parameter.
- **Pinning tests**: `tests/test_verify*.py`,
  `tests/test_claim_lattice.py` — every verifier path runs without a
  network call.
- **Smell-test**: if a future PR adds a `chat_client` argument to a
  verifier, D1 violated.

### D2 — Make Hermes emit pointer clauses

The model's prose surface is *citation-style* prose with bracketed
evidence-id tags (`Claim. [E12]`) or schema-constrained JSON
(`{"claims":[{"text":"...", "evidence_ids":["E12"]}]}`). The model
never types the quote string itself.

- **Status**: ✓ enforced for lattice modes.
- **Code anchors**:
  `aborist/qa/parse_claims.py:parse_pointer_claims`,
  `aborist/qa/verify.py:CLAIM_LATTICE_JSON_SCHEMA`,
  `aborist/qa/runner.py` (system prompts).
- **Pinning tests**: `tests/test_claim_lattice.py`,
  `tests/test_verify_json.py`.
- **Bench signal**: rows with `answer_mode ∈
  {claim_lattice_pointer, claim_lattice}` fall under this directive;
  rows with `answer_mode == "quote"` predate it (legacy).

### D3 — Build CTI internally

Clause-tree intelligence (frame lattice + multi-frame answer
compilation) lives **on our side of the wire**. The model proposes
clauses; the runtime composes them. Reference-frame queries get
multiple frames acknowledged in the answer (literal vs
fictional-actual vs in-universe-propaganda) rather than picking one
frame and discarding the others.

- **Status**: ½ — phrase-route closed the **retrieval** side
  (commit `1b8677d`); answer-side compilation pending.
- **Code anchors**: today, none. Future:
  `aborist/qa/frame.py:detect_frame`, runtime-side polarity contract
  emission.
- **Pinning tests**: today, only the phrase-route tests in
  `test_query.py::test_phrase_match_surfaces_topical_doc`. Future:
  per-shape frame-detector tests + multi-frame answer-shape live
  fixtures.
- **Open ticket**:
  [#000002 Reference-Frame Polarity Contract](ticket-000002-reference-frame-polarity-contract.md).
- **Bench signal**: today, `phrase_match_surfaced` flag on bench rows
  (true when phrase route fired). Once Module L lands, additionally
  `frame_kind ∈ {literal, reference, ambiguous}`.

### D4 — Bind the retrieval map AND the evidence map

Provenance binding covers both *what got retrieved* (sources) and
*how retrieval got there* (the operator-influenceable inputs:
keywords, top_k, over_fetch, max_context_chars, shard set).

- **Status**: ½ — `evidence_map_root` lives in the 9-stage run-DAG
  (`aborist/qa/dag.py`); retrieval-plan binding (the input side) is
  the open work.
- **Code anchors**: `aborist/qa/dag.py:build_run_dag`,
  `aborist/qa/evidence.py:evidence_map_root`.
- **Pinning tests**: `tests/test_qa_dag.py`,
  `tests/test_evidence.py`.
- **Open ticket**:
  [#000001 Retrieval-keywords audit gap](ticket-000001-retrieval-keywords-audit-gap.md).
- **Bench signal**: every lattice-mode row has `run_dag_root`
  populated; future bench column will report on `retrieval_plan_hash`
  presence once #000001 lands.

### D5 — Verify pointers deterministically

The hard verifier runs only deterministic checks (parser succeeded,
evidence_id resolves, source_role allowed, claim text non-empty,
citation coverage, pointer-count cap, lexical anchor warrant).
No NLI, no embeddings, no model self-grading.

- **Status**: ✓ enforced.
- **Code anchors**: `aborist/qa/verify.py:verify_claim_lattice` —
  seven hard checks documented inline.
- **Pinning tests**: `tests/test_claim_lattice.py` (per-rule
  coverage), `tests/test_verify_json.py`,
  `tests/test_verify.py`.
- **Bench signal**: every row carries `verifier_method ∈ {quote,
  span, entity, paraphrase, claim_lattice, none}` (CHECK constraint
  in `aborist/store.py`).

### D6 — General anchor-class warrant before semantic NLI

The lexical anchor check (Rule 7 in `verify_claim_lattice`) catches
the lazy-anchor failure where pointer / role / coverage all pass but
the cited chunk never names the answer entity. Today this fires
only on relation-shape questions. Generalizing the anchor-class
detector to entity-list / why-cause / when-date shapes pins a
broader class of failures structurally before any NLI substrate
exists in the proof path.

- **Status**: ½ — relation-shape only. Entity-list / why-cause /
  when-date shapes pass through unchecked.
- **Code anchors**:
  `aborist/qa/verify.py:_extract_anchor_candidates`,
  `aborist/qa/verify.py:_question_is_relation_shape`.
- **Pinning tests**: `tests/test_claim_lattice.py` covers
  relation-shape; entity-list / cause / date shapes have no warrant
  test today.
- **Open ticket**:
  [#000003 Anchor-class warrant generalization](ticket-000003-anchor-class-warrant.md).
- **Bench signal**: future column `warrant_fired: bool` (true when
  Rule 7 ran on the row, regardless of pass/fail).

### D7 — Rename labels honestly

`STRICT` for claim-lattice modes overclaims because synthesis-heavy
claims pass all hard checks without semantic entailment of the
joined assertion. The renderer relabels at display time:

```
STRICT     → EVIDENCE-LINKED · via claim_lattice
HYBRID     → EVIDENCE-LINKED-PARTIAL · via claim_lattice_pointer
UNGROUNDED → UNGROUNDED · via claim_lattice
```

Quote / span / entity / paraphrase modes keep their original tokens
(they verify against pinned spans, not synthesis). The schema column
stays `STRICT`/`HYBRID`/`UNGROUNDED` so v9.8 cache_key invariants
hold.

- **Status**: ✓ enforced for lattice modes.
- **Code anchors**: `aborist/cli.py:_render_audit_label`.
- **Pinning tests**: `tests/test_cli_render.py` —
  `test_render_label_strict_in_claim_lattice_becomes_evidence_linked`
  and siblings.

### D8 — Automate only after the invariants are test-pinned

The five-step algorithm in CLAUDE.md applies to automation in
particular: bench harnesses, distill runners, ingest CLIs all earned
their automation only after the underlying ops were simplified into
single-purpose stages. **The bench harness IS the automation
substrate**, so it must enforce the directives before any feature
gets layered on top.

- **Status**: ongoing discipline.
- **Code anchors**: `bench/qa_sweep.py`,
  `tests/test_qa_quality_live.py`.
- **Bench signal**: per-row `directive_compliance` column maps
  each directive to a pass/fail boolean; the markdown summary
  aggregates per-mode-per-directive coverage.

## How to use this doc

1. **Before opening a new ticket**, scan the seven directives. Which
   one does the work serve? Add the directive number to the ticket
   header (`Directive: D6`) so the design log threads back here.

2. **Before merging a feature**, audit it against D8: does a test
   pin the new invariant? If not, the feature is automation
   pretending to be code-level discipline.

3. **When reading a bench summary**, the per-directive coverage row
   tells you which directives are passing on the corpus today and
   which are pending implementation. Bench scores that climb without
   directive coverage climbing are graveyard-digging (step 4 of
   the five-step algorithm).

4. **When triaging a fixture failure**, identify which directive
   the fixture pins. A fixture that fails by name maps to a specific
   regression in the directive's structural invariant; an unnamed
   bench delta is just noise.

## Status snapshot at landing

| # | Directive                                    | Status     | Tickets         |
|---|----------------------------------------------|------------|-----------------|
| 1 | Stop making Hermes prove things              | ✓          |                 |
| 2 | Hermes emits pointer clauses                 | ✓          |                 |
| 3 | Build CTI internally                         | ✓          | #000002 (closed)|
| 4 | Bind retrieval map AND evidence map          | ✓          | #000001 (closed)|
| 5 | Verify pointers deterministically            | ✓          |                 |
| 6 | Anchor-class warrant before NLI              | ✓          | #000003 (closed)|
| 7 | Rename labels honestly                       | ✓          | #000005 (closed)|
| 8 | Automate only after test-pinning             | discipline | #000004 (closed)|

All seven structural directives now ✓:

  D1, D2, D5 — were ✓ at landing.
  D3 closed 2026-05-02 via #000002 (frame detector + polarity
    preamble + renderer extension; multi-frame answers on Orwell
    and other reference-frame queries).
  D4 closed 2026-05-02 via #000001 (retrieval_plan_hash binding
    in the run-DAG; SQL column + audit events deferred).
  D6 closed 2026-05-02 via #000003 (anchor-class warrant
    generalization: entity-list / count / why-cause shapes).
  D7 closed 2026-05-02 via #000005 (four-rung ladder:
    POINTER-LINKED → ANCHOR-WARRANTED → EVIDENCE-WARRANTED →
    ENTAILMENT-VERIFIED).

D8 is the meta-discipline that gates how all of the above land;
the bench coverage substrate (#000004) is closed. Tests: 734
passed.

## Anti-regression test layer

Each directive has a structural pin in `tests/test_directives.py`.
A future PR that silently weakens a directive fails by name:

| Directive | Pin tests in `tests/test_directives.py`                                  |
|-----------|---------------------------------------------------------------------------|
| D1        | `test_d1_verify_*_signature_has_no_chat_client`, `test_d1_verifier_method_enum_excludes_llm_judges` |
| D2        | `test_d2_answer_modes_include_lattice_variants`, `test_d2_pointer_parser_exists_*` |
| D3        | `test_d3_runtime_owns_claim_lattice_construction`, `test_d3_evidence_map_built_*` |
| D4        | `test_d4_run_dag_carries_evidence_map_root` (✓), `test_d4_retrieval_plan_binding_status` (marker — flips when #000001 lands) |
| D5        | `test_d5_verify_claim_lattice_is_deterministic`, `test_d5_seven_hard_checks_are_pure_functions` |
| D6        | `test_d6_warrant_*` (relation + date), `test_d6_warrant_generalization_status` (marker — flips when #000003 lands) |
| D7        | `test_d7_renderer_*`, `test_d7_audit_mode_enum_canonical_set`        |
| D8        | `test_d8_seven_point_program_doc_exists`, `test_d8_bench_directive_compliance_helper_exists`, `test_d8_tickets_index_exists_and_pins_open_directives` |

The "marker" tests for D4 and D6 document open work: today they
assert that a parameter / detector is **absent**; when the
corresponding ticket lands, the assertion flips to **present**. The
test failing during ticket implementation is the expected signal,
not a regression.