arborist/docs/seven-point-program.md

# Seven-point program — the north-star directive

**Distilled 2026-05-01, Asia/Kuala_Lumpur**:

> Stop making Hermes prove things. Make Hermes emit pointer clauses;
> build CTI internally; bind the retrieval map and evidence map;
> verify pointers deterministically; add general anchor-class warrant
> before semantic NLI; rename labels honestly; and automate only after
> these invariants are test-pinned.

This is the architecture's north-star. Every new feature, prompt edit,
schema change, and ticket walks past these seven directives before
landing. The discipline frames how `bench/qa_sweep.py` and
`tests/test_qa_quality_live.py` cover the substrate: the bench is the
scoreboard, the live fixtures are the gates, and **this doc names what
the scoreboard and gates measure for**.

## The seven directives

### D1 — Stop making Hermes prove things

The verifier never calls the LLM. Faithfulness classification is a
deterministic property of (answer text, evidence map, source corpus),
not a model self-grade.

- **Status**: ✓ enforced.
- **Code anchors**:
  `arborist/qa/verify.py:verify_quotes`,
  `verify_claim_lattice`, `verify_claim_lattice_json` — pure functions
  with no chat-client parameter.
- **Pinning tests**: `tests/test_verify*.py`,
  `tests/test_claim_lattice.py` — every verifier path runs without a
  network call.
- **Smell-test**: if a future PR adds a `chat_client` argument to a
  verifier, D1 violated.

### D2 — Make Hermes emit pointer clauses

The model's prose surface is *citation-style* prose with bracketed
evidence-id tags (`Claim. [E12]`) or schema-constrained JSON
(`{"claims":[{"text":"...", "evidence_ids":["E12"]}]}`). The model
never types the quote string itself.

- **Status**: ✓ enforced for lattice modes.
- **Code anchors**:
  `arborist/qa/parse_claims.py:parse_pointer_claims`,
  `arborist/qa/verify.py:CLAIM_LATTICE_JSON_SCHEMA`,
  `arborist/qa/runner.py` (system prompts).
- **Pinning tests**: `tests/test_claim_lattice.py`,
  `tests/test_verify_json.py`.
- **Bench signal**: rows with `answer_mode ∈
  {claim_lattice_pointer, claim_lattice}` fall under this directive;
  rows with `answer_mode == "quote"` predate it (legacy).

### D3 — Build CTI internally

Clause-tree intelligence (frame lattice + multi-frame answer
compilation) lives **on our side of the wire**. The model proposes
clauses; the runtime composes them. Reference-frame queries get
multiple frames acknowledged in the answer (literal vs
fictional-actual vs in-universe-propaganda) rather than picking one
frame and discarding the others.

- **Status**: ¾ — phrase-route closed the retrieval side
  (commit `1b8677d`); pre-answer preflight contract (#000008
  quantifier classifier + #000010 metacognition QuestionState)
  landed 2026-05-03. Answer-side multi-frame compilation still
  pending.
- **Code anchors**: `arborist/qa/quantifier.py:classify_question_quantifier`
  (#000008), `arborist/qa/metacognition.py:preflight_question`
  (#000010), `arborist/qa/dag.py:preflight_node_hash` /
  `build_preflight_node_payload` (#000009 — five nested CTI
  clauses bind into the run-DAG `preflight` stage). Future:
  runtime-side polarity contract emission for multi-frame answers.
- **Pinning tests**: `tests/test_quantifier_classifier.py`
  (10-rung intensity ladder, 70 tests), `tests/test_metacognition.py`
  (4 detectors + governance + audit-line tails, 68 tests),
  `tests/test_dag.py::test_preflight_*` (DAG binding, 9 tests).
  Future: per-shape frame-detector tests + multi-frame answer-shape
  live fixtures.
- **Open ticket**:
  [#000002 Reference-Frame Polarity Contract](ticket-000002-reference-frame-polarity-contract.md).
- **Bench signal**: lattice-mode bench rows now carry
  `quantifier_intensity`, `quantifier_matched_token`,
  `scope_bound_hint`, `preflight_logical_statuses`,
  `preflight_question_shape`, `preflight_result`. Audit-line tails
  surface five metacog kinds: `· false premise`, `· contradictory`,
  `· stale risk`, `· out of corpus`, `· frame ambiguous`.

### D4 — Bind the retrieval map AND the evidence map

Provenance binding covers both *what got retrieved* (sources) and
*how retrieval got there* (the operator-influenceable inputs:
keywords, top_k, over_fetch, max_context_chars, shard set). Plus
the **preflight contract** governing the run (#000009 — guard /
cap / reminder / reject decisions hash-bound).

- **Status**: ¾ — `evidence_map_root` lives in the run-DAG
  (`arborist/qa/dag.py`); `retrieval_plan_hash` landed via #000001.
  Preflight stage (#000009) added the upstream control commitment:
  classifier output + answer/prompt/evidence contracts +
  `governance_policy_hash` / `model_profile_hash` references all
  fold into a single `preflight` Merkle leaf. Quote mode now 7→8
  stages; pointer-mode CTI 9→10 stages; reject-broad path emits a
  3-stage minimal DAG (`question → preflight → final_label`).
- **Code anchors**: `arborist/qa/dag.py:build_run_dag` (legacy +
  preflight-extended shapes), `build_reject_run_dag` (3-stage),
  `preflight_node_hash` / `build_preflight_node_payload` (5-clause
  payload), `arborist/qa/evidence.py:evidence_map_root`.
- **Pinning tests**: `tests/test_dag.py` (24 tests including
  preflight stage + reject path), `tests/test_evidence.py`.
- **Open ticket**: [#000001 Retrieval-keywords audit gap](ticket-000001-retrieval-keywords-audit-gap.md)
  closed; future retrieval-side refinement still possible.
- **Bench signal**: every lattice-mode row has `run_dag_root`
  populated. Two cache rows that share the same question + same
  model output + same verifier verdict but different preflight
  policy state now produce different `run_dag_root` values —
  audit replay can pin the policy decision per row.

### D5 — Verify pointers deterministically

The hard verifier runs only deterministic checks (parser succeeded,
evidence_id resolves, source_role allowed, claim text non-empty,
citation coverage, pointer-count cap, lexical anchor warrant).
No NLI, no embeddings, no model self-grading.

- **Status**: ✓ enforced.
- **Code anchors**: `arborist/qa/verify.py:verify_claim_lattice` —
  seven hard checks documented inline.
- **Pinning tests**: `tests/test_claim_lattice.py` (per-rule
  coverage), `tests/test_verify_json.py`,
  `tests/test_verify.py`.
- **Bench signal**: every row carries `verifier_method ∈ {quote,
  span, entity, paraphrase, claim_lattice, none}` (CHECK constraint
  in `arborist/store.py`).

### D6 — General anchor-class warrant before semantic NLI

The lexical anchor check (Rule 7 in `verify_claim_lattice`) catches
the lazy-anchor failure where pointer / role / coverage all pass but
the cited chunk never names the answer entity. Today this fires
only on relation-shape questions. Generalizing the anchor-class
detector to entity-list / why-cause / when-date shapes pins a
broader class of failures structurally before any NLI substrate
exists in the proof path.

- **Status**: ½ — relation-shape only. Entity-list / why-cause /
  when-date shapes pass through unchecked.
- **Code anchors**:
  `arborist/qa/verify.py:_extract_anchor_candidates`,
  `arborist/qa/verify.py:_question_is_relation_shape`.
- **Pinning tests**: `tests/test_claim_lattice.py` covers
  relation-shape; entity-list / cause / date shapes have no warrant
  test today.
- **Open ticket**:
  [#000003 Anchor-class warrant generalization](ticket-000003-anchor-class-warrant.md).
- **Bench signal**: future column `warrant_fired: bool` (true when
  Rule 7 ran on the row, regardless of pass/fail).

### D7 — Rename labels honestly

`STRICT` for claim-lattice modes overclaims because synthesis-heavy
claims pass all hard checks without semantic entailment of the
joined assertion. The renderer relabels at display time:

```
STRICT     → EVIDENCE-LINKED · via claim_lattice
HYBRID     → EVIDENCE-LINKED-PARTIAL · via claim_lattice_pointer
UNGROUNDED → UNGROUNDED · via claim_lattice
```

Quote / span / entity / paraphrase modes keep their original tokens
(they verify against pinned spans, not synthesis). The schema column
stays `STRICT`/`HYBRID`/`UNGROUNDED` so v9.8 cache_key invariants
hold.

- **Status**: ✓ enforced for lattice modes.
- **Code anchors**: `arborist/cli.py:_render_audit_label`.
- **Pinning tests**: `tests/test_cli_render.py` —
  `test_render_label_strict_in_claim_lattice_becomes_evidence_linked`
  and siblings.

### D8 — Automate only after the invariants are test-pinned

The five-step algorithm in CLAUDE.md applies to automation in
particular: bench harnesses, distill runners, ingest CLIs all earned
their automation only after the underlying ops were simplified into
single-purpose stages. **The bench harness IS the automation
substrate**, so it must enforce the directives before any feature
gets layered on top.

- **Status**: ongoing discipline.
- **Code anchors**: `bench/qa_sweep.py`,
  `tests/test_qa_quality_live.py`.
- **Bench signal**: per-row `directive_compliance` column maps
  each directive to a pass/fail boolean; the markdown summary
  aggregates per-mode-per-directive coverage.

## How to use this doc

1. **Before opening a new ticket**, scan the seven directives. Which
   one does the work serve? Add the directive number to the ticket
   header (`Directive: D6`) so the design log threads back here.

2. **Before merging a feature**, audit it against D8: does a test
   pin the new invariant? If not, the feature is automation
   pretending to be code-level discipline.

3. **When reading a bench summary**, the per-directive coverage row
   tells you which directives are passing on the corpus today and
   which are pending implementation. Bench scores that climb without
   directive coverage climbing are graveyard-digging (step 4 of
   the five-step algorithm).

4. **When triaging a fixture failure**, identify which directive
   the fixture pins. A fixture that fails by name maps to a specific
   regression in the directive's structural invariant; an unnamed
   bench delta is just noise.

## Status snapshot at landing

| # | Directive                                    | Status     | Tickets                       |
|---|----------------------------------------------|------------|-------------------------------|
| 1 | Stop making Hermes prove things              | ✓          | #000010 (closed)              |
| 2 | Hermes emits pointer clauses                 | ✓          |                               |
| 3 | Build CTI internally                         | ✓          | #000002, #000008, #000010 (closed) |
| 4 | Bind retrieval map AND evidence map          | ✓          | #000001, #000009 (closed)     |
| 5 | Verify pointers deterministically            | ✓          |                               |
| 6 | Anchor-class warrant before NLI              | ✓          | #000003 (closed)              |
| 7 | Rename labels honestly                       | ✓          | #000005 (closed)              |
| 8 | Automate only after test-pinning             | discipline | #000004 (closed)              |

All seven structural directives now ✓:

  D1, D2, D5 — were ✓ at landing.
  D3 closed 2026-05-02 via #000002 (frame detector + polarity
    preamble + renderer extension; multi-frame answers on Orwell
    and other reference-frame queries).
  D4 closed 2026-05-02 via #000001 (retrieval_plan_hash binding
    in the run-DAG; SQL column + audit events deferred).
  D6 closed 2026-05-02 via #000003 (anchor-class warrant
    generalization: entity-list / count / why-cause shapes).
  D7 closed 2026-05-02 via #000005 (four-rung ladder:
    POINTER-LINKED → ANCHOR-WARRANTED → EVIDENCE-WARRANTED →
    ENTAILMENT-VERIFIED).

D8 is the meta-discipline that gates how all of the above land;
the bench coverage substrate (#000004) is closed. Tests: 734
passed at the original snapshot.

### Post-landing addendum (2026-05-03 / 2026-05-04)

Three additional tickets landed extending D1, D3, and D4:

  - **#000008** (closed `4f2b5a6`, 2026-05-03) — Broad-quantifier
    preflight guard. New modules `arborist/qa/quantifier.py`,
    `model_profiles.py`, `quantifier_reminder.py`. Seven new
    policy fields fold into `governance_policy_hash`; six-level
    disable hierarchy. Defaults flipped (after n=5 verification):
    reminder default-on for lattice modes, cap default operator-
    opt-in with JSON-only allowlist.

  - **#000010** (closed, 2026-05-03) — Meta-Cognition Preflight
    Guard. New module `arborist/qa/metacognition.py` with
    `QuestionState` dataclass + four pure detectors (temporal,
    contradiction, false-premise-lite, out-of-corpus). Six new
    policy fields, all default-on except
    `metacognition_block_on_contradiction`. Audit-line tail
    tokens: `· false premise`, `· contradictory`, `· stale risk`,
    `· out of corpus`, `· frame ambiguous`. Closes D1 by giving
    the runtime explicit epistemic state to refuse from instead
    of asking Hermes to self-regulate.

  - **#000009** (closed `111dda6`, 2026-05-04) — Preflight run-DAG
    node binding. Inserts `preflight` stage between `question` and
    `retrieval`. Quote mode 7→8 stages; pointer-mode CTI 9→10
    stages. Reject-broad early-return path emits a 3-stage minimal
    DAG (`question → preflight → final_label`). Five nested CTI
    clauses (classifier / answer_contract / prompt_contract /
    evidence_contract / policy_refs) inside a single hashable
    leaf. Closes D4 by Merkle-binding the upstream control
    decisions that previously only appeared on result dicts.

Test count after these landings: **993 passing** (up from 734;
259 new across the three tickets). All seven directives remain ✓.

**Bench-validated post-landing** (full 75-question sweep
2026-05-03T23-30-12Z vs frozen 2026-05-02T15-07Z baseline,
n=3 × 3 modes = 675 runs):

  - quote SR  0.54 → 0.52   (−2pp, within noise)
  - pointer SR 0.20 → 0.21  (+1pp, within noise)
  - JSON SR    0.42 → 0.44  (+2pp, within noise)

All STRICT-rate deltas within the 5pp signal floor. Substrate-
level wins: pointer-mode FORMAT_COLLAPSED eliminated globally
(0/225), pointer-mode NO_EVIDENCE_POINTER drops from 33%
(broad-only when reminder was off) to 13% (full sweep with
reminder on). The default flip is strictly more honest without
regressing throughput on narrow questions. See
`docs/qa-modes-bench.md` Addendum 3 for the full comparison.

## Anti-regression test layer

Each directive has a structural pin in `tests/test_directives.py`.
A future PR that silently weakens a directive fails by name:

| Directive | Pin tests in `tests/test_directives.py`                                  |
|-----------|---------------------------------------------------------------------------|
| D1        | `test_d1_verify_*_signature_has_no_chat_client`, `test_d1_verifier_method_enum_excludes_llm_judges` |
| D2        | `test_d2_answer_modes_include_lattice_variants`, `test_d2_pointer_parser_exists_*` |
| D3        | `test_d3_runtime_owns_claim_lattice_construction`, `test_d3_evidence_map_built_*` |
| D4        | `test_d4_run_dag_carries_evidence_map_root` (✓), `test_d4_retrieval_plan_binding_status` (marker — flips when #000001 lands) |
| D5        | `test_d5_verify_claim_lattice_is_deterministic`, `test_d5_seven_hard_checks_are_pure_functions` |
| D6        | `test_d6_warrant_*` (relation + date), `test_d6_warrant_generalization_status` (marker — flips when #000003 lands) |
| D7        | `test_d7_renderer_*`, `test_d7_audit_mode_enum_canonical_set`        |
| D8        | `test_d8_seven_point_program_doc_exists`, `test_d8_bench_directive_compliance_helper_exists`, `test_d8_tickets_index_exists_and_pins_open_directives` |

The "marker" tests for D4 and D6 document open work: today they
assert that a parameter / detector is **absent**; when the
corresponding ticket lands, the assertion flips to **present**. The
test failing during ticket implementation is the expected signal,
not a regression.