arborist/docs/seven-point-program.md

Seven-point program — the north-star directive

Distilled 2026-05-01, Asia/Kuala_Lumpur:

Stop making Hermes prove things. Make Hermes emit pointer clauses; build CTI internally; bind the retrieval map and evidence map; verify pointers deterministically; add general anchor-class warrant before semantic NLI; rename labels honestly; and automate only after these invariants are test-pinned.

This is the architecture's north-star. Every new feature, prompt edit, schema change, and ticket walks past these seven directives before landing. The discipline frames how bench/qa_sweep.py and tests/test_qa_quality_live.py cover the substrate: the bench is the scoreboard, the live fixtures are the gates, and this doc names what the scoreboard and gates measure for.

The seven directives

D1 — Stop making Hermes prove things

The verifier never calls the LLM. Faithfulness classification is a deterministic property of (answer text, evidence map, source corpus), not a model self-grade.

• Status: ✓ enforced.
• Code anchors: aborist/qa/verify.py:verify_quotes, verify_claim_lattice, verify_claim_lattice_json — pure functions with no chat-client parameter.
• Pinning tests: tests/test_verify*.py, tests/test_claim_lattice.py — every verifier path runs without a network call.
• Smell-test: if a future PR adds a chat_client argument to a verifier, D1 violated.

D2 — Make Hermes emit pointer clauses

The model's prose surface is citation-style prose with bracketed evidence-id tags (Claim. [E12]) or schema-constrained JSON ({"claims":[{"text":"...", "evidence_ids":["E12"]}]}). The model never types the quote string itself.

• Status: ✓ enforced for lattice modes.
• Code anchors: aborist/qa/parse_claims.py:parse_pointer_claims, aborist/qa/verify.py:CLAIM_LATTICE_JSON_SCHEMA, aborist/qa/runner.py (system prompts).
• Pinning tests: tests/test_claim_lattice.py, tests/test_verify_json.py.
• Bench signal: rows with answer_mode ∈ {claim_lattice_pointer, claim_lattice} fall under this directive; rows with answer_mode == "quote" predate it (legacy).

D3 — Build CTI internally

Clause-tree intelligence (frame lattice + multi-frame answer compilation) lives on our side of the wire. The model proposes clauses; the runtime composes them. Reference-frame queries get multiple frames acknowledged in the answer (literal vs fictional-actual vs in-universe-propaganda) rather than picking one frame and discarding the others.

• Status: ½ — phrase-route closed the retrieval side (commit 1b8677d); answer-side compilation pending.
• Code anchors: today, none. Future: aborist/qa/frame.py:detect_frame, runtime-side polarity contract emission.
• Pinning tests: today, only the phrase-route tests in test_query.py::test_phrase_match_surfaces_topical_doc. Future: per-shape frame-detector tests + multi-frame answer-shape live fixtures.
• Open ticket: #000002 Reference-Frame Polarity Contract.
• Bench signal: today, phrase_match_surfaced flag on bench rows (true when phrase route fired). Once Module L lands, additionally frame_kind ∈ {literal, reference, ambiguous}.

D4 — Bind the retrieval map AND the evidence map

Provenance binding covers both what got retrieved (sources) and how retrieval got there (the operator-influenceable inputs: keywords, top_k, over_fetch, max_context_chars, shard set).

• Status: ½ — evidence_map_root lives in the 9-stage run-DAG (aborist/qa/dag.py); retrieval-plan binding (the input side) is the open work.
• Code anchors: aborist/qa/dag.py:build_run_dag, aborist/qa/evidence.py:evidence_map_root.
• Pinning tests: tests/test_qa_dag.py, tests/test_evidence.py.
• Open ticket: #000001 Retrieval-keywords audit gap.
• Bench signal: every lattice-mode row has run_dag_root populated; future bench column will report on retrieval_plan_hash presence once #000001 lands.

D5 — Verify pointers deterministically

The hard verifier runs only deterministic checks (parser succeeded, evidence_id resolves, source_role allowed, claim text non-empty, citation coverage, pointer-count cap, lexical anchor warrant). No NLI, no embeddings, no model self-grading.

• Status: ✓ enforced.
• Code anchors: aborist/qa/verify.py:verify_claim_lattice — seven hard checks documented inline.
• Pinning tests: tests/test_claim_lattice.py (per-rule coverage), tests/test_verify_json.py, tests/test_verify.py.
• Bench signal: every row carries verifier_method ∈ {quote, span, entity, paraphrase, claim_lattice, none} (CHECK constraint in aborist/store.py).

D6 — General anchor-class warrant before semantic NLI

The lexical anchor check (Rule 7 in verify_claim_lattice) catches the lazy-anchor failure where pointer / role / coverage all pass but the cited chunk never names the answer entity. Today this fires only on relation-shape questions. Generalizing the anchor-class detector to entity-list / why-cause / when-date shapes pins a broader class of failures structurally before any NLI substrate exists in the proof path.

• Status: ½ — relation-shape only. Entity-list / why-cause / when-date shapes pass through unchecked.
• Code anchors: aborist/qa/verify.py:_extract_anchor_candidates, aborist/qa/verify.py:_question_is_relation_shape.
• Pinning tests: tests/test_claim_lattice.py covers relation-shape; entity-list / cause / date shapes have no warrant test today.
• Open ticket: #000003 Anchor-class warrant generalization.
• Bench signal: future column warrant_fired: bool (true when Rule 7 ran on the row, regardless of pass/fail).

D7 — Rename labels honestly

STRICT for claim-lattice modes overclaims because synthesis-heavy claims pass all hard checks without semantic entailment of the joined assertion. The renderer relabels at display time:

STRICT     → EVIDENCE-LINKED · via claim_lattice
HYBRID     → EVIDENCE-LINKED-PARTIAL · via claim_lattice_pointer
UNGROUNDED → UNGROUNDED · via claim_lattice

Quote / span / entity / paraphrase modes keep their original tokens (they verify against pinned spans, not synthesis). The schema column stays STRICT/HYBRID/UNGROUNDED so v9.8 cache_key invariants hold.

• Status: ✓ enforced for lattice modes.
• Code anchors: aborist/cli.py:_render_audit_label.
• Pinning tests: tests/test_cli_render.py — test_render_label_strict_in_claim_lattice_becomes_evidence_linked and siblings.

D8 — Automate only after the invariants are test-pinned

The five-step algorithm in CLAUDE.md applies to automation in particular: bench harnesses, distill runners, ingest CLIs all earned their automation only after the underlying ops were simplified into single-purpose stages. The bench harness IS the automation substrate, so it must enforce the directives before any feature gets layered on top.

• Status: ongoing discipline.
• Code anchors: bench/qa_sweep.py, tests/test_qa_quality_live.py.
• Bench signal: per-row directive_compliance column maps each directive to a pass/fail boolean; the markdown summary aggregates per-mode-per-directive coverage.

How to use this doc

1. Before opening a new ticket, scan the seven directives. Which one does the work serve? Add the directive number to the ticket header (Directive: D6) so the design log threads back here.

2. Before merging a feature, audit it against D8: does a test pin the new invariant? If not, the feature is automation pretending to be code-level discipline.

3. When reading a bench summary, the per-directive coverage row tells you which directives are passing on the corpus today and which are pending implementation. Bench scores that climb without directive coverage climbing are graveyard-digging (step 4 of the five-step algorithm).

4. When triaging a fixture failure, identify which directive the fixture pins. A fixture that fails by name maps to a specific regression in the directive's structural invariant; an unnamed bench delta is just noise.

Status snapshot at landing

#	Directive	Status	Tickets
1	Stop making Hermes prove things	✓
2	Hermes emits pointer clauses	✓
3	Build CTI internally	✓	#000002 (closed)
4	Bind retrieval map AND evidence map	✓	#000001 (closed)
5	Verify pointers deterministically	✓
6	Anchor-class warrant before NLI	✓	#000003 (closed)
7	Rename labels honestly	✓	#000005 (closed)
8	Automate only after test-pinning	discipline	#000004 (closed)

All seven structural directives now ✓:

D1, D2, D5 — were ✓ at landing. D3 closed 2026-05-02 via #000002 (frame detector + polarity preamble + renderer extension; multi-frame answers on Orwell and other reference-frame queries). D4 closed 2026-05-02 via #000001 (retrieval_plan_hash binding in the run-DAG; SQL column + audit events deferred). D6 closed 2026-05-02 via #000003 (anchor-class warrant generalization: entity-list / count / why-cause shapes). D7 closed 2026-05-02 via #000005 (four-rung ladder: POINTER-LINKED → ANCHOR-WARRANTED → EVIDENCE-WARRANTED → ENTAILMENT-VERIFIED).

D8 is the meta-discipline that gates how all of the above land; the bench coverage substrate (#000004) is closed. Tests: 734 passed.

Anti-regression test layer

Each directive has a structural pin in tests/test_directives.py. A future PR that silently weakens a directive fails by name:

Directive	Pin tests in tests/test_directives.py
D1	test_d1_verify_*_signature_has_no_chat_client, test_d1_verifier_method_enum_excludes_llm_judges
D2	test_d2_answer_modes_include_lattice_variants, test_d2_pointer_parser_exists_*
D3	test_d3_runtime_owns_claim_lattice_construction, test_d3_evidence_map_built_*
D4	test_d4_run_dag_carries_evidence_map_root (✓), test_d4_retrieval_plan_binding_status (marker — flips when #000001 lands)
D5	test_d5_verify_claim_lattice_is_deterministic, test_d5_seven_hard_checks_are_pure_functions
D6	test_d6_warrant_* (relation + date), test_d6_warrant_generalization_status (marker — flips when #000003 lands)
D7	test_d7_renderer_*, test_d7_audit_mode_enum_canonical_set
D8	test_d8_seven_point_program_doc_exists, test_d8_bench_directive_compliance_helper_exists, test_d8_tickets_index_exists_and_pins_open_directives

The "marker" tests for D4 and D6 document open work: today they assert that a parameter / detector is absent; when the corresponding ticket lands, the assertion flips to present. The test failing during ticket implementation is the expected signal, not a regression.