arborist/docs/seven-point-program.md

Seven-point program — the north-star directive

Distilled 2026-05-01, Asia/Kuala_Lumpur:

Stop making Hermes prove things. Make Hermes emit pointer clauses; build CTI internally; bind the retrieval map and evidence map; verify pointers deterministically; add general anchor-class warrant before semantic NLI; rename labels honestly; and automate only after these invariants are test-pinned.

This is the architecture's north-star. Every new feature, prompt edit, schema change, and ticket walks past these seven directives before landing. The discipline frames how bench/qa_sweep.py and tests/test_qa_quality_live.py cover the substrate: the bench is the scoreboard, the live fixtures are the gates, and this doc names what the scoreboard and gates measure for.

The seven directives

D1 — Stop making Hermes prove things

The verifier never calls the LLM. Faithfulness classification is a deterministic property of (answer text, evidence map, source corpus), not a model self-grade.

• Status: ✓ enforced.
• Code anchors: arborist/qa/verify.py:verify_quotes, verify_claim_lattice, verify_claim_lattice_json — pure functions with no chat-client parameter.
• Pinning tests: tests/test_verify*.py, tests/test_claim_lattice.py — every verifier path runs without a network call.
• Smell-test: if a future PR adds a chat_client argument to a verifier, D1 violated.

D2 — Make Hermes emit pointer clauses

The model's prose surface is citation-style prose with bracketed evidence-id tags (Claim. [E12]) or schema-constrained JSON ({"claims":[{"text":"...", "evidence_ids":["E12"]}]}). The model never types the quote string itself.

• Status: ✓ enforced for lattice modes.
• Code anchors: arborist/qa/parse_claims.py:parse_pointer_claims, arborist/qa/verify.py:CLAIM_LATTICE_JSON_SCHEMA, arborist/qa/runner.py (system prompts).
• Pinning tests: tests/test_claim_lattice.py, tests/test_verify_json.py.
• Bench signal: rows with answer_mode ∈ {claim_lattice_pointer, claim_lattice} fall under this directive; rows with answer_mode == "quote" predate it (legacy).

D3 — Build CTI internally

Clause-tree intelligence (frame lattice + multi-frame answer compilation) lives on our side of the wire. The model proposes clauses; the runtime composes them. Reference-frame queries get multiple frames acknowledged in the answer (literal vs fictional-actual vs in-universe-propaganda) rather than picking one frame and discarding the others.

• Status: ¾ — phrase-route closed the retrieval side (commit 1b8677d); pre-answer preflight contract (#000008 quantifier classifier + #000010 metacognition QuestionState) landed 2026-05-03. Answer-side multi-frame compilation still pending.
• Code anchors: arborist/qa/quantifier.py:classify_question_quantifier (#000008), arborist/qa/metacognition.py:preflight_question (#000010), arborist/qa/dag.py:preflight_node_hash / build_preflight_node_payload (#000009 — five nested CTI clauses bind into the run-DAG preflight stage). Future: runtime-side polarity contract emission for multi-frame answers.
• Pinning tests: tests/test_quantifier_classifier.py (10-rung intensity ladder, 70 tests), tests/test_metacognition.py (4 detectors + governance + audit-line tails, 58 tests), tests/test_dag.py::test_preflight_* (DAG binding; subset of the test_dag.py suite tagged below). Future: per-shape frame-detector tests + multi-frame answer-shape live fixtures.
• Open ticket: #000002 Reference-Frame Polarity Contract.
• Bench signal: lattice-mode bench rows now carry quantifier_intensity, quantifier_matched_token, scope_bound_hint, preflight_logical_statuses, preflight_question_shape, preflight_result. Audit-line tails surface five metacog kinds: · false premise, · contradictory, · stale risk, · out of corpus, · frame ambiguous.

D4 — Bind the retrieval map AND the evidence map

Provenance binding covers both what got retrieved (sources) and how retrieval got there (the operator-influenceable inputs: keywords, top_k, over_fetch, max_context_chars, shard set). Plus the preflight contract governing the run (#000009 — guard / cap / reminder / reject decisions hash-bound).

• Status: ¾ — evidence_map_root lives in the run-DAG (arborist/qa/dag.py); retrieval_plan_hash landed via #000001. Preflight stage (#000009) added the upstream control commitment: classifier output + answer/prompt/evidence contracts + governance_policy_hash / model_profile_hash references all fold into a single preflight Merkle leaf. Quote mode now 7→8 stages; pointer-mode CTI 9→10 stages; reject-broad path emits a 3-stage minimal DAG (question → preflight → final_label).
• Code anchors: arborist/qa/dag.py:build_run_dag (legacy + preflight-extended shapes), build_reject_run_dag (3-stage), preflight_node_hash / build_preflight_node_payload (5-clause payload), arborist/qa/evidence.py:evidence_map_root.
• Pinning tests: tests/test_dag.py (33 tests including preflight stage + reject path), tests/test_evidence.py.
• Open ticket: #000001 Retrieval-keywords audit gap closed; future retrieval-side refinement still possible.
• Bench signal: every lattice-mode row has run_dag_root populated. Two cache rows that share the same question + same model output + same verifier verdict but different preflight policy state now produce different run_dag_root values — audit replay can pin the policy decision per row.

D5 — Verify pointers deterministically

The hard verifier runs only deterministic checks (parser succeeded, evidence_id resolves, source_role allowed, claim text non-empty, citation coverage, pointer-count cap, lexical anchor warrant). No NLI, no embeddings, no model self-grading.

• Status: ✓ enforced.
• Code anchors: arborist/qa/verify.py:verify_claim_lattice — seven hard checks documented inline.
• Pinning tests: tests/test_claim_lattice.py (per-rule coverage), tests/test_verify_json.py, tests/test_verify.py.
• Bench signal: every row carries verifier_method ∈ {quote, span, entity, paraphrase, claim_lattice, none} (CHECK constraint in arborist/store.py).

D6 — General anchor-class warrant before semantic NLI

The lexical anchor check (Rule 7 in verify_claim_lattice) catches the lazy-anchor failure where pointer / role / coverage all pass but the cited chunk never names the answer entity. Today this fires only on relation-shape questions. Generalizing the anchor-class detector to entity-list / why-cause / when-date shapes pins a broader class of failures structurally before any NLI substrate exists in the proof path.

• Status: ½ — relation-shape only. Entity-list / why-cause / when-date shapes pass through unchecked.
• Code anchors: arborist/qa/verify.py:_extract_anchor_candidates, arborist/qa/verify.py:_question_is_relation_shape.
• Pinning tests: tests/test_claim_lattice.py covers relation-shape; entity-list / cause / date shapes have no warrant test today.
• Open ticket: #000003 Anchor-class warrant generalization.
• Bench signal: future column warrant_fired: bool (true when Rule 7 ran on the row, regardless of pass/fail).

D7 — Rename labels honestly

STRICT for claim-lattice modes overclaims because synthesis-heavy claims pass all hard checks without semantic entailment of the joined assertion. The renderer relabels at display time:

STRICT     → EVIDENCE-LINKED · via claim_lattice
HYBRID     → EVIDENCE-LINKED-PARTIAL · via claim_lattice_pointer
UNGROUNDED → UNGROUNDED · via claim_lattice

Quote / span / entity / paraphrase modes keep their original tokens (they verify against pinned spans, not synthesis). The schema column stays STRICT/HYBRID/UNGROUNDED so v9.8 cache_key invariants hold.

• Status: ✓ enforced for lattice modes.
• Code anchors: arborist/cli.py:_render_audit_label.
• Pinning tests: tests/test_cli_render.py — test_render_label_strict_in_claim_lattice_becomes_evidence_linked and siblings.

D8 — Automate only after the invariants are test-pinned

The five-step algorithm in CLAUDE.md applies to automation in particular: bench harnesses, distill runners, ingest CLIs all earned their automation only after the underlying ops were simplified into single-purpose stages. The bench harness IS the automation substrate, so it must enforce the directives before any feature gets layered on top.

• Status: ongoing discipline.
• Code anchors: bench/qa_sweep.py, tests/test_qa_quality_live.py.
• Bench signal: per-row directive_compliance column maps each directive to a pass/fail boolean; the markdown summary aggregates per-mode-per-directive coverage.

How to use this doc

1. Before opening a new ticket, scan the seven directives. Which one does the work serve? Add the directive number to the ticket header (Directive: D6) so the design log threads back here.

2. Before merging a feature, audit it against D8: does a test pin the new invariant? If not, the feature is automation pretending to be code-level discipline.

3. When reading a bench summary, the per-directive coverage row tells you which directives are passing on the corpus today and which are pending implementation. Bench scores that climb without directive coverage climbing are graveyard-digging (step 4 of the five-step algorithm).

4. When triaging a fixture failure, identify which directive the fixture pins. A fixture that fails by name maps to a specific regression in the directive's structural invariant; an unnamed bench delta is just noise.

Status snapshot at landing

#	Directive	Status	Tickets
1	Stop making Hermes prove things	✓	#000010 (closed)
2	Hermes emits pointer clauses	✓
3	Build CTI internally	✓	#000002, #000008, #000010 (closed)
4	Bind retrieval map AND evidence map	✓	#000001, #000009 (closed)
5	Verify pointers deterministically	✓
6	Anchor-class warrant before NLI	✓	#000003 (closed)
7	Rename labels honestly	✓	#000005 (closed)
8	Automate only after test-pinning	discipline	#000004 (closed)

All seven structural directives now ✓:

D1, D2, D5 — were ✓ at landing. D3 closed 2026-05-02 via #000002 (frame detector + polarity preamble + renderer extension; multi-frame answers on Orwell and other reference-frame queries). D4 closed 2026-05-02 via #000001 (retrieval_plan_hash binding in the run-DAG; SQL column + audit events deferred). D6 closed 2026-05-02 via #000003 (anchor-class warrant generalization: entity-list / count / why-cause shapes). D7 closed 2026-05-02 via #000005 (four-rung ladder: POINTER-LINKED → ANCHOR-WARRANTED → EVIDENCE-WARRANTED → ENTAILMENT-VERIFIED).

D8 is the meta-discipline that gates how all of the above land; the bench coverage substrate (#000004) is closed. Tests: 734 passed at the original snapshot.

Post-landing addendum (2026-05-03 / 2026-05-04)

Three additional tickets landed extending D1, D3, and D4:

• #000008 (closed 4f2b5a6, 2026-05-03) — Broad-quantifier preflight guard. New modules arborist/qa/quantifier.py, model_profiles.py, quantifier_reminder.py. Seven new policy fields fold into governance_policy_hash; six-level disable hierarchy. Defaults flipped (after n=5 verification): reminder default-on for lattice modes, cap default operator- opt-in with JSON-only allowlist.

• #000010 (closed, 2026-05-03) — Meta-Cognition Preflight Guard. New module arborist/qa/metacognition.py with QuestionState dataclass + four pure detectors (temporal, contradiction, false-premise-lite, out-of-corpus). Six new policy fields, all default-on except metacognition_block_on_contradiction. Audit-line tail tokens: · false premise, · contradictory, · stale risk, · out of corpus, · frame ambiguous. Closes D1 by giving the runtime explicit epistemic state to refuse from instead of asking Hermes to self-regulate.

• #000009 (closed 111dda6, 2026-05-04) — Preflight run-DAG node binding. Inserts preflight stage between question and retrieval. Quote mode 7→8 stages; pointer-mode CTI 9→10 stages. Reject-broad early-return path emits a 3-stage minimal DAG (question → preflight → final_label). Five nested CTI clauses (classifier / answer_contract / prompt_contract / evidence_contract / policy_refs) inside a single hashable leaf. Closes D4 by Merkle-binding the upstream control decisions that previously only appeared on result dicts.

Test count after these landings: 993 passing (up from 734; 259 new across the three tickets). All seven directives remain ✓.

Bench-validated post-landing (full 75-question sweep 2026-05-03T23-30-12Z vs frozen 2026-05-02T15-07Z baseline, n=3 × 3 modes = 675 runs):

• quote SR 0.54 → 0.52 (−2pp, within noise)
• pointer SR 0.20 → 0.21 (+1pp, within noise)
• JSON SR 0.42 → 0.44 (+2pp, within noise)

All STRICT-rate deltas within the 5pp signal floor. Substrate- level wins: pointer-mode FORMAT_COLLAPSED eliminated globally (0/225), pointer-mode NO_EVIDENCE_POINTER drops from 33% (broad-only when reminder was off) to 13% (full sweep with reminder on). The default flip is strictly more honest without regressing throughput on narrow questions. See docs/qa-modes-bench.md Addendum 3 for the full comparison.

Anti-regression test layer

Each directive has a structural pin in tests/test_directives.py. A future PR that silently weakens a directive fails by name:

Directive	Pin tests in tests/test_directives.py
D1	test_d1_verify_*_signature_has_no_chat_client, test_d1_verifier_method_enum_excludes_llm_judges
D2	test_d2_answer_modes_include_lattice_variants, test_d2_pointer_parser_exists_*
D3	test_d3_runtime_owns_claim_lattice_construction, test_d3_evidence_map_built_*
D4	test_d4_run_dag_carries_evidence_map_root (✓), test_d4_retrieval_plan_binding_status (marker — flips when #000001 lands)
D5	test_d5_verify_claim_lattice_is_deterministic, test_d5_seven_hard_checks_are_pure_functions
D6	test_d6_warrant_* (relation + date), test_d6_warrant_generalization_status (marker — flips when #000003 lands)
D7	test_d7_renderer_*, test_d7_audit_mode_enum_canonical_set
D8	test_d8_seven_point_program_doc_exists, test_d8_bench_directive_compliance_helper_exists, test_d8_tickets_index_exists_and_pins_open_directives

The "marker" tests for D4 and D6 document open work: today they assert that a parameter / detector is absent; when the corresponding ticket lands, the assertion flips to present. The test failing during ticket implementation is the expected signal, not a regression.