arborist/docs/tickets/ticket-000010-metacognition-preflight-guard.md

Ticket #000010 — Meta-Cognition Preflight Guard (M0 / MCTL)

Status: closed · landed 2026-05-03 (zero-shot, phases 1–4 complete; Phase 5 DAG binding joins #000009; cross-model bench deferred manual task) Opened: 2026-05-03 Scope: Build a runtime epistemic control layer that classifies a question's shape (false-premise, contradictory, under-specified, broad-quantifier, time-sensitive, out-of-corpus, reference-frame ambiguous) BEFORE generation. Forces Hermes to answer only through the resulting QuestionState constraints rather than from surface form alone. Audience: fox + future blackops shifts. Source: ~/Downloads/meta-cognition_for_hermes(1).txt (2026-05-03, Asia/Kuala_Lumpur). Hard constraint: No schema_version, canonicalization_version, or chunking_version bumps. Per #000008 §1: pure additive policy folding into governance_policy_hash. No LLM in hard preflight path — model-assisted preflight, if added, labels itself SOFT_PREFLIGHT_HINT and never produces a PREFLIGHT_OK / PREFLIGHT_BLOCKED without deterministic support (seven-point program D1 — verifier never calls LLM).

---

1. Premise

A model cannot reliably infer the limits of its own knowledge from inside generation. But a runtime can maintain explicit epistemic state — corpus boundaries, source roots, retrieval coverage, model profile, cutoff metadata, verifier policy, admissibility labels — and force every answer through that state.

The solution is not "ask Hermes to be more self-aware." The solution is to build a meta-cognition preflight + epistemic boundary guard around Hermes.

This fits the existing stack:

Hermes proposes.
CTI structures.
Reverse-RAG binds claims to evidence.
Merkle-AGI-DAG commits the trace.
PROMETHEUS-Σ governs admissibility.
Meta-Cognition Preflight decides whether the question is
answerable, ill-posed, stale, unbounded, contradictory, or
outside corpus before generation.

2. What meta-cognition means here

Not inner consciousness. A runtime control layer that answers these questions BEFORE the model answers the user:

• What kind of question is this?
• Is the question logically well-formed?
• Does it contain false or unproven premises?
• Is it bounded enough to answer?
• Does the current corpus contain the required evidence?
• Is the answer time-sensitive relative to model/source cutoff?
• Can the claim be grounded by retrieved evidence?
• Should the system answer, narrow scope, ask clarification, reject, or label uncertainty?

Call it M0 — Meta-Cognition Preflight Guard or in PROMETHEUS-Σ language, MCTL — Meta-Cognition Trigger Layer.

3. What cannot be solved completely

You cannot fully prevent every illogical question or trap because:

• Language is ambiguous;
• Unknown unknowns exist;
• Source corpora are incomplete;
• A false premise may be subtle;
• Retrieval may miss the correcting evidence;
• The model may know a true fact but the corpus may not support it;
• Some questions are undecidable, subjective, or under-specified;
• Time-sensitive facts can change after the corpus cutoff.

So the target is not "never be trapped." The target is:

When trapped, fail visibly, locally, reproducibly, and non-reusably.

In our labels:

ANSWERABLE
PARTIAL
UNSUPPORTED
OUT_OF_CORPUS
STALE_RISK
FALSE_PREMISE_SUSPECTED
CONTRADICTORY_QUESTION
UNDER_SPECIFIED
BROAD_QUANTIFIER_UNBOUNDED
REFERENCE_FRAME_AMBIGUOUS
NEEDS_CLARIFICATION

4. The core design

Pre-generation layer:

User question
→ Meta-Cognition Preflight
→ CTI question-state object (QuestionState)
→ retrieval plan
→ evidence map
→ Hermes pointer-claim generation
→ deterministic verification
→ anchor/warrant checks
→ final admissibility label
→ Merkle-DAG commit

The preflight does not answer the question. It classifies the question and sets constraints.

5. New object: QuestionState

from dataclasses import dataclass
from typing import Literal

LogicalStatus = Literal[
    "well_formed",
    "under_specified",
    "false_premise_suspected",
    "contradictory_question",
    "out_of_corpus_risk",
    "stale_risk",
    "reference_frame_ambiguous",
    "broad_quantifier_unbounded",
]

@dataclass(frozen=True)
class QuestionState:
    raw_question: str
    question_hash: str
    logical_statuses: list[LogicalStatus]
    question_shape: str
    quantifier_intensity: str | None
    reference_frames: list[str]
    temporal_sensitivity: str
    corpus_requirement: str
    known_boundaries: list[str]
    answer_constraints: dict
    preflight_policy_hash: str

This object is a CTI root node and gets Merkle-committed (Phase 5 follow-up; first pass surfaces it on the result dict only).

6. Detector classes

6.1 Quantifier (REUSE from #000008)

Already shipped as arborist/qa/quantifier.py. Returns intensity + scope_bound_hint. Meta-cognition wraps it; doesn't duplicate.

6.2 Temporal sensitivity

Lexical patterns: current, latest, today, now, as of, this year, plus role-shape patterns: CEO, president, prime minister, current champion, current price. Sets temporal_sensitivity ∈ {high, medium, low}. When high, cross- checks model_profile.cutoff and corpus_latest_timestamp against current_date and emits stale_risk if cutoff is older.

6.3 Contradiction detector (lexical pairs)

unmarried + spouse
always + sometimes not
never + did
only + also
before + after
married + unmarried
alive + dead

Conservative — only fires on clear lexical contradictions. False positives are operator-hostile.

6.4 False-premise detector (lite)

Presupposition patterns:

when did X stop Y?         → presupposes X did Y
why did X cause Y?         → presupposes X caused Y
who is X's Y?              → presupposes X has Y
which X is the real Y?     → presupposes a real Y exists
how did X become Y?        → presupposes X became Y

First pass: extract the implied relation as a required_evidence hint that the verifier can use. Don't BLOCK the question — just record false_premise_suspected so the audit-line can surface it.

6.5 Reference-frame detector

Reuse the existing frame-detection logic from ticket #000002 (arborist/qa/query.py:_detect_frame). Already classifies reference vs literal vs ambiguous. Meta-cognition wraps the output as reference_frames: list[str].

6.6 Out-of-corpus detector

Lexical patterns: my unpublished, my uploaded, the file I sent, what does it say about. When detected on a corpus that has no matching upload, sets corpus_status: "out_of_corpus". Conservative — defaults to "likely_in_corpus" for typical encyclopedic questions.

7. Implementation in Arborist

7.1 New module

arborist/qa/metacognition.py with:

def preflight_question(
    question: str,
    *,
    model_profile_id: str | None = None,
    corpus_profile: dict | None = None,
    policy: dict | None = None,
) -> QuestionState:
    ...

Pure and deterministic. Reuses the Phase 1 quantifier classifier plus 4 new lightweight detectors. No I/O, no model call, no retrieval call.

7.2 Wire into query() / runner.ask()

Same insertion pattern as the quantifier classifier: after policy resolution, before retrieval. Result dict surfaces the QuestionState so bench rows pick it up.

7.3 Policy fields (additive)

metacognition_enabled              True
metacognition_temporal_check       True
metacognition_contradiction_check  True
metacognition_false_premise_check  True
metacognition_out_of_corpus_check  True
metacognition_block_on_contradiction False (label-only by default)

All fold into _VERIFIER_POLICY_FIELDS so flipping any of them invalidates prior cache records.

7.4 Labels

Composable with existing audit-line ladder. Examples:

POINTER-LINKED · PREFLIGHT_OK
ANCHOR-WARRANTED · PREFLIGHT_OK
EVIDENCE-WARRANTED · PREFLIGHT_OK
POINTER-LINKED · STALE_RISK
UNGROUNDED · FALSE_PREMISE_SUSPECTED
UNGROUNDED · CONTRADICTORY_QUESTION
UNGROUNDED · OUT_OF_CORPUS

8. Hard rule (D1)

No LLM in preflight hard path. Hermes may propose SOFT_PREFLIGHT_HINT (sidecar) but cannot create PREFLIGHT_OK / PREFLIGHT_BLOCKED without deterministic support. Preserves seven-point program D1 (verifier never calls LLM).

9. Implementation order (per source doc §17)

1. metacognition.py with QuestionState.
2. Deterministic detectors:
   quantifier (REUSE), temporal, reference-frame (REUSE),
   contradiction, false-premise-lite, out-of-corpus.
3. Wire into query() / runner.ask().
4. Add preflight labels to renderer.
5. Add policy fields to governance_policy_hash.
6. Add bench fields.
7. Add tests.
8. Run-DAG node binding (defer to Phase 5 ticket #000009 work —
   same audit-replay gap; both nodes can land together).
9. Only later: model-assisted preflight as soft sidecar.

10. Test surface

Per source doc §14:

• False premise: When did Mr. Burns become Homer's biological father? → false_premise_suspected.
• Contradiction: Which unmarried spouse is Homer married to? → contradictory_question.
• Broad quantifier: Winners of all major sports? → broad_quantifier_unbounded.
• Reference frame: Has Oceania always been at war with East Asia? → reference frame includes orwell_1984.
• Time-sensitive: Who is the current CEO of OpenAI? → stale_risk unless current source retrieved.
• Out-of-corpus: What does my uploaded contract say about clause 9? → out_of_corpus.
• Model cutoff: model profile cutoff 2025-08, current date 2026-05-03, question latest winner of X → requires_current_source.

11. Status

Open · in progress 2026-05-03. Zero-shot implementation per fox directive. Updates land here as phases ship; blockers escalate to fox.

12. Implementation log

12.1 Phase 1 — module + detectors (LANDED 55efb04)

arborist/qa/metacognition.py shipped with:

• QuestionState dataclass (frozen, JSON-serializable via to_dict)
• preflight_question() pure function
• 4 new deterministic detectors (temporal, contradiction, false- premise-lite, out-of-corpus)
• Reuses arborist.qa.quantifier.classify_question_quantifier
• 8 LogicalStatus values + 3 PreflightResult values
• 6 per-detector enable switches in policy
• preflight_policy_hash for governance binding (Phase 3)
• PREFLIGHT_VERSION = "metacognition-v0.1"

42 new tests: per-detector unit tests + the seven test cases from source doc §14 + gating + determinism + serialization.

12.2 Phase 2 — wire into query() and runner.ask() (in same commit chain)

preflight_question() runs after policy resolution + quantifier classification, before retrieval. Surfaces question_state (from to_dict()) on:

• query() miss-path result dict
• query() cache-hit path (re-classifies cheaply since pure)
• query() reject-broad early-return path
• runner.ask() miss-path + cache-hit-path

Schema stays column-aligned across all four return shapes. First pass does NOT bind into run_dag_root — deferred to ticket #000009 (Phase 5) where the quantifier_preflight node lands too. Both nodes can land together to keep the run-DAG schema atomic.

12.3 Phase 3 — policy fields + governance hash + CLI

Six new policy fields (default-on except block_on_contradiction):

metacognition_enabled                    True
metacognition_temporal_check             True
metacognition_contradiction_check        True
metacognition_false_premise_check        True
metacognition_out_of_corpus_check        True
metacognition_block_on_contradiction     False

All folded into _VERIFIER_POLICY_FIELDS so flipping any of them invalidates prior cache records.

CLI flags on arborist query:

--no-preflight                disable the guard for one call
--block-on-contradiction      hard-block on lexical contradictions
                              (default: label-only)

12.4 Phase 4 — audit-line labels + bench fields + tests

_render_warrant_tail extended with five metacog tail tokens (when corresponding logical_status fires):

· false premise
· contradictory
· stale risk
· out of corpus
· frame ambiguous

Bench rows in bench/qa_sweep.py:_run_one gain seven new fields (bounded-size projection of QuestionState; full state stays on result dict for CLI render):

preflight_logical_statuses        list[str]
preflight_question_shape          str
preflight_result                  PREFLIGHT_OK / _PARTIAL / _BLOCKED
preflight_temporal_sensitivity    high / medium / low
preflight_has_false_premise       bool
preflight_has_contradiction       bool
preflight_corpus_requirement      str

Tests grew to 68 in the metacognition suite (42 Phase 1 + 16 new Phase 4: 6 governance + 6 audit-line tail + 4 default-policy pinning).

12.5 Live verification

make query-dry end-to-end:

$ make query-dry Q="Who is the current CEO of OpenAI?" BURN=1
  Who is the current CEO of OpenAI?
    UNGROUNDED · via claim_lattice · stale risk  0/0  73.4s  (fresh)

$ make query-dry Q="When did Mr. Burns become Homer's biological father?" BURN=1
  UNGROUNDED · via claim_lattice · false premise  0/0  46.4s  (fresh)

Both metacog tails land on the audit line; preflight runs pre- retrieval; QuestionState surfaces on the result dict.

12.6 What's NOT shipped in this ticket

• Run-DAG node binding for the metacognition_preflight stage. Hashing the QuestionState into run_dag_root so audit replay can distinguish guard-on vs guard-off, contradiction-detected vs not, etc. Same audit-replay gap as ticket #000009. Both Phase 5 work; will land together.
• Reference-frame plumbing. preflight_question() accepts reference_frames but the query() callsite passes empty — frame_detection runs after retrieval (it needs source titles) and preflight here is pre-retrieval. Wiring the post-retrieval frame info back into the QuestionState requires either a two- pass call or a post-classification update; deferred.
• SOFT_PREFLIGHT_HINT (model-assisted preflight). Source doc §18 reserves this label for future model-assisted preflight that runs as a sidecar. Not implemented; hard rule preserved (no LLM in the preflight hard path).
• Bench measurement of preflight effect. The mechanism is shipped; bench A/B (preflight on vs off) on the broad subset not yet run. Quick to do once stack settles.

13. Status — closed

Phases 1–4 landed 2026-05-03 in commit chain 55efb04 → (this commit). Mechanism complete; defaults preserve the dry-run discipline pattern from #000008.

Follow-ups split into separate tickets:

• Phase 5 (run-DAG node binding): joins ticket #000009's scope
• Cross-model bench: manual operator task

This ticket file remains in place as the design log per docs/TICKETS.md convention.

13.1 Default-flip validation bench (2026-05-03T23-06-21Z)

Post-landing validation: ran a preflight-OFF cell on the 9-question broad subset (metacognition_enabled=false, quantifier_reminder_enabled=false) to confirm the §12.10 default flip wasn't a regression. Compared against §12.6 reminder-only baseline.

Metric	OFF	ON	Δ
pointer mean ratio	0.483	0.643	+16pp
JSON mean ratio	0.570	0.735	+17pp
JSON UNGROUNDED rate	7/27 (26%)	1/27 (4%)	−22pp
pointer FORMAT_COLLAPSED	2/27	0/27	−100%
pointer NO_EVIDENCE_POINTER	8/27	6/27	−7pp
any-mode STRICT-rate	within ±11pp noise band on 27-sample cells

Verdict: defaults stay on. Mean-ratio + UNGROUNDED + FORMAT_ COLLAPSED metrics all clear the 5pp signal floor on lattice modes; STRICT-rate moves are within Hermes nondeterminism for n=3.

On this subset the metacognition detectors don't fire (no temporal / contradiction / false-premise / out-of-corpus shapes), so the delta isolates the reminder contribution. A future bench cycle on a metacog-trigger subset would isolate the metacog contribution; deferred.

Bench artifact: bench/qa_results/2026-05-03T23-06-21Z.{jsonl,md}. Documented in docs/qa-modes-bench.md Addendum 2.

13.2 Full-bench regression check (2026-05-03T23-30-12Z)

The §13.1 validation covered the 9-question broad subset where the preflight contract activates. To confirm the default flip doesn't regress narrow-question performance (89% of the bench), ran the full 75-question bench/qa_questions.txt × n=3 × 3 modes = 675 runs. Compared against the frozen 2026-05-02T15-07Z baseline (the authoritative pre-#000008/9/10 state, 213 runs per mode):

Mode	Pre-flip	Post-flip	Δ SR	Δ ratio
quote	116S/51H/46U · 0.54	116S/60H/49U · 0.52	−2pp	−1pp
claim_lattice_pointer	43S/146H/21U · 0.20	48S/145H/32U · 0.21	+1pp	+2pp
claim_lattice (JSON)	89S/82H/39U · 0.42	99S/80H/46U · 0.44	+2pp	+3pp

All deltas within the 5pp signal floor. Same question set ±2.

Substrate-level wins beyond headline numbers:

• pointer FORMAT_COLLAPSED: 0/225 across the full sweep (default-on reminder eliminates it globally, not just on broad)
• pointer NO_EVIDENCE_POINTER: 13% (vs 33% on broad-only when reminder was off) — citation discipline propagates to non-broad questions even though the reminder text only fires on broad
• JSON mean ratio +3pp consistent with broad-subset finding

Verdict: the default flip ships clean across the full corpus. Defaults stay on. The substrate is strictly more honest under preflight (FORMAT_COLLAPSED → 0, NO_EVIDENCE_POINTER ↓) without sacrificing throughput on non-broad questions.

Bench artifact: bench/qa_results/2026-05-03T23-30-12Z.{jsonl,md}. Documented in docs/qa-modes-bench.md Addendum 3.

13.3 Metacog-trigger detector validation (2026-05-04T02-18-42Z)

§13.1 + §13.2 measured preflight against the broad-quantifier subset & the full corpus — neither contains questions that fire the metacog detectors (temporal / contradiction / false-premise / out-of-corpus). This addendum closes the empirical gap.

Fixture bench/qa_questions_metacog_subset.txt (6 questions × n=3 × 3 modes = 54 rows). Each question targets one detector kind plus a control.

Headline: detector accuracy is 6/6 on the fixture set. All trigger questions fire the expected logical_statuses value; audit-line tails surface (· stale risk, · false premise, · contradictory, · out of corpus).

Concerning finding: 2 of 5 trigger questions return STRICT on a lattice mode despite the metacog warning:

• JSON STRICT on "when did George Washington stop being president of France?" — false premise; corpus accidentally grounds an invented answer.
• JSON STRICT on "what does my uploaded contract say?" — model fabricates a grounding for an out-of-corpus reference.

Quote mode is the most honest fallback (4/5 triggers land all- UNGROUNDED; the paraphrase verifier won't substring-match across the corpus when the premise has no anchor).

Implications:

• #000011 SOFT_PREFLIGHT_HINT design is validated. The deterministic detectors flag the shapes correctly; a model- assisted soft sidecar could add independent semantic skepticism on cases where lexical-pattern evidence accidentally matches.
• metacognition_block_on_contradiction=False default stays. Schrödinger's cat (alive+dead) JSON STRICT is defensible — the corpus contains legitimate quantum-mechanics articles. A hard- block would have refused a real-world answerable question. The label-only default is correct; operators opt in via --block-on-contradiction.

Bench artifact: bench/qa_results/2026-05-04T02-18-42Z.{jsonl,md}. Documented in docs/qa-modes-bench.md Addendum 4.