arborist/docs
russell@unturf.com cd0e8ef64f
bench(#000010): preflight on-vs-off validation — defaults stay on
Post-landing validation cell. Same 9-question broad subset as
§12.6 reminder-only baseline; this run flips both
metacognition_enabled and quantifier_reminder_enabled to False so
we can isolate the preflight contribution.

  Metric                    | OFF       | ON       | Δ
  --------------------------+-----------+----------+----------
  pointer mean ratio        | 0.483     | 0.643    | +16pp
  JSON mean ratio           | 0.570     | 0.735    | +17pp
  JSON UNGROUNDED rate      | 7/27      | 1/27     | -22pp
  pointer FORMAT_COLLAPSED  | 2/27      | 0/27     | -100%
  any-mode STRICT-rate      | within ±11pp noise (27-sample n=3 floor)

Mean-ratio + UNGROUNDED + FORMAT_COLLAPSED metrics all clear the
5pp signal floor on lattice modes. STRICT-rate moves are within
Hermes nondeterminism. The #000010 default flip is doing what
was claimed.

On this subset none of the metacognition detectors fire (no
temporal / contradiction / false-premise / out-of-corpus shapes),
so the delta effectively isolates the reminder contribution from
#000008. A metacog-trigger subset bench is deferred.

Documentation:
  - docs/qa-modes-bench.md Addendum 2 captures the comparison
    table + interpretation + verdict.
  - docs/tickets/ticket-000010-... §13.1 cross-references with
    bench artifact stamp.

Bench artifact: bench/qa_results/2026-05-03T23-06-21Z.{jsonl,md}.
2026-05-03 19:26:33 -04:00
..
diagrams docs: distill #000008+#000009+#000010 into core docs + diagrams 2026-05-03 19:18:22 -04:00
tickets bench(#000010): preflight on-vs-off validation — defaults stay on 2026-05-03 19:26:33 -04:00
bench-emergent-design.md bench: emergent stress-test — 3-word triangulation (blue-moon cadence) 2026-05-02 12:42:56 -04:00
bench-maxing.md docs(bench): rename qa-modes-bench-2026-05-02.md → qa-modes-bench.md 2026-05-02 12:27:38 -04:00
concept-relations-design.md make: speed up tests + automate concept backfill (test 38s→11s) 2026-05-02 10:11:16 -04:00
cti-architecture.md docs: distill #000008+#000009+#000010 into core docs + diagrams 2026-05-03 19:18:22 -04:00
mesh-deploy.md docs: mesh-deploy runbook for two-host gossip setup 2026-04-28 17:24:37 -04:00
mesh.md docs: mesh multiplayer protocol with dot diagrams 2026-04-28 12:54:36 -04:00
modules.md docs: distill #000008+#000009+#000010 into core docs + diagrams 2026-05-03 19:18:22 -04:00
qa-modes-bench-2026-04-30.md docs(qa): final stop-sequence bench row, day-journey net summary 2026-04-30 16:32:41 -04:00
qa-modes-bench.md bench(#000010): preflight on-vs-off validation — defaults stay on 2026-05-03 19:26:33 -04:00
self-reference-design.md docs: consolidate self-reference design — flat MVP + fact-Core distillation 2026-05-02 12:39:42 -04:00
seven-point-program.md docs: distill #000008+#000009+#000010 into core docs + diagrams 2026-05-03 19:18:22 -04:00
TICKETS.md qa(#000009): §8 corrections — reject-path DAG + nested CTI clauses 2026-05-03 18:49:56 -04:00
verifier-semantic-gap-design.md docs(qa): design proposal for verifier semantic-gap NLI sidecar 2026-04-30 19:50:05 -04:00