engineering/arborist
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
arborist/docs
History
russell@unturf.com fa8d93c8ef
ticket(#000008,#000006,index): §12.6 reminder A/B verdict + cross-refs 
§12.6 in #000008 captures the 2026-05-03T12-38-53Z reminder-only
A/B (apply_caps=False, reminder=True) on the same 9-question
broad subset:

  Mode      | Strict-rate    | Mean ratio        | UNGROUNDED
  ----------+----------------+-------------------+-----------
  quote     | 0.56 → 0.52    | 0.900 → 0.845     |  0 → 0
  pointer   | 0.00 → 0.00    | 0.473 → 0.643     |  9 → 6
  JSON      | 0.19 → 0.22    | 0.524 → 0.735     |  7 → 1

Pointer-mode violation deltas:
  FORMAT_COLLAPSED      2 →  0  (-100%)
  NO_EVIDENCE_POINTER   9 →  6  ( -33%)
  TITLE_MISMATCH       10 → 15  ( +50%)  ← side effect
  TOO_MANY_CLAIMS       7 →  8  ( +14%)

§10.8 gate verdict: MET. Both FORMAT_COLLAPSED and NO_EVIDENCE_
POINTER cleared the 5pp floor (−7pp absolute / −11pp absolute
respectively). Strongest signals are mean-ratio improvements
(+17pp pointer, +21pp JSON) — grounded rows ground BETTER under
reminder. JSON-mode UNGROUNDED dropped 7 → 1, a 22pp redistribution
from "didn't ground" to "partially grounded".

Caveat: TITLE_MISMATCH increased (+50%). Reminder may pressure
Hermes to cite *something* rather than say "no evidence", picking
up wrong-source citations as a side effect.

Recommendation: §10.8 gate met but hold default flip until §12.7
(cap-only) and §12.8 (cap+reminder) cells run, per §10.8 "if A+B
together outperform either alone by ≥5pp: land Option D".

Cross-references:

- TICKETS.md index: #000008 status flipped to "open · phases 0–4
  landed; bench A/B in progress".
- #000006 rolling log: cross-reference to #000008's bench cycles
  + the d24291b classifier-defect fix surfaced from the
  distribution scan.
2026-05-03 08:48:19 -04:00
..
diagrams docs: README label refresh + per-module reference + Graphviz diagrams 2026-05-01 23:19:01 -04:00
tickets ticket(#000008,#000006,index): §12.6 reminder A/B verdict + cross-refs 2026-05-03 08:48:19 -04:00
bench-emergent-design.md bench: emergent stress-test — 3-word triangulation (blue-moon cadence) 2026-05-02 12:42:56 -04:00
bench-maxing.md docs(bench): rename qa-modes-bench-2026-05-02.md → qa-modes-bench.md 2026-05-02 12:27:38 -04:00
concept-relations-design.md make: speed up tests + automate concept backfill (test 38s→11s) 2026-05-02 10:11:16 -04:00
cti-architecture.md docs: tickets/ subdir + delete naming-deferral, ref-frame, coverage-audit 2026-05-02 12:51:09 -04:00
mesh-deploy.md docs: mesh-deploy runbook for two-host gossip setup 2026-04-28 17:24:37 -04:00
mesh.md docs: mesh multiplayer protocol with dot diagrams 2026-04-28 12:54:36 -04:00
modules.md docs: consolidate docs/modules/* → docs/modules.md (single file) 2026-05-02 13:23:43 -04:00
qa-modes-bench-2026-04-30.md docs(qa): final stop-sequence bench row, day-journey net summary 2026-04-30 16:32:41 -04:00
qa-modes-bench.md docs(bench): rename qa-modes-bench-2026-05-02.md → qa-modes-bench.md 2026-05-02 12:27:38 -04:00
self-reference-design.md docs: consolidate self-reference design — flat MVP + fact-Core distillation 2026-05-02 12:39:42 -04:00
seven-point-program.md qa(frame): #000002 land — reference-frame polarity contract (D3 → ✓) 2026-05-01 20:18:13 -04:00
TICKETS.md ticket(#000008,#000006,index): §12.6 reminder A/B verdict + cross-refs 2026-05-03 08:48:19 -04:00
verifier-semantic-gap-design.md docs(qa): design proposal for verifier semantic-gap NLI sidecar 2026-04-30 19:50:05 -04:00