russell@unturf.com
621f0b2cda
Fan-out execution of the deferred-but-not-blocking pile from prior status reports. #000009 §7.2 — bench harness preflight_hash field: - aborist/qa/query.py surfaces `preflight_hash` on result dict (miss path, reject path, and cache-hit path via new helper `_extract_preflight_hash_from_blob` that pulls the stage hash out of persisted run_dag_blob). - bench/qa_sweep.py adds 12-char preflight_hash prefix to bench rows. Mirrors cache_key truncation pattern. Operators can grep / SQL-filter bench JSONL by preflight policy state. - 3 new tests in tests/test_dag.py for the extract helper. #000009 §7.2 — `aborist providence --show-preflight CACHE_KEY_PREFIX`: - New CLI flag pulls the preflight stage payload from a row's run_dag_blob. Match by 12-char prefix. Renders preflight stage hash + run-DAG stage list. Operator tool for inspecting which policy state governed a cached row. - Live verified on a real cache row (8a212fecb2a9 — current CEO of OpenAI question, 10-stage CTI shape with preflight at idx 1). - Legacy rows (predating #000009) report a clean fall-through message: "run_dag has no preflight stage (predates #000009)". #000010 §12.6 — reference-frame plumbing into QuestionState: - Pre-retrieval preflight runs with reference_frames=() (frame_detection needs source titles, not available yet). Post-retrieval, query.py re-runs preflight_question() with the detected frames so the result-dict + run-DAG QuestionState carry frame-aware logical_statuses (specifically `reference_frame_ambiguous` when 2+ frames match). - Live verified on Orwell-style question; logical_statuses now correctly includes `reference_frame_ambiguous` in the result. Metacog-trigger bench fixture (#000010 §13.3): - bench/qa_questions_metacog_subset.txt — 6 questions, one per detector kind plus a well-formed control. - Bench artifact 2026-05-04T02-18-42Z. Detector accuracy 6/6 on fixture; 2 of 5 trigger questions return STRICT on lattice mode despite metacog warning (JSON STRICT on George-Washington-stop-being-president-of-France false-premise + uploaded-contract out-of-corpus questions). Audit-line tails correctly surface the warnings. - qa-modes-bench.md Addendum 4 captures the per-question matrix + interpretation. #000010 §13.3 cross-references with bench artifact stamp. #000011 SOFT_PREFLIGHT_HINT design ticket opened: - docs/tickets/ticket-000011-soft-preflight-hint-sidecar.md captures the design proposal per #000010 §18 / source doc. Implementation deferred — design only. - Sidecar would add model-assisted preflight as a soft signal (`SOFT_FALSE_PREMISE_SUSPECTED` etc.) that NEVER enters the verifier hard path. Strict guardrail: cannot create PREFLIGHT_OK or PREFLIGHT_BLOCKED without deterministic support. - Validated by §13.3 finding: deterministic detectors flag correctly; corpus-accidental grounding produces 2/5 STRICT on trigger questions; soft sidecar would add independent semantic skepticism. - TICKETS.md index row added; Next ID bumped to 000012. 996 tests passing (3 new for the extract helper). Cross-doc consistency: - qa-modes-bench.md Addenda 1+2+3+4 chronological - #000010 §13.1 (broad subset) + §13.2 (full bench) + §13.3 (metacog trigger subset) - #000011 design captured but not implemented |
||
|---|---|---|
| .. | ||
| diagrams | ||
| tickets | ||
| bench-emergent-design.md | ||
| bench-maxing.md | ||
| concept-relations-design.md | ||
| cti-architecture.md | ||
| mesh-deploy.md | ||
| mesh.md | ||
| modules.md | ||
| qa-modes-bench-2026-04-30.md | ||
| qa-modes-bench.md | ||
| self-reference-design.md | ||
| seven-point-program.md | ||
| TICKETS.md | ||
| verifier-semantic-gap-design.md | ||