5f: Phase 1b.2 — Formulate runner wires to arborist.qa.parse_claims (live)

First sub-battery to bridge from synthetic gold output to actual
organism behavior. run_formulate now supports two fixture modes
selected per-task:

- Embedded (Phase 1a): produced_lattice in the fixture. 10 seed
  fixtures continue to pass via this path.
- Live (Phase 1b.2): only input_text in the fixture; runner calls
  arborist.qa.parse_claims.parse_pointer_claims(input_text) and
  matches the live output against expected_lattice.

Embedded takes precedence if both fields are present. Per-task
detail.source ("embedded" | "live") surfaces in bench output so
synthetic vs live signal is distinguishable.

Surface:

- bench/batteries/b_5f.py — _live_produced_lattice helper +
  two-mode dispatch in run_formulate
- bench/fixtures/5f/formulate-live-v1.jsonl — 15 live-mode fixtures
  with input_text + expected_lattice (no produced_lattice)
- Makefile: bench-5f-formulate-live target
- Tests: 4 new in tests/test_bench_batteries.py
  - live path routes through real parser, all 15 pass
  - embedded path still works (10 Phase-1a fixtures)
  - _live_produced_lattice helper directly verifies parse output
  - fixture missing both fields fails cleanly with explanatory reason

Phase 1a fixture digests unchanged. _DEFAULT_FIXTURES still points
at formulate-v1.jsonl so `runner --all` behavior is identical;
live-mode fixtures invoked via explicit --fixtures path.

Full suite: 1196 passed, 36 skipped.

Pattern set. Function/Finetuning/Falsification/Feedback Loop
follow in subsequent commits.
This commit is contained in:
russell@unturf.com 2026-05-08 08:20:51 -04:00
parent 6e20c792c4
commit ba653755e4
No known key found for this signature in database
5 changed files with 141 additions and 5 deletions

View file

@ -517,6 +517,26 @@ test_5f_feedback_loop_observation_affects_downstream_snapshot
14. Threshold calibration for v8 selection acceptance handed off
to ticket #000012.
### Phase 1b.2 — first live wire-up landed 2026-05-08
`run_formulate` now supports two fixture modes:
- **Embedded** (Phase 1a): fixture provides `produced_lattice`
directly. The 10 seed fixtures continue to pass.
- **Live**: fixture provides `input_text` only; runner calls
`arborist.qa.parse_claims.parse_pointer_claims` on it and
matches the live output against `expected_lattice`.
`bench/fixtures/5f/formulate-live-v1.jsonl` ships 15 live-mode
fixtures. `make bench-5f-formulate-live` runs them. Per-task
`detail.source` reports `"embedded"` or `"live"` so bench output
distinguishes synthetic from live signal.
This is the **first sub-battery to bridge from synthetic gold to
actual organism behavior** — proves the Phase 1b.2 pattern. The
remaining sub-batteries (Function, Finetuning, Falsification,
Feedback Loop) follow the same pattern in subsequent commits.
---
## 11. Status