fan-out: witness audit + 5F extractor + function-sampled demo + docs

Three small streams in one commit:

#000028 follow-up — witness divergence → 5F fixtures
=====================================================

Witness fan-out now writes a `providence_canonical_witness` audit
event when it fires (next to the capital-ledger record landed in
708aa45). Body carries pi_star_ref, question_text, agreement_label,
canonical_answer_text, llm_raw_text, llm_canonical_bytes,
cache_status. Best-effort write — chain failure never fails the
query.

New extractor `bench/scripts/witness_to_5f.py` reads those events
from a qa.db and writes them out as 5F-Falsification fixtures
matching the existing `falsification-live-v1` schema. Filtering
includes only divergence labels (LLM-DIVERGED / KERNEL-LLM-DIVERGED
/ CACHE-DRIFT); skips KERNEL-LLM-AGREE / STRICT-WITNESSED (no
calibration signal) and KERNEL-ONLY (LLM unparseable, not a
supervised-correction sample).

Idempotent: sorted by audit-event seq, so re-running against the
same qa.db produces byte-equal fixture files. The existing
fixture-digest discipline stays valid.

Makefile: `make bench-witness-divergence` (override default
qa.db / output path via WITNESS_QA_DB / WITNESS_OUT env-vars).

Closes the divergence → calibration data loop the witness ticket
imagined: every LLM hallucination on a canonical-shape question
becomes a supervised-correction fixture downstream prompt
improvements can grade against.

#000030 Phase 7 demo — function-sampled@v1 end-to-end
======================================================

`bench/scripts/demo_plot.py` — closes the loop on opencompletion's
activity24-math-plot.yaml. SymPy expression → quantized
integer-vector signature (canonical bytes) → optional matplotlib
PNG. Canonical bytes are the proof; PNG is just a downstream view
of the same evidence.

  $ make demo-plot Q='sin(x)' PNG=/tmp/sin.png

Output JSON contains canonical_bytes_sha256 + canonical_bytes_preview
+ canonical_bytes_total_chars + grid metadata + the optional png_path.
matplotlib is gated — when absent, --png prints a warning to stderr
and skips the render; the canonical bytes still print. Tests skip
the PNG-presence assertion via `pytest.importorskip("matplotlib")`.

Public docs polish (#7)
========================

- docs/_source/bench.rst: updated fixture-count narrative (~660 →
  662 default tasks + ~110 math π* fixtures); `make` quick-reference
  now lists all per-π* 5S targets (tabular, calculus-limit/series,
  linear-algebra, function-sampled) plus bench-real-shard,
  bench-fork-baseline/score, bench-witness-divergence.
- docs/_source/v8-fork-score.rst: CLI section gained --out flag
  documentation + a Make-harness sub-section covering
  bench-fork-baseline / bench-fork-score / FORK_PARENT/CHILD/REPORT
  env-vars.

Tests
=====

- tests/test_witness_to_5f.py — 8 new tests covering the audit-event
  write (3) + extractor logic (5).
- tests/test_demo_plot.py — 6 new tests covering canonical-bytes
  determinism + equivalence-class collapse + matplotlib gating.
Full suite: 1624 passed, 37 skipped (was 1568; +56).

bench/scripts/demo_plot.py

@@ -0,0 +1,160 @@
+"""End-to-end demo for ``function-sampled@v1`` (#000030 Phase 7).
+
+Closes the loop on opencompletion's ``activity24-math-plot.yaml``:
+SymPy expression → quantized integer-vector signature (canonical
+bytes) → optional matplotlib PNG (a downstream view of the
+canonical evidence).
+
+The canonical bytes are the proof: two function-sampled outputs
+that match byte-for-byte represent the same function on the same
+quantization grid. The PNG is just a human-readable rendering of
+those same bytes — different DPIs / colormaps / axis spans don't
+change identity.
+
+Usage::
+
+    python -m bench.scripts.demo_plot \\
+        --expr 'sin(x)' --x-min 0 --x-max 6.283 --n-samples 200 --dv 0.01 \\
+        --png /tmp/sin.png
+
+    # Or via Make (PNG optional):
+    make demo-plot Q='sin(x)'
+    make demo-plot Q='x**2 + 2*x + 1' PNG=/tmp/parabola.png
+
+The matplotlib import is gated — the canonical bytes always print
+to stdout; the PNG is written only when ``--png`` is provided AND
+matplotlib is importable.
+"""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import sys
+from pathlib import Path
+
+
+def _build_payload(
+    expr: str, x: str, x_min: float, x_max: float,
+    n_samples: int, dv: float,
+) -> bytes:
+    return json.dumps({
+        "f": expr, "x": x,
+        "x_min": x_min, "x_max": x_max,
+        "n_samples": n_samples, "dv": dv,
+    }).encode("utf-8")
+
+
+def _maybe_render_png(
+    *, expr: str, x_var: str, x_min: float, x_max: float,
+    n_samples: int, png_path: Path,
+) -> None:
+    """Render a PNG via matplotlib for human-readable inspection.
+    No-op if matplotlib isn't installed."""
+    try:
+        import matplotlib
+        matplotlib.use("Agg")
+        import matplotlib.pyplot as plt
+        import sympy as sp
+    except ImportError:
+        print(
+            "warning: matplotlib + sympy required for PNG render; "
+            "skipping --png",
+            file=sys.stderr,
+        )
+        return
+    x_sym = sp.Symbol(x_var)
+    f_expr = sp.sympify(expr, locals={x_var: x_sym})
+    f = sp.lambdify(x_sym, f_expr, "math")
+    step = (x_max - x_min) / (n_samples - 1)
+    xs = [x_min + i * step for i in range(n_samples)]
+    ys = []
+    for xi in xs:
+        try:
+            yi = f(xi)
+            if isinstance(yi, complex):
+                yi = yi.real
+            ys.append(float(yi))
+        except (ValueError, ZeroDivisionError, OverflowError):
+            ys.append(float("nan"))
+    plt.figure(figsize=(8, 5))
+    plt.plot(xs, ys, linewidth=2, color="#3366cc")
+    plt.title(f"y = {expr}")
+    plt.xlabel(x_var)
+    plt.ylabel("y")
+    plt.grid(True, alpha=0.3)
+    png_path.parent.mkdir(parents=True, exist_ok=True)
+    plt.tight_layout()
+    plt.savefig(png_path, dpi=120)
+    plt.close()
+
+
+def main(argv: list[str] | None = None) -> int:
+    p = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
+    p.add_argument("--expr", required=True, help="SymPy expression in the variable --x")
+    p.add_argument("--x", default="x", help="independent-variable name (default 'x')")
+    p.add_argument("--x-min", dest="x_min", type=float, default=-3.14159)
+    p.add_argument("--x-max", dest="x_max", type=float, default=3.14159)
+    p.add_argument("--n-samples", dest="n_samples", type=int, default=200)
+    p.add_argument("--dv", type=float, default=0.01,
+                   help="value-quantization step (smaller = tighter)")
+    p.add_argument("--png", type=Path, default=None,
+                   help="optional PNG output path (requires matplotlib)")
+    args = p.parse_args(argv)
+
+    from arborist.pi_star import PiStarError, get
+    try:
+        ps = get("function-sampled@v1")
+    except KeyError:
+        print(
+            "error: function-sampled@v1 not registered (sympy missing?); "
+            "install with `pip install arborist[math]`",
+            file=sys.stderr,
+        )
+        return 2
+
+    payload = _build_payload(
+        args.expr, args.x, args.x_min, args.x_max,
+        args.n_samples, args.dv,
+    )
+    try:
+        canonical_bytes = ps.canonicalize(payload)
+    except PiStarError as exc:
+        print(f"error: {exc}", file=sys.stderr)
+        return 1
+
+    digest = hashlib.sha256(canonical_bytes).hexdigest()
+
+    out = {
+        "expr": args.expr,
+        "x": args.x,
+        "grid": {
+            "x_min": args.x_min,
+            "x_max": args.x_max,
+            "n_samples": args.n_samples,
+            "dv": args.dv,
+        },
+        "canonical_bytes_sha256": digest,
+        "canonical_bytes_preview": canonical_bytes.decode("utf-8")[:200] + (
+            "…" if len(canonical_bytes) > 200 else ""
+        ),
+        "canonical_bytes_total_chars": len(canonical_bytes),
+        "pi_star_ref": "function-sampled@v1",
+    }
+
+    if args.png is not None:
+        _maybe_render_png(
+            expr=args.expr, x_var=args.x,
+            x_min=args.x_min, x_max=args.x_max,
+            n_samples=args.n_samples,
+            png_path=args.png,
+        )
+        out["png_path"] = str(args.png)
+
+    print(json.dumps(out, indent=2, ensure_ascii=False))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

bench/scripts/witness_to_5f.py

@@ -0,0 +1,192 @@
+"""Witness-divergence → 5F Falsification fixture extractor.
+
+Pulls ``providence_canonical_witness`` audit events from a qa.db
+and writes them out as 5F Falsification fixtures (matching the
+existing ``falsification-live-v1`` schema). Each diverging witness
+event becomes a supervised-correction sample the LLM can be
+calibrated against — the kernel's canonical bytes are ground
+truth, so an LLM raw answer that doesn't fold to those bytes is
+falsifiable evidence of an arithmetic / logic / symbolic
+hallucination.
+
+#000028 follow-up — closes the divergence → calibration data loop.
+The witness path landed in commit `656b573`; the audit-event write
+landed in this same commit. The extractor here is the missing
+piece that converts audit-chain entries into 5F-runner-compatible
+fixtures.
+
+Usage::
+
+    python -m bench.scripts.witness_to_5f \\
+        --qa-db ~/.arborist/shards/qa.db \\
+        --out bench/fixtures/5f/falsification-witness-v1.jsonl
+
+Or via Makefile::
+
+    make bench-witness-divergence
+
+Filtering rules
+---------------
+Includes events whose ``agreement_label`` indicates LLM divergence:
+
+- ``LLM-DIVERGED`` (kernel + cache agree; LLM differs)
+- ``KERNEL-LLM-DIVERGED`` (no cache; LLM differs from kernel)
+- ``CACHE-DRIFT`` (kernel + LLM agree; cache differs — flagged
+  as a calibration-data point too since the cache row is wrong)
+
+Skips:
+
+- ``STRICT-WITNESSED`` / ``KERNEL-LLM-AGREE`` /
+  ``KERNEL-CACHE-AGREE`` / ``KERNEL-ONLY`` — agreement, no signal.
+- Events without ``llm_raw_text`` — LLM was unparseable; not a
+  useful supervised-correction sample.
+
+Idempotency
+-----------
+Output is sorted by audit-event seq for deterministic byte-equality
+across runs against the same qa.db. Re-running with no new
+divergence events writes the same bytes. The fixture-digest
+discipline (#000021 §3) stays valid.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+from pathlib import Path
+
+from arborist.store import connect
+
+
+_DIVERGENCE_LABELS = {
+    "LLM-DIVERGED",
+    "KERNEL-LLM-DIVERGED",
+    "CACHE-DRIFT",
+}
+
+
+def extract_divergence_fixtures(
+    qa_db: Path,
+    *,
+    fixture_id_prefix: str = "5f-fal-witness",
+    verifier_method_root: str = "verify_quotes-v1",
+) -> list[dict]:
+    """Pull divergence events from `qa_db`'s audit chain. Returns a
+    list of 5F-Falsification fixture dicts (sorted by audit-event
+    seq for determinism)."""
+    conn = connect(qa_db)
+    try:
+        rows = conn.execute(
+            "SELECT seq, body FROM audit_events "
+            "WHERE event_type = 'providence_canonical_witness' "
+            "ORDER BY seq ASC"
+        ).fetchall()
+    finally:
+        conn.close()
+
+    fixtures: list[dict] = []
+    seen_ids: set[str] = set()
+    for r in rows:
+        try:
+            body = json.loads(r["body"])
+        except (json.JSONDecodeError, TypeError):
+            continue
+        agreement = body.get("agreement_label")
+        if agreement not in _DIVERGENCE_LABELS:
+            continue
+        llm_raw = body.get("llm_raw_text")
+        if not llm_raw:
+            # Unparseable LLM output isn't a calibration sample.
+            continue
+        canonical_text = body.get("canonical_answer_text") or ""
+        question_text = body.get("question_text") or ""
+        pi_star_ref = body.get("pi_star_ref") or "arithmetic@v1"
+
+        # Stable id from seq — re-running against the same qa.db
+        # produces the same fixture bytes (idempotency contract).
+        fixture_id = f"{fixture_id_prefix}-{int(r['seq']):06d}"
+        if fixture_id in seen_ids:
+            continue
+        seen_ids.add(fixture_id)
+        fixtures.append({
+            "id": fixture_id,
+            "battery": "5f",
+            "sub_battery": "falsification",
+            "version": "v1",
+            "carrier": "providence_record",
+            "domain": "claim_lattice",
+            "pi_star_ref": pi_star_ref,
+            "answer_text": llm_raw,
+            "context": (
+                f"canonical_kernel_answer={canonical_text}\n"
+                f"question={question_text}"
+            ),
+            "expected_reason": "UNGROUNDED",
+            "verifier_method_root": verifier_method_root,
+            "expected": "pass",
+            "_witness_meta": {
+                "agreement_label": agreement,
+                "audit_seq": int(r["seq"]),
+                "pi_star_ref": pi_star_ref,
+            },
+        })
+    return fixtures
+
+
+def write_fixtures(out_path: Path, fixtures: list[dict]) -> None:
+    out_path.parent.mkdir(parents=True, exist_ok=True)
+    meta = {
+        "_meta": {
+            "battery": "5f",
+            "sub_battery": "falsification",
+            "version": "v1",
+            "task_count": len(fixtures),
+            "notes": (
+                "Witness-divergence-extracted falsification fixtures. "
+                "Auto-generated by `make bench-witness-divergence` "
+                "from providence_canonical_witness audit events. "
+                "Each row pairs an LLM raw answer with the kernel's "
+                "canonical answer; verify_quotes is expected to "
+                "return UNGROUNDED because the LLM prose doesn't "
+                "substring-match the kernel's terse canonical form. "
+                "_witness_meta carries the original agreement_label "
+                "+ audit_seq for traceability."
+            ),
+        },
+    }
+    with out_path.open("w", encoding="utf-8") as fh:
+        fh.write(json.dumps(meta, ensure_ascii=False) + "\n")
+        for fx in fixtures:
+            fh.write(json.dumps(fx, ensure_ascii=False) + "\n")
+
+
+def main(argv: list[str] | None = None) -> int:
+    p = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
+    p.add_argument(
+        "--qa-db", type=Path, required=True,
+        help="path to qa.db (the providence_cache + audit_events store)",
+    )
+    p.add_argument(
+        "--out", type=Path,
+        default=Path("bench/fixtures/5f/falsification-witness-v1.jsonl"),
+        help="output JSONL path",
+    )
+    args = p.parse_args(argv)
+
+    if not args.qa_db.is_file():
+        print(f"error: qa-db not found: {args.qa_db}", file=sys.stderr)
+        return 2
+
+    fixtures = extract_divergence_fixtures(args.qa_db)
+    write_fixtures(args.out, fixtures)
+    print(
+        f"wrote {len(fixtures)} divergence fixtures to {args.out}",
+        file=sys.stderr,
+    )
+    print(json.dumps({"divergence_count": len(fixtures)}))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())