engineering/arborist
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
arborist/bench
History
russell@unturf.com 708aa450cb
fan-out: warrant ladder wiring · witness follow-ups · 5F Phase 1d 
Three small streams in one commit; each closes / expands a
recently-landed ticket without changing its hard contract.

#000026 Phase 3 wiring — authorship warrant ladder visible
============================================================

Phase 3 sidecar (arborist/qa/warrant_authorship.py landed in 60b5748)
exposed the classifier but didn't surface it. Two wirings:

- arborist/qa/inspect.py — diagnose_authorship_warrant runs against
  the cached row's question + answer + per-source raw chunks +
  URIs + titles; result lands as `authorship` field alongside the
  other sidecars.
- arborist/cli.py _render_warrant_tail — appends ` · warrant:
  <readable-tier>` when result['authorship'] is populated with a
  non-quiet tier. AUTHOR_COPYRIGHT_FOOTER → "copyright-footer", etc.
  NO_AUTHORSHIP_SIGNAL stays silent. Backward-compat: results
  without an `authorship` key render unchanged.

Tests: 3 inspect-path tests (no-signal, copyright-footer,
repository-owner) + 4 render-tail tests (presence, no-signal
silence, missing-key silence, all-six-tiers readable mapping).

#000028 follow-ups — capital ledger + sample-rate
==================================================

Two policy fields layered on top of canonical_witness_enabled:

- canonical_witness_sample_rate (0.0..1.0; default 1.0). Operators
  wanting passive calibration set 0.05 to fire witness on 5% of
  canonical questions while paying 5% of LLM cost. 0.0 effectively
  off; 1.0 = current always-on behavior. Gating uses random.random()
  so distribution is uniform; clamped to [0, 1].
- Capital ledger row written for each FIRED witness (not skipped
  ones). op_type='canonical_witness'; estimator inputs include
  prompt_chars + answer_chars + llm_seconds + agreement_label +
  pi_star_ref. Best-effort: ledger-write failure must never fail
  the query (sidecar discipline).

Tests: 4 new — sample_rate=0.0 skips (no LLM call, no ledger row);
sample_rate=1.0 always fires; capital_ledger row written under
op_type='canonical_witness' with full input blob; sampled-out
witness records zero ledger rows.

Both fields fold into governance_policy_hash naturally via the
existing policy-hash machinery — flipping witness mode invalidates
prior records as expected.

#000025 Phase 1d — 5F fixture catalog 30 → 50
==============================================

Both synthetic and live sides of all 5 sub-batteries expanded
30 → 50 (+200 fixtures total: 5 × 20 synthetic, 5 × 20 live).

  function       — claim_count cycles 2..7 across new fixtures
  falsification  — 10-violation palette across new ids
  feedback-loop  — fact-N learning chains
  finetuning     — capability transitions across canonical π*
                   (math/logic/algebra/calculus pool)
  formulate      — multi-pointer claim shapes

500/500 pass through respective runners. test_session_integration
total bumped 562 → 662. Pinned test_5f_*_runs counts updated 30 →
50 (synthetic main + embedded + live).

Tests
=====

Full suite: 1467 passed, 36 skipped (was 1388; +79 across warrant
render + witness sample/ledger + 5F implicit coverage).
2026-05-09 12:42:56 -04:00
..
batteries ticket #000030 Phases 1+2: algebra-symbolic@v1 + calculus-derivative@v1 2026-05-09 12:35:18 -04:00
fixtures fan-out: warrant ladder wiring · witness follow-ups · 5F Phase 1d 2026-05-09 12:42:56 -04:00
results modified: .gitignore 2026-05-08 16:38:09 -04:00
scripts #000026 Phase 2: real-shard baseline + lazy concept loading 2026-05-08 13:10:58 -04:00
emergent_log.jsonl ticket(#000006): 300-cycle update — zero false-positive STRICT post-hardening 2026-05-03 23:16:45 -04:00
qa_questions.txt aborist/arborist 2026-05-07 09:31:49 -04:00
qa_questions_metacog_subset.txt qa(#000011 + 4 more): SOFT_PREFLIGHT_HINT impl + 5-task fan-out 2026-05-03 23:00:56 -04:00
qa_questions_quantifier_baseline.txt bench(#000008): harness extension — FC rate, violation kinds, raw brackets 2026-05-02 18:35:08 -04:00
qa_questions_quantifier_subset.txt ticket(#000008): §12 dry-run bench findings + --policy harness flag 2026-05-03 08:39:20 -04:00
qa_questions_smoke.txt speed: pytest-xdist, bench smoke, concurrency default; UTF surrogate fix 2026-05-02 09:29:40 -04:00
qa_sweep.py aborist/arborist 2026-05-07 09:31:49 -04:00
run.sh aborist/arborist 2026-05-07 09:31:49 -04:00