engineering/arborist
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
arborist/bench
History
russell@unturf.com 3bff55234f
5f: efficiency metrics with explicit zero-cost guards 
Per fox's 2026-05-08 review of fbd99a8: implement adaptation_efficiency
and feedback_efficiency in run_finetuning + run_feedback_loop with
explicit sentinels — not Python floating-point accidents.

_efficiency(gain, cost) helper:

- cost > 0:       standard ratio
- cost == 0, gain > 0:  EFFICIENCY_INFINITE  (free improvement)
- cost == 0, gain == 0: EFFICIENCY_UNDEFINED (= 0.0; no signal)
- cost == 0, gain < 0:  -EFFICIENCY_INFINITE (free regression)

Battery-level metrics report mean_finite (computed over finite
values only) + infinite_count + neg_infinite_count so the mean
stays dimensionally truthful and consumers can pivot on the special
buckets separately.

Phase 1a cost proxies:
- run_finetuning: _capital_cost_delta sums resource_budget
  (max_compute_ms_delta * 1e-3 + max_storage_delta_bytes / 1e6).
  Phase 1b.2 will replace with real capital_ledger reads.
- run_feedback_loop: chain length = cost. Phase 1b.2 capital_ledger
  integration replaces it.

Tests added (7):
- _efficiency over four boundary cases
- 5F finetuning + feedback_loop emit the new metrics keys
- Synthetic zero-cost finetuning fixture verifies +inf path

Full suite: 1110 passed, 36 skipped.

Closes the one actionable from the fbd99a8 review.
2026-05-07 20:57:34 -04:00
..
batteries 5f: efficiency metrics with explicit zero-cost guards 2026-05-07 20:57:34 -04:00
fixtures bench: land #000023 + #000024 + #000025 (Phase 1a/1b — Dav1DPrometheus suite) 2026-05-07 20:14:44 -04:00
emergent_log.jsonl ticket(#000006): 300-cycle update — zero false-positive STRICT post-hardening 2026-05-03 23:16:45 -04:00
qa_questions.txt aborist/arborist 2026-05-07 09:31:49 -04:00
qa_questions_metacog_subset.txt qa(#000011 + 4 more): SOFT_PREFLIGHT_HINT impl + 5-task fan-out 2026-05-03 23:00:56 -04:00
qa_questions_quantifier_baseline.txt bench(#000008): harness extension — FC rate, violation kinds, raw brackets 2026-05-02 18:35:08 -04:00
qa_questions_quantifier_subset.txt ticket(#000008): §12 dry-run bench findings + --policy harness flag 2026-05-03 08:39:20 -04:00
qa_questions_smoke.txt speed: pytest-xdist, bench smoke, concurrency default; UTF surrogate fix 2026-05-02 09:29:40 -04:00
qa_sweep.py aborist/arborist 2026-05-07 09:31:49 -04:00
run.sh aborist/arborist 2026-05-07 09:31:49 -04:00