engineering/arborist
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(#000057): bench/watt_calibrate.py — separate prefill vs decode energy

Browse source 
fox 2026-05-21: account wattage for input and output separately. Prefill
(process all prompt tokens, parallel/compute-bound) and decode (generate
output, autoregressive/bandwidth-bound) are different GPU ops with
different J/token — a single per-token number can't represent both.

Slope calibration (no sub-request power alignment): sweep prompt length
at tiny max_tokens -> prefill J/input-tok (fixed overhead cancels in the
slope); fix a tiny prompt and sweep forced output length (ignore_eos) ->
decode J/output-tok. Prefill kept COLD (unique filler so cached_tokens=0).
Reuses watt_bench probes. Bad points (context overflow) skip, not abort.

Measured qwen-nothink/4090 @$0.33/kWh: prefill 0.175 J/tok
($0.016/M-input-tok), decode 6.16 J/tok ($0.564/M-output-tok) — decode
35x dearer per token. Predicts measured substrate J/q within ~5%. 14
tests (+ slope). Validated live.
This commit is contained in:
russell@unturf.com 2026-05-21 11:48:36 -04:00
parent 459060c774
commit 892d9ed037
No known key found for this signature in database
2 changed files with 230 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

12
tests/test_watt_cogs.py
View file

@ -8,9 +8,21 @@ the serving-floor marginal, and the data-derived band split.
See docs/calculator-test-patterns.md.
"""
from bench.watt_bench import energy_cogs
from bench.watt_calibrate import _slope
from bench.watt_probe import classify_power_bands
def test_slope_recovers_known_rate():
# y = 0.175*x + 60 (decode-like: per-token slope + fixed overhead).
xs = [32.0, 128.0, 512.0]
ys = [0.175 * x + 60 for x in xs]
assert round(_slope(xs, ys), 4) == 0.175 # slope ignores intercept
def test_slope_two_points():
assert _slope([100.0, 200.0], [10.0, 30.0]) == 0.2
# ---- energy_cogs -------------------------------------------------------
def test_gross_cogs_one_kwh():