engineering/arborist
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(#000057): control_ab header prints actual model/answer_mode/judge

Browse source 
Was a stale hardcoded 'same Hermes; judge=Opus hermetic' label that
misreported any run with --model/--judge overrides (e.g. qwen + code
judge). Now reflects the real config — honest header for the artifact.
This commit is contained in:
russell@unturf.com 2026-05-21 12:37:24 -04:00
parent 26e4db67d1
commit 2fd3523777
No known key found for this signature in database
1 changed files with 3 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

5
bench/control_ab.py
View file

@ -219,8 +219,9 @@ def main() -> int:
def rate(arm, *labels): def rate(arm, *labels):
return sum(tally[arm][x] for x in labels) return sum(tally[arm][x] for x in labels)
print(f"\n=== #000057 v1 control A/B (n={n}, single-shot, same " print(f"\n=== #000057 v1 control A/B (n={n}, single-shot, "
f"Hermes; judge=Opus hermetic) ===") f"model={a.model}; answer_mode={a.answer_mode}; "
f"judge={a.judge}) ===")
print(f"fixture={a.fixture} log={outp}") print(f"fixture={a.fixture} log={outp}")
for arm in ("solo", "arborist"): for arm in ("solo", "arborist"):
c = tally[arm] c = tally[arm]