docs(CLAUDE.md): budget discipline — Hermes/Qwen first, Opus deferred
2026-05-19: huge-N #000057 control-arm sweep (f63b00d→9dc02e4) burned our Opus quota. Pin the policy into repo CLAUDE.md so future agents don't reach for Opus on autopilot: - Sweep + bench-qa default endpoints: Hermes-3-8B + Qwen via uncloseai - Opus drops out of control-arm grids until fox explicitly re-adds it - Existing bench/results/ data stays valid; just no new Opus expansion - Translation still uses local pinned opus-mt (sandwich MT edges), not Hermes/Qwen 8B - Grok rejoin path noted as an HTML comment (paid spend, fox-gated) Adds two subsections under '## Live endpoints': - 'Budget discipline — Hermes / Qwen first, Opus deferred' - 'Per-call model selection (general agent work, not sweep arms)'
This commit is contained in:
parent
981ba84f4a
commit
2365bd1da4
1 changed files with 27 additions and 0 deletions
27
CLAUDE.md
27
CLAUDE.md
|
|
@ -347,6 +347,33 @@ revert without reading why. When in doubt, walk the
|
|||
- Wikipedia dumps: `https://dumps.wikimedia.org/archive/2003/2003-05-16/en/`.
|
||||
`robots.txt` returned 404 → no rules.
|
||||
|
||||
### Budget discipline — Hermes / Qwen first, Opus deferred
|
||||
|
||||
2026-05-19: the huge-N `#000057` control-arm sweep
|
||||
(`f63b00d` → `9dc02e4`) burned our Opus quota. Until fox explicitly
|
||||
re-adds it, sweep + `bench-qa` LLM calls route to **Hermes-3-8B**
|
||||
(`hermes.ai.unturf.com/v1`) and **Qwen via uncloseai.com**; Opus drops
|
||||
out of control-arm grids. This is a budget rule, not a quality claim —
|
||||
bench results already on disk under `bench/results/` stay valid; just
|
||||
don't expand them with new Opus tokens without a go. Translation at
|
||||
scale is still NOT a Hermes/Qwen 8B job (sandwich MT uses local pinned
|
||||
`opus-mt`, see crosslang rule). Grok candidacy noted from ajax synthetic-
|
||||
data-distillation benchmarks.
|
||||
|
||||
<!-- Grok API rejoins the grid once fox approves credit-card spend.
|
||||
When unflagged, add a `grok-*` arm to `bench/control_sweep.py`
|
||||
and document model id + endpoint here. -->
|
||||
|
||||
### Per-call model selection (general agent work, not sweep arms)
|
||||
|
||||
Within a 4-hour Max-plan window, haiku/sonnet/opus draw equally from
|
||||
quota — prefer the largest model that fits (fewer retries = better
|
||||
quota efficiency). During **paid overflow** (beyond Max), pay-per-token
|
||||
applies: start haiku, escalate sonnet then opus only on failure.
|
||||
Hermes/Qwen via `uncloseai.com` serve classification & code-adjacent
|
||||
work, **never human-language translation** (8B quality too low; that's
|
||||
the sandwich-MT `opus-mt` edge).
|
||||
|
||||
## Retrieval pipeline (`arborist/qa/query.py`)
|
||||
|
||||
Multi-stage. Each stage exists because something earlier wasn't
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue