bend: 3090-only production default; 4090 reserved for qwen

Fox decision: don't routine-fan-out to ai.foxhop.net (4090) when
qwen LLM holds GPU residency. The radix-sort 4090 OOM caveat
surfaced today demonstrated the cost of casual co-residency —
secp daemon parked 24 GiB up front leaving 47 MiB free.

bend.lsp's *bend-workers* default already empty (single-host
fallback). Updated the docstring example to drop the ai.foxhop.net
entry; multi-host fan-out is OPT-IN per call via bend-set-workers!
or BEND_WORKERS env.

bend.html fleet section now reflects the policy:
  - 3090-ai.foxhop.net:9091  active production worker
  - ai.foxhop.net:9092       reserved for qwen; bend per workload

ai.foxhop.net worker process killed; 4090 VRAM returned to qwen
(1.6 GiB free post-kill vs 47 MiB while bend was running).

When we have a long-running parallel sweep that justifies the
4090's marginal throughput, the caller opts in explicitly. Don't
auto-route.
This commit is contained in:
russell@unturf.com 2026-06-05 21:55:41 -04:00
parent d74953723f
commit 9e4e9b48f9
No known key found for this signature in database
2 changed files with 12 additions and 7 deletions

View file

@ -87,9 +87,14 @@
;;; falls back to a single *bend-worker-host* / *bend-worker-port* pair ;;; falls back to a single *bend-worker-host* / *bend-worker-port* pair
;;; (full back-compat with single-host callers). ;;; (full back-compat with single-host callers).
;;; ;;;
;;; Set via (bend-set-workers! '(("3090-ai.foxhop.net" . 9091) ;;; Set via (bend-set-workers! '(("3090-ai.foxhop.net" . 9091)))
;;; ("ai.foxhop.net" . 9092)))
;;; or environment variable BEND_WORKERS="host:port,host:port". ;;; or environment variable BEND_WORKERS="host:port,host:port".
;;;
;;; Production default: 3090-ai only. ai.foxhop.net (4090) is
;;; reserved for qwen LLM; we do NOT add it to the round-robin by
;;; default. Multi-host fan-out gets enabled per long-running
;;; parallel workload — caller opts in explicitly via
;;; bend-set-workers! or BEND_WORKERS env.
(define *bend-workers* '()) (define *bend-workers* '())
(define *bend-rr-idx* 0) (define *bend-rr-idx* 0)

View file

@ -70,15 +70,15 @@ make gpu-worker LUMBDA=asm # smallest footprint</code></pre>
<section id="fleet"> <section id="fleet">
<h2>Fleet</h2> <h2>Fleet</h2>
<p>bend runs on a 2-host LAN cluster; round-robin selection lives in <code>bend.lsp</code> via <code>*bend-workers*</code> + <code>BEND_WORKERS</code> env.</p> <p>Production default: single host. Multi-host fan-out is built (round-robin in <code>bend.lsp</code> via <code>*bend-workers*</code> + <code>BEND_WORKERS</code> env), available on demand for long-running parallel workloads &mdash; not used routinely.</p>
<table> <table>
<thead><tr><th>host</th><th>GPU</th><th>arch</th><th>port</th><th>coexists with</th></tr></thead> <thead><tr><th>host</th><th>GPU</th><th>arch</th><th>port</th><th>status</th></tr></thead>
<tbody> <tbody>
<tr><td><code>3090-ai.foxhop.net</code></td><td>RTX 3090 (24 GB)</td><td>sm_86</td><td>9091</td><td>idle</td></tr> <tr><td><code>3090-ai.foxhop.net</code></td><td>RTX 3090 (24 GB)</td><td>sm_86</td><td>9091</td><td><strong>active</strong> &mdash; production worker</td></tr>
<tr><td><code>ai.foxhop.net</code></td><td>RTX 4090 (24 GB)</td><td>sm_89</td><td>9092</td><td>qwen LLM (llama.cpp) on GPU</td></tr> <tr><td><code>ai.foxhop.net</code></td><td>RTX 4090 (24 GB)</td><td>sm_89</td><td>9092</td><td>reserved for qwen LLM (llama.cpp); bend worker enabled per workload</td></tr>
</tbody> </tbody>
</table> </table>
<p>The 4090 box shares its GPU with a qwen LLM &mdash; bend kernels burn SMs for milliseconds then release; qwen keeps its weights resident; 24 GB VRAM holds both. Cluster aggregate (sequential round-robin dispatcher): 1.6&times; at 200 × <code>n=1k</code> mod-mul, 1.2&times; at 6 × <code>n=100k</code>; an async dispatcher unlocks the remaining 2&times; headroom.</p> <p>Multi-host fan-out was validated at 1.6&times; aggregate throughput on small workloads, but routine round-robin against the 4090 would steal VRAM from qwen. Caller opts in explicitly when a workload justifies fan-out: <code>(bend-set-workers! '((&quot;3090-ai.foxhop.net&quot; . 9091) (&quot;ai.foxhop.net&quot; . 9092)))</code> brings the 4090 online for that call.</p>
</section> </section>
<section id="catalog"> <section id="catalog">