bend: 3090-only production default; 4090 reserved for qwen
Fox decision: don't routine-fan-out to ai.foxhop.net (4090) when qwen LLM holds GPU residency. The radix-sort 4090 OOM caveat surfaced today demonstrated the cost of casual co-residency — secp daemon parked 24 GiB up front leaving 47 MiB free. bend.lsp's *bend-workers* default already empty (single-host fallback). Updated the docstring example to drop the ai.foxhop.net entry; multi-host fan-out is OPT-IN per call via bend-set-workers! or BEND_WORKERS env. bend.html fleet section now reflects the policy: - 3090-ai.foxhop.net:9091 active production worker - ai.foxhop.net:9092 reserved for qwen; bend per workload ai.foxhop.net worker process killed; 4090 VRAM returned to qwen (1.6 GiB free post-kill vs 47 MiB while bend was running). When we have a long-running parallel sweep that justifies the 4090's marginal throughput, the caller opts in explicitly. Don't auto-route.
This commit is contained in:
parent
d74953723f
commit
9e4e9b48f9
2 changed files with 12 additions and 7 deletions
|
|
@ -87,9 +87,14 @@
|
||||||
;;; falls back to a single *bend-worker-host* / *bend-worker-port* pair
|
;;; falls back to a single *bend-worker-host* / *bend-worker-port* pair
|
||||||
;;; (full back-compat with single-host callers).
|
;;; (full back-compat with single-host callers).
|
||||||
;;;
|
;;;
|
||||||
;;; Set via (bend-set-workers! '(("3090-ai.foxhop.net" . 9091)
|
;;; Set via (bend-set-workers! '(("3090-ai.foxhop.net" . 9091)))
|
||||||
;;; ("ai.foxhop.net" . 9092)))
|
|
||||||
;;; or environment variable BEND_WORKERS="host:port,host:port".
|
;;; or environment variable BEND_WORKERS="host:port,host:port".
|
||||||
|
;;;
|
||||||
|
;;; Production default: 3090-ai only. ai.foxhop.net (4090) is
|
||||||
|
;;; reserved for qwen LLM; we do NOT add it to the round-robin by
|
||||||
|
;;; default. Multi-host fan-out gets enabled per long-running
|
||||||
|
;;; parallel workload — caller opts in explicitly via
|
||||||
|
;;; bend-set-workers! or BEND_WORKERS env.
|
||||||
|
|
||||||
(define *bend-workers* '())
|
(define *bend-workers* '())
|
||||||
(define *bend-rr-idx* 0)
|
(define *bend-rr-idx* 0)
|
||||||
|
|
|
||||||
|
|
@ -70,15 +70,15 @@ make gpu-worker LUMBDA=asm # smallest footprint</code></pre>
|
||||||
|
|
||||||
<section id="fleet">
|
<section id="fleet">
|
||||||
<h2>Fleet</h2>
|
<h2>Fleet</h2>
|
||||||
<p>bend runs on a 2-host LAN cluster; round-robin selection lives in <code>bend.lsp</code> via <code>*bend-workers*</code> + <code>BEND_WORKERS</code> env.</p>
|
<p>Production default: single host. Multi-host fan-out is built (round-robin in <code>bend.lsp</code> via <code>*bend-workers*</code> + <code>BEND_WORKERS</code> env), available on demand for long-running parallel workloads — not used routinely.</p>
|
||||||
<table>
|
<table>
|
||||||
<thead><tr><th>host</th><th>GPU</th><th>arch</th><th>port</th><th>coexists with</th></tr></thead>
|
<thead><tr><th>host</th><th>GPU</th><th>arch</th><th>port</th><th>status</th></tr></thead>
|
||||||
<tbody>
|
<tbody>
|
||||||
<tr><td><code>3090-ai.foxhop.net</code></td><td>RTX 3090 (24 GB)</td><td>sm_86</td><td>9091</td><td>idle</td></tr>
|
<tr><td><code>3090-ai.foxhop.net</code></td><td>RTX 3090 (24 GB)</td><td>sm_86</td><td>9091</td><td><strong>active</strong> — production worker</td></tr>
|
||||||
<tr><td><code>ai.foxhop.net</code></td><td>RTX 4090 (24 GB)</td><td>sm_89</td><td>9092</td><td>qwen LLM (llama.cpp) on GPU</td></tr>
|
<tr><td><code>ai.foxhop.net</code></td><td>RTX 4090 (24 GB)</td><td>sm_89</td><td>9092</td><td>reserved for qwen LLM (llama.cpp); bend worker enabled per workload</td></tr>
|
||||||
</tbody>
|
</tbody>
|
||||||
</table>
|
</table>
|
||||||
<p>The 4090 box shares its GPU with a qwen LLM — bend kernels burn SMs for milliseconds then release; qwen keeps its weights resident; 24 GB VRAM holds both. Cluster aggregate (sequential round-robin dispatcher): 1.6× at 200 × <code>n=1k</code> mod-mul, 1.2× at 6 × <code>n=100k</code>; an async dispatcher unlocks the remaining 2× headroom.</p>
|
<p>Multi-host fan-out was validated at 1.6× aggregate throughput on small workloads, but routine round-robin against the 4090 would steal VRAM from qwen. Caller opts in explicitly when a workload justifies fan-out: <code>(bend-set-workers! '(("3090-ai.foxhop.net" . 9091) ("ai.foxhop.net" . 9092)))</code> brings the 4090 online for that call.</p>
|
||||||
</section>
|
</section>
|
||||||
|
|
||||||
<section id="catalog">
|
<section id="catalog">
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue