From 9e4e9b48f973ff94baf4631b404b6e948efc8882 Mon Sep 17 00:00:00 2001 From: "russell@unturf.com" Date: Fri, 5 Jun 2026 21:55:41 -0400 Subject: [PATCH] bend: 3090-only production default; 4090 reserved for qwen MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fox decision: don't routine-fan-out to ai.foxhop.net (4090) when qwen LLM holds GPU residency. The radix-sort 4090 OOM caveat surfaced today demonstrated the cost of casual co-residency — secp daemon parked 24 GiB up front leaving 47 MiB free. bend.lsp's *bend-workers* default already empty (single-host fallback). Updated the docstring example to drop the ai.foxhop.net entry; multi-host fan-out is OPT-IN per call via bend-set-workers! or BEND_WORKERS env. bend.html fleet section now reflects the policy: - 3090-ai.foxhop.net:9091 active production worker - ai.foxhop.net:9092 reserved for qwen; bend per workload ai.foxhop.net worker process killed; 4090 VRAM returned to qwen (1.6 GiB free post-kill vs 47 MiB while bend was running). When we have a long-running parallel sweep that justifies the 4090's marginal throughput, the caller opts in explicitly. Don't auto-route. --- examples/cuda-fanout/bend.lsp | 9 +++++++-- www/bend.html | 10 +++++----- 2 files changed, 12 insertions(+), 7 deletions(-) diff --git a/examples/cuda-fanout/bend.lsp b/examples/cuda-fanout/bend.lsp index afa8d03..c027c33 100644 --- a/examples/cuda-fanout/bend.lsp +++ b/examples/cuda-fanout/bend.lsp @@ -87,9 +87,14 @@ ;;; falls back to a single *bend-worker-host* / *bend-worker-port* pair ;;; (full back-compat with single-host callers). ;;; -;;; Set via (bend-set-workers! '(("3090-ai.foxhop.net" . 9091) -;;; ("ai.foxhop.net" . 9092))) +;;; Set via (bend-set-workers! '(("3090-ai.foxhop.net" . 9091))) ;;; or environment variable BEND_WORKERS="host:port,host:port". +;;; +;;; Production default: 3090-ai only. ai.foxhop.net (4090) is +;;; reserved for qwen LLM; we do NOT add it to the round-robin by +;;; default. Multi-host fan-out gets enabled per long-running +;;; parallel workload — caller opts in explicitly via +;;; bend-set-workers! or BEND_WORKERS env. (define *bend-workers* '()) (define *bend-rr-idx* 0) diff --git a/www/bend.html b/www/bend.html index a2c3c5b..bd3b6d0 100644 --- a/www/bend.html +++ b/www/bend.html @@ -70,15 +70,15 @@ make gpu-worker LUMBDA=asm # smallest footprint

Fleet

-

bend runs on a 2-host LAN cluster; round-robin selection lives in bend.lsp via *bend-workers* + BEND_WORKERS env.

+

Production default: single host. Multi-host fan-out is built (round-robin in bend.lsp via *bend-workers* + BEND_WORKERS env), available on demand for long-running parallel workloads — not used routinely.

- + - - + +
hostGPUarchportcoexists with
hostGPUarchportstatus
3090-ai.foxhop.netRTX 3090 (24 GB)sm_869091idle
ai.foxhop.netRTX 4090 (24 GB)sm_899092qwen LLM (llama.cpp) on GPU
3090-ai.foxhop.netRTX 3090 (24 GB)sm_869091active — production worker
ai.foxhop.netRTX 4090 (24 GB)sm_899092reserved for qwen LLM (llama.cpp); bend worker enabled per workload
-

The 4090 box shares its GPU with a qwen LLM — bend kernels burn SMs for milliseconds then release; qwen keeps its weights resident; 24 GB VRAM holds both. Cluster aggregate (sequential round-robin dispatcher): 1.6× at 200 × n=1k mod-mul, 1.2× at 6 × n=100k; an async dispatcher unlocks the remaining 2× headroom.

+

Multi-host fan-out was validated at 1.6× aggregate throughput on small workloads, but routine round-robin against the 4090 would steal VRAM from qwen. Caller opts in explicitly when a workload justifies fan-out: (bend-set-workers! '(("3090-ai.foxhop.net" . 9091) ("ai.foxhop.net" . 9092))) brings the 4090 online for that call.