diff --git a/examples/cuda-fanout/bend.lsp b/examples/cuda-fanout/bend.lsp index afa8d03..c027c33 100644 --- a/examples/cuda-fanout/bend.lsp +++ b/examples/cuda-fanout/bend.lsp @@ -87,9 +87,14 @@ ;;; falls back to a single *bend-worker-host* / *bend-worker-port* pair ;;; (full back-compat with single-host callers). ;;; -;;; Set via (bend-set-workers! '(("3090-ai.foxhop.net" . 9091) -;;; ("ai.foxhop.net" . 9092))) +;;; Set via (bend-set-workers! '(("3090-ai.foxhop.net" . 9091))) ;;; or environment variable BEND_WORKERS="host:port,host:port". +;;; +;;; Production default: 3090-ai only. ai.foxhop.net (4090) is +;;; reserved for qwen LLM; we do NOT add it to the round-robin by +;;; default. Multi-host fan-out gets enabled per long-running +;;; parallel workload — caller opts in explicitly via +;;; bend-set-workers! or BEND_WORKERS env. (define *bend-workers* '()) (define *bend-rr-idx* 0) diff --git a/www/bend.html b/www/bend.html index a2c3c5b..bd3b6d0 100644 --- a/www/bend.html +++ b/www/bend.html @@ -70,15 +70,15 @@ make gpu-worker LUMBDA=asm # smallest footprint

Fleet

-

bend runs on a 2-host LAN cluster; round-robin selection lives in bend.lsp via *bend-workers* + BEND_WORKERS env.

+

Production default: single host. Multi-host fan-out is built (round-robin in bend.lsp via *bend-workers* + BEND_WORKERS env), available on demand for long-running parallel workloads — not used routinely.

- + - - + +
hostGPUarchportcoexists with
hostGPUarchportstatus
3090-ai.foxhop.netRTX 3090 (24 GB)sm_869091idle
ai.foxhop.netRTX 4090 (24 GB)sm_899092qwen LLM (llama.cpp) on GPU
3090-ai.foxhop.netRTX 3090 (24 GB)sm_869091active — production worker
ai.foxhop.netRTX 4090 (24 GB)sm_899092reserved for qwen LLM (llama.cpp); bend worker enabled per workload
-

The 4090 box shares its GPU with a qwen LLM — bend kernels burn SMs for milliseconds then release; qwen keeps its weights resident; 24 GB VRAM holds both. Cluster aggregate (sequential round-robin dispatcher): 1.6× at 200 × n=1k mod-mul, 1.2× at 6 × n=100k; an async dispatcher unlocks the remaining 2× headroom.

+

Multi-host fan-out was validated at 1.6× aggregate throughput on small workloads, but routine round-robin against the 4090 would steal VRAM from qwen. Caller opts in explicitly when a workload justifies fan-out: (bend-set-workers! '(("3090-ai.foxhop.net" . 9091) ("ai.foxhop.net" . 9092))) brings the 4090 online for that call.