From dca3c2d0162db57ac83b6a67dcf6a1a96859c776 Mon Sep 17 00:00:00 2001 From: "russell@unturf.com" Date: Fri, 5 Jun 2026 20:59:06 -0400 Subject: [PATCH] bend.html: qwen runs on llama.cpp, not vllm --- www/bend.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/www/bend.html b/www/bend.html index b3a35ad..01bb7d3 100644 --- a/www/bend.html +++ b/www/bend.html @@ -104,7 +104,7 @@ make gpu-worker LUMBDA=asm # smallest footprint -

The 4090 box shares the GPU with a qwen LLM (vllm). Our bend kernels fit alongside qwen comfortably because the secp256k1 / CGBN workloads burn the SMs only for milliseconds at a time, then release. The qwen server keeps its model weights resident; our kernels allocate transient VRAM for the per-call batch, run, free, return. 24 GB VRAM is enough for both.

+

The 4090 box shares the GPU with a qwen LLM (llama.cpp). Our bend kernels fit alongside qwen comfortably because the secp256k1 / CGBN workloads burn the SMs only for milliseconds at a time, then release. The qwen server keeps its model weights resident; our kernels allocate transient VRAM for the per-call batch, run, free, return. 24 GB VRAM is enough for both.

Cross-GPU parity at small batches

Single-call kernel time at n=100k CGBN 256-bit mod-mul: