engineering/lumbda
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

bend.html: qwen runs on llama.cpp, not vllm

Browse source
This commit is contained in:
russell@unturf.com 2026-06-05 20:59:06 -04:00
parent 01a36f3fcd
commit dca3c2d016
No known key found for this signature in database
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
www/bend.html
View file

@ -104,7 +104,7 @@ make gpu-worker LUMBDA=asm # smallest footprint</code></pre>
</tbody>
</table>
<p>The 4090 box shares the GPU with a qwen LLM (vllm). Our bend kernels fit alongside qwen comfortably because the secp256k1 / CGBN workloads burn the SMs only for milliseconds at a time, then release. The qwen server keeps its model weights resident; our kernels allocate transient VRAM for the per-call batch, run, free, return. 24 GB VRAM is enough for both.</p>
<p>The 4090 box shares the GPU with a qwen LLM (llama.cpp). Our bend kernels fit alongside qwen comfortably because the secp256k1 / CGBN workloads burn the SMs only for milliseconds at a time, then release. The qwen server keeps its model weights resident; our kernels allocate transient VRAM for the per-call batch, run, free, return. 24 GB VRAM is enough for both.</p>
<h3>Cross-GPU parity at small batches</h3>
<p>Single-call kernel time at <code>n=100k</code> CGBN 256-bit mod-mul:</p>