speech.py: disable tts-1-qwen registration — free VRAM for llama-qwen LLM

qwen3-tts (1.7B params) on speech.py was holding 6-7 GB VRAM that
collided with llama-qwen.service (Qwen3.6-27B, 20 GB @ -c 65536) on
the same 4090. Result: speech.py OOMed mid-inference, returning HTTP
500 InternalServerError that browsers surfaced as NetworkError. Reverse
order made llama-qwen OOM during model load.

Drop tts-1-qwen registration so only f5-tts (336M params, ~1.3 GB
VRAM) lands on the GPU. Speech requests for tts-1-f5 work; tts-1-qwen
returns 500 (model still listed in /v1/models via the loop at line
758 — refine to a proper 4xx in a later pass).

To reverse: uncomment line 1497 + reduce llama-qwen context (e.g.
-c 32768 frees ~10 GB, enough for both engines).
This commit is contained in:
timehexon 2026-06-07 08:39:49 -04:00
parent 33bd85f1cb
commit d4ff4e8da4

View file

@ -1494,7 +1494,7 @@ if __name__ == "__main__":
xtts = xtts_wrapper(args.preload, device=args.xtts_device, unload_timer=args.unload_timer)
# Register Qwen + F5 by default (other models disabled)
app.register_model('tts-1-qwen')
# app.register_model('tts-1-qwen') # disabled — frees VRAM for llama-qwen LLM
app.register_model('tts-1-f5')
# To enable other models, uncomment below:
# app.register_model('tts-1')