Previous default ('--engines unset' == all engines enabled) made it too
easy for stray tts-1-qwen requests to silently load 10 GiB of weights
onto a GPU shared with an LLM. Flipped: the lean F5-only allowlist is
now the default, and operators explicitly opt into more.
python speech.py # default: {tts-1-f5}
python speech.py --engines f5,piper # F5 + Piper
python speech.py --engines all # every importable engine (old default)
python speech.py --engines qwen,f5 # back to Qwen + F5 for GPU servers
Implementation: _parse_engines_env() now returns set(DEFAULT_ENGINES)
when SPEECH_ENABLED_ENGINES is unset, and treats the literal 'all' as a
None sentinel (no allowlist applied). DEFAULT_ENGINES = frozenset({'tts-1-f5'}).
Workers still pick this up via SPEECH_ENABLED_ENGINES env var; __main__
only sets the var when --engines was passed, so default-path workers
re-parse the empty env -> DEFAULT_ENGINES path.
Operators can now restrict speech.py to a subset of TTS engines via
--engines (or the SPEECH_ENABLED_ENGINES env var). Disabled engines:
- hidden from /v1/voices
- short-circuited at the TTS request handler with a clean
BadRequestError ('Model X is not enabled on this server')
- skipped at app.register_model() time so they don't appear in
/v1/models
Default (--engines unset) preserves current behavior: every engine whose
Python deps are importable is enabled.
Motivation: running speech.py on a GPU shared with an LLM server (e.g.
llama-qwen on the same 4090) used to require commenting out
register_model lines AND auditing every caller to never hit
tts-1-qwen. Both forms of discipline broke in practice — speech.py was
seen squatting 10 GiB of VRAM for 3 days because the qwen branch loaded
on a stray request. Single allowlist closes that hole.
Usage:
python speech.py --engines f5,piper # lean: F5 + Piper only
python speech.py --engines tts-1-f5 # equivalent (full IDs OK)
python speech.py # default: all engines
Workers (uvicorn -W N) re-import this module, so the allowlist lives in
SPEECH_ENABLED_ENGINES env var (set by __main__ before uvicorn.run);
each worker re-parses at module load. Module-level set ENABLED_ENGINES
is populated by _parse_engines_env() — None = no restriction.
Validation routes through is_engine_available(model_id), which now ANDs
two gates: (a) operator allowlist, (b) Python-deps importable. Same
helper drives /v1/voices filtering, request-handler short-circuit, and
app.register_model() loop in __main__.
qwen3-tts (1.7B params) on speech.py was holding 6-7 GB VRAM that
collided with llama-qwen.service (Qwen3.6-27B, 20 GB @ -c 65536) on
the same 4090. Result: speech.py OOMed mid-inference, returning HTTP
500 InternalServerError that browsers surfaced as NetworkError. Reverse
order made llama-qwen OOM during model load.
Drop tts-1-qwen registration so only f5-tts (336M params, ~1.3 GB
VRAM) lands on the GPU. Speech requests for tts-1-f5 work; tts-1-qwen
returns 500 (model still listed in /v1/models via the loop at line
758 — refine to a proper 4xx in a later pass).
To reverse: uncomment line 1497 + reduce llama-qwen context (e.g.
-c 32768 frees ~10 GB, enough for both engines).
Records the pre_process_map.yaml respelling pattern we used for
Provenance so future blackops sessions don't have to rediscover
that preprocess() reads per-request and no restart is needed.
Both respellings present but disabled so you can hear before/after
without redeploying. Uncomment one or both lines on the live config
(preprocess() re-reads per request).
Earlier patch targeted the wrong word; the actual mispronunciation
fox heard was 'Provenance'. Respell as 'Prov-uh-nuns' to hint
syllables + closing /nz/ sound.
F5-TTS mispronounces "Providence" — add hyphenation regex
(Prov-uh-dence). Promote config/pre_process_map.yaml from
gitignored generated file to tracked repo file so the regex
map ships via git pull instead of needing per-host manual edits.
.default.yaml retained as seed for fresh installs.
Add `sse: bool` to the request. For tts-1-f5, stream text/event-stream:
one 'sentence' event per sentence the moment it renders, carrying that
sentence's mp3 (base64) plus exact start_ms/end_ms, then a 'done' event
with total duration_ms. Clients begin playback after the first sentence
(no wait for the full clip) and drive a per-sentence highlight off the
timing. A single generation feeds both audio and timing — no double render.
Add `timestamps: bool` to GenerateSpeechRequest. When set for tts-1-f5,
generate the whole clip up front and return JSON {audio (base64), format,
sample_rate, duration_ms, sentences:[{index,text,start_ms,end_ms}]} instead
of streaming audio. F5 already synthesizes one PCM chunk per sentence, so
each chunk's sample count is its exact duration — no forced aligner needed.
Lets clients highlight each sentence as it is spoken. Non-F5 models with
timestamps=true get a clear 400. Default off preserves streaming behavior.
Slot 41 intentionally empty (Hitchhiker tribute). Voice is self-recorded by
speaker, AGPL-cleared, registered in voice_registry.json under new
`self-recorded` corpus. ref_text is a placeholder pending `make whisper-refs`
on a GPU host.
cloned-voices/foxhop.wav 313K mono 16kHz s16 PCM, ~10s, -3dBFS peak
voice_registry.json +foxhop entry, +self-recorded corpus
voice_to_speaker.default.yaml +foxhop block in tts-1-qwen and tts-1-f5
cloned-voices/voices_metadata.json +foxhop entry
normalize_text() handles three Whisper quirks that produced messy F5
ref_texts:
1. Lowercase output with no terminal period — capitalize first letter,
append "." if missing.
2. Hallucinated "' clusters Whisper inserts when it interprets a
fragment as quoted dialogue (cora, ivan, atlas, hope had these).
Strip everywhere; never legitimate English punctuation.
3. Trailing apostrophe-then-period (.'.) from earlier rounds where a
closing-quoted line got an extra "." appended — collapse to single
terminal. Function is now idempotent.
Adds --from-cache flag: skip ASR, re-apply normalize from cached
whisper_refs.json. No GPU needed, useful after tuning the normalizer.
Lazy-imports torch so --from-cache works on any host.
Affects 9 of 40 voices: clara, grace, hazel, iris, felix, hugo
(lowercase fix); cora, ivan, atlas, hope (quote-cluster fix).
The prior docs claimed prod = 3090-ai running uncloseai-speech-server-1 via
docker compose. Reality (verified today): prod is bare python on the 4090
(ai.foxhop.net) using the f5-sidecar venv at /mnt/data/f5-sidecar/venv,
fronted by Caddy on 80/443. The container on 3090-ai exited 2 months ago and
docker compose v2 isn't installed there.
Cost of the drift today: one wrong-host deploy attempt that bounced off dead
infrastructure. This rewrites Production Deployment, Quick Production
Commands, Testing, When Things Break, and Common Mistakes to match reality.
Also drops hardcoded tmux window numbers (they shift) — every example now
reads <prod-window> with a reminder to rediscover via tmux-hosts.
Previous ref_texts were LibriSpeech dataset labels (ALL CAPS, no
punctuation). F5-TTS conditions on ref_text to align reference audio
prosody — commas, periods, casing matter. Now using whisper-large-v3
transcripts of the actual cloned-voices/*.wav files.
Generated via: make whisper-refs (on a GPU host).
Affects all 40 voices in both tts-1-f5 and tts-1-qwen engine blocks.
Sidecar cloned-voices/whisper_refs.json kept for traceability.
F5-TTS cloning quality depends on ref_text matching the prosody of ref_audio
(commas, periods, casing). Previous ref_texts were LibriSpeech ground-truth
labels: ALL CAPS, no punctuation — wrong signal for a flow-matching TTS
conditioned on text. Whisper hears what F5 will hear.
- scripts/whisper_refs.py — transcribe all wavs, rewrite
voice_to_speaker.default.yaml + cloned-voices/voices_metadata.json
in place. Also writes cloned-voices/whisper_refs.json sidecar.
- Makefile: whisper-refs target. Idempotent, rerun whenever
cloned-voices/ changes.
Run on a GPU host (4090/3090). ~30s for 40 short clips on a 4090.
Runs scripts/download_diverse_voices.py which pulls speaker samples
from LibriSpeech and assigns permanent human names via voice registry.
Works with venv or system python. Also adds Voices section to help.
- voice_registry.json: append-only registry with 50 name pools per gender,
locked speaker assignments, and multi-corpus support
- Rewrite download script to be registry-driven: loads registry, assigns
names deterministically (sorted by speaker ID), never changes existing
assignments
- Update docs/VOICES.md with registry system documentation
- Support --registry and --corpora CLI flags for multi-corpus downloads
Firefox MediaSource API supports audio/webm;codecs=opus but not
audio/ogg. Adding webm format lets Firefox clients use true
streaming playback via MediaSource instead of full buffering.
Previous version accumulated sentences until 500 chars, defeating
the purpose of streaming. Now splits on every sentence, only
combining very short sentences (<50 chars) with the next.
- Fix subprocess deadlock in Qwen TTS by using threading for stdin write
(prevents pipe buffer deadlock on large audio output)
- Set WORKERS=1 for GPU models to avoid VRAM duplication
(4 workers × 3GB model = OOM, 1 worker works fine)
- Update CLAUDE.md: use git push/pull instead of rsync for deployment
