Earlier patch targeted the wrong word; the actual mispronunciation
fox heard was 'Provenance'. Respell as 'Prov-uh-nuns' to hint
syllables + closing /nz/ sound.
F5-TTS mispronounces "Providence" — add hyphenation regex
(Prov-uh-dence). Promote config/pre_process_map.yaml from
gitignored generated file to tracked repo file so the regex
map ships via git pull instead of needing per-host manual edits.
.default.yaml retained as seed for fresh installs.
Add `sse: bool` to the request. For tts-1-f5, stream text/event-stream:
one 'sentence' event per sentence the moment it renders, carrying that
sentence's mp3 (base64) plus exact start_ms/end_ms, then a 'done' event
with total duration_ms. Clients begin playback after the first sentence
(no wait for the full clip) and drive a per-sentence highlight off the
timing. A single generation feeds both audio and timing — no double render.
Add `timestamps: bool` to GenerateSpeechRequest. When set for tts-1-f5,
generate the whole clip up front and return JSON {audio (base64), format,
sample_rate, duration_ms, sentences:[{index,text,start_ms,end_ms}]} instead
of streaming audio. F5 already synthesizes one PCM chunk per sentence, so
each chunk's sample count is its exact duration — no forced aligner needed.
Lets clients highlight each sentence as it is spoken. Non-F5 models with
timestamps=true get a clear 400. Default off preserves streaming behavior.
Slot 41 intentionally empty (Hitchhiker tribute). Voice is self-recorded by
speaker, AGPL-cleared, registered in voice_registry.json under new
`self-recorded` corpus. ref_text is a placeholder pending `make whisper-refs`
on a GPU host.
cloned-voices/foxhop.wav 313K mono 16kHz s16 PCM, ~10s, -3dBFS peak
voice_registry.json +foxhop entry, +self-recorded corpus
voice_to_speaker.default.yaml +foxhop block in tts-1-qwen and tts-1-f5
cloned-voices/voices_metadata.json +foxhop entry
normalize_text() handles three Whisper quirks that produced messy F5
ref_texts:
1. Lowercase output with no terminal period — capitalize first letter,
append "." if missing.
2. Hallucinated "' clusters Whisper inserts when it interprets a
fragment as quoted dialogue (cora, ivan, atlas, hope had these).
Strip everywhere; never legitimate English punctuation.
3. Trailing apostrophe-then-period (.'.) from earlier rounds where a
closing-quoted line got an extra "." appended — collapse to single
terminal. Function is now idempotent.
Adds --from-cache flag: skip ASR, re-apply normalize from cached
whisper_refs.json. No GPU needed, useful after tuning the normalizer.
Lazy-imports torch so --from-cache works on any host.
Affects 9 of 40 voices: clara, grace, hazel, iris, felix, hugo
(lowercase fix); cora, ivan, atlas, hope (quote-cluster fix).
The prior docs claimed prod = 3090-ai running uncloseai-speech-server-1 via
docker compose. Reality (verified today): prod is bare python on the 4090
(ai.foxhop.net) using the f5-sidecar venv at /mnt/data/f5-sidecar/venv,
fronted by Caddy on 80/443. The container on 3090-ai exited 2 months ago and
docker compose v2 isn't installed there.
Cost of the drift today: one wrong-host deploy attempt that bounced off dead
infrastructure. This rewrites Production Deployment, Quick Production
Commands, Testing, When Things Break, and Common Mistakes to match reality.
Also drops hardcoded tmux window numbers (they shift) — every example now
reads <prod-window> with a reminder to rediscover via tmux-hosts.
Previous ref_texts were LibriSpeech dataset labels (ALL CAPS, no
punctuation). F5-TTS conditions on ref_text to align reference audio
prosody — commas, periods, casing matter. Now using whisper-large-v3
transcripts of the actual cloned-voices/*.wav files.
Generated via: make whisper-refs (on a GPU host).
Affects all 40 voices in both tts-1-f5 and tts-1-qwen engine blocks.
Sidecar cloned-voices/whisper_refs.json kept for traceability.
F5-TTS cloning quality depends on ref_text matching the prosody of ref_audio
(commas, periods, casing). Previous ref_texts were LibriSpeech ground-truth
labels: ALL CAPS, no punctuation — wrong signal for a flow-matching TTS
conditioned on text. Whisper hears what F5 will hear.
- scripts/whisper_refs.py — transcribe all wavs, rewrite
voice_to_speaker.default.yaml + cloned-voices/voices_metadata.json
in place. Also writes cloned-voices/whisper_refs.json sidecar.
- Makefile: whisper-refs target. Idempotent, rerun whenever
cloned-voices/ changes.
Run on a GPU host (4090/3090). ~30s for 40 short clips on a 4090.
Runs scripts/download_diverse_voices.py which pulls speaker samples
from LibriSpeech and assigns permanent human names via voice registry.
Works with venv or system python. Also adds Voices section to help.
- voice_registry.json: append-only registry with 50 name pools per gender,
locked speaker assignments, and multi-corpus support
- Rewrite download script to be registry-driven: loads registry, assigns
names deterministically (sorted by speaker ID), never changes existing
assignments
- Update docs/VOICES.md with registry system documentation
- Support --registry and --corpora CLI flags for multi-corpus downloads
Firefox MediaSource API supports audio/webm;codecs=opus but not
audio/ogg. Adding webm format lets Firefox clients use true
streaming playback via MediaSource instead of full buffering.
Previous version accumulated sentences until 500 chars, defeating
the purpose of streaming. Now splits on every sentence, only
combining very short sentences (<50 chars) with the next.
- Fix subprocess deadlock in Qwen TTS by using threading for stdin write
(prevents pipe buffer deadlock on large audio output)
- Set WORKERS=1 for GPU models to avoid VRAM duplication
(4 workers × 3GB model = OOM, 1 worker works fine)
- Update CLAUDE.md: use git push/pull instead of rsync for deployment
