Add F5-TTS as tts-1-f5 engine (additive, alongside tts-1-qwen)

This commit is contained in:
russell@unturf.com 2026-05-23 13:24:25 -04:00
parent bfc5260dfc
commit 2df34f85cc
No known key found for this signature in database
6 changed files with 511 additions and 5 deletions

View file

@ -125,6 +125,66 @@ make test-qwen # Test Qwen3-TTS explicitly
---
### F5-TTS ✅ (also enabled by default)
**Status:** INTEGRATED as tts-1-f5 (additive, alongside tts-1-qwen)
**Project:** SWivid/F5-TTS (community-maintained, gated HuggingFace checkpoint)
**License:** MIT (model + code)
**Model:** F5-TTS_v1
**Why Integrated:**
- Empirical benchmark (Richard, 2026-05-23): faster inference + better voice clones than Qwen3-TTS on the same reference clips
- Smaller model (~336M params vs Qwen3-TTS 1.7B) — lower VRAM, fits comfortably on modest GPUs
- Flow-matching architecture, zero-shot cloning, no fine-tuning needed
- 24kHz output, matches Qwen3-TTS sample rate (drop-in voice swap for clients)
**Model Specs:**
- Parameters: ~336M
- Sample Rate: 24kHz
- Input: Text + Reference Audio (3+ seconds) + Reference Transcript
- Languages: English (primary); community fine-tunes available for others
- Size: ~1.5GB (F5-TTS_v1 + Vocos vocoder)
**Model Source:**
- HuggingFace: `SWivid/F5-TTS` (**gated** — accept license at https://huggingface.co/SWivid/F5-TTS)
- Auto-downloaded on first use; requires `HF_TOKEN` env var
**Integration:**
- Used for `tts-1-f5` model (additive default)
- Voice cloning with `ref_audio` + `ref_text` (same shape as tts-1-qwen)
- Reuses the same 40 LibriSpeech voices as tts-1-qwen
**Example Config:**
```yaml
tts-1-f5:
aria:
ref_audio: cloned-voices/aria.wav
ref_text: "BUT THE WINDOWS ARE PATCHED WITH WOODEN PANES AND THE DOOR I THINK IS LIKE THE GATE IT IS NEVER OPENED"
language: English
```
**Tuning Knobs (engine-specific, not OpenAI-compatible):**
- `nfe_step` (default 32) — ODE step count; higher = better quality, slower
- `cfg_strength` (default 2.0) — classifier-free guidance strength
- `speed` (default 1.0) — pitch-preserving speed multiplier
- Does NOT support `temperature` / `top_p` / `top_k` (flow-matching, not autoregressive)
**Makefile Targets:**
```bash
make test-f5 # Test F5-TTS voice cloning
```
**Hardware Requirements:**
- GPU: NVIDIA with 4GB+ VRAM (lighter than Qwen3-TTS)
- CPU: Works but many× realtime
- MPS (Apple Silicon): supported, ~1.52× realtime per VoiceClone benchmarks
**Source of Inspiration:** [MonumentalSystems/VoiceClone](https://github.com/MonumentalSystems/VoiceClone) — a single-file F5-TTS web app that proved the engine on our reference workload. Our wrapper mirrors their `F5TTS.infer()` call pattern.
**Raccoon Priority:** ⭐⭐⭐⭐⭐ (Empirically beats current default, MIT, lower VRAM)
---
### 1. Piper TTS (disabled by default) ✅
> 📖 **See [detailed documentation](models/piper-tts.md)** for comprehensive technical specs and integration guide