Add F5-TTS as tts-1-f5 engine (additive, alongside tts-1-qwen)
This commit is contained in:
parent
bfc5260dfc
commit
2df34f85cc
6 changed files with 511 additions and 5 deletions
|
|
@ -125,6 +125,66 @@ make test-qwen # Test Qwen3-TTS explicitly
|
|||
|
||||
---
|
||||
|
||||
### F5-TTS ✅ (also enabled by default)
|
||||
|
||||
**Status:** INTEGRATED as tts-1-f5 (additive, alongside tts-1-qwen)
|
||||
**Project:** SWivid/F5-TTS (community-maintained, gated HuggingFace checkpoint)
|
||||
**License:** MIT (model + code)
|
||||
**Model:** F5-TTS_v1
|
||||
|
||||
**Why Integrated:**
|
||||
- Empirical benchmark (Richard, 2026-05-23): faster inference + better voice clones than Qwen3-TTS on the same reference clips
|
||||
- Smaller model (~336M params vs Qwen3-TTS 1.7B) — lower VRAM, fits comfortably on modest GPUs
|
||||
- Flow-matching architecture, zero-shot cloning, no fine-tuning needed
|
||||
- 24kHz output, matches Qwen3-TTS sample rate (drop-in voice swap for clients)
|
||||
|
||||
**Model Specs:**
|
||||
- Parameters: ~336M
|
||||
- Sample Rate: 24kHz
|
||||
- Input: Text + Reference Audio (3+ seconds) + Reference Transcript
|
||||
- Languages: English (primary); community fine-tunes available for others
|
||||
- Size: ~1.5GB (F5-TTS_v1 + Vocos vocoder)
|
||||
|
||||
**Model Source:**
|
||||
- HuggingFace: `SWivid/F5-TTS` (**gated** — accept license at https://huggingface.co/SWivid/F5-TTS)
|
||||
- Auto-downloaded on first use; requires `HF_TOKEN` env var
|
||||
|
||||
**Integration:**
|
||||
- Used for `tts-1-f5` model (additive default)
|
||||
- Voice cloning with `ref_audio` + `ref_text` (same shape as tts-1-qwen)
|
||||
- Reuses the same 40 LibriSpeech voices as tts-1-qwen
|
||||
|
||||
**Example Config:**
|
||||
```yaml
|
||||
tts-1-f5:
|
||||
aria:
|
||||
ref_audio: cloned-voices/aria.wav
|
||||
ref_text: "BUT THE WINDOWS ARE PATCHED WITH WOODEN PANES AND THE DOOR I THINK IS LIKE THE GATE IT IS NEVER OPENED"
|
||||
language: English
|
||||
```
|
||||
|
||||
**Tuning Knobs (engine-specific, not OpenAI-compatible):**
|
||||
- `nfe_step` (default 32) — ODE step count; higher = better quality, slower
|
||||
- `cfg_strength` (default 2.0) — classifier-free guidance strength
|
||||
- `speed` (default 1.0) — pitch-preserving speed multiplier
|
||||
- Does NOT support `temperature` / `top_p` / `top_k` (flow-matching, not autoregressive)
|
||||
|
||||
**Makefile Targets:**
|
||||
```bash
|
||||
make test-f5 # Test F5-TTS voice cloning
|
||||
```
|
||||
|
||||
**Hardware Requirements:**
|
||||
- GPU: NVIDIA with 4GB+ VRAM (lighter than Qwen3-TTS)
|
||||
- CPU: Works but many× realtime
|
||||
- MPS (Apple Silicon): supported, ~1.5–2× realtime per VoiceClone benchmarks
|
||||
|
||||
**Source of Inspiration:** [MonumentalSystems/VoiceClone](https://github.com/MonumentalSystems/VoiceClone) — a single-file F5-TTS web app that proved the engine on our reference workload. Our wrapper mirrors their `F5TTS.infer()` call pattern.
|
||||
|
||||
**Raccoon Priority:** ⭐⭐⭐⭐⭐ (Empirically beats current default, MIT, lower VRAM)
|
||||
|
||||
---
|
||||
|
||||
### 1. Piper TTS (disabled by default) ✅
|
||||
|
||||
> 📖 **See [detailed documentation](models/piper-tts.md)** for comprehensive technical specs and integration guide
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue