Make Qwen3-TTS the default engine, add CPU-only docker support
- Switch default TTS engine from Piper to Qwen3-TTS (1.7B params) - Upgrade to Python 3.12 - Add docker-compose.cpu.yml for CPU-only deployments - Improve GPU configuration with NVIDIA environment variables - Comment out optional engines (Piper, XTTS, Silero, Kokoro) in requirements - Update Makefile with local/local-cpu targets and venv support - Simplify voice_to_speaker.default.yaml for Qwen3-TTS voices - Update docs/MODELS.md with Qwen3-TTS documentation - Add git commit guidelines to CLAUDE.md
This commit is contained in:
parent
4a019cf897
commit
b315659be6
9 changed files with 572 additions and 951 deletions
104
docs/MODELS.md
104
docs/MODELS.md
|
|
@ -2,6 +2,22 @@
|
|||
|
||||
**Raccoon Mission:** Rescue abandoned open-source TTS models and integrate them into uncloseai-speech
|
||||
|
||||
## Default Model: Qwen3-TTS
|
||||
|
||||
**🎯 Qwen3-TTS is now the default and only enabled model.**
|
||||
|
||||
All other models (Piper, XTTS, Silero, Kokoro) are disabled by default. To enable them, uncomment their sections in `voice_to_speaker.yaml`.
|
||||
|
||||
### Quick Start
|
||||
```bash
|
||||
# Test Qwen3-TTS (default)
|
||||
make test
|
||||
|
||||
# The model downloads automatically on first use (~3.4GB)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Documentation Index
|
||||
|
||||
### Comprehensive Research
|
||||
|
|
@ -33,7 +49,83 @@ Each model has detailed documentation covering technical specs, integration stat
|
|||
|
||||
## Currently Integrated
|
||||
|
||||
### 1. Piper TTS ✅
|
||||
### 0. Qwen3-TTS ✅ (DEFAULT)
|
||||
|
||||
**Status:** INTEGRATED as tts-1-qwen (DEFAULT MODEL)
|
||||
**Project:** Qwen/Qwen3-TTS (Alibaba, actively maintained)
|
||||
**License:** Apache 2.0
|
||||
**Model:** Qwen3-TTS-12Hz-1.7B-Base
|
||||
|
||||
**Why Default:**
|
||||
- State-of-the-art quality with 1.7B parameters
|
||||
- Extremely low latency (97ms first packet)
|
||||
- Voice cloning from 3-second samples
|
||||
- 10 languages: Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, Italian
|
||||
- Apache 2.0 license (commercial-friendly)
|
||||
- Actively maintained by Alibaba
|
||||
|
||||
**Features:**
|
||||
- Universal end-to-end architecture (no cascading errors)
|
||||
- 12Hz acoustic tokenizer for efficient compression
|
||||
- Dual-track streaming/non-streaming generation
|
||||
- High-fidelity speech reconstruction
|
||||
- Natural language instruction control
|
||||
- Supports both GPU and CPU inference
|
||||
|
||||
**Model Specs:**
|
||||
- Parameters: 1.7B
|
||||
- Sample Rate: ~24kHz
|
||||
- Input: Text + Reference Audio (3+ seconds)
|
||||
- Languages: 10 (zh, en, ja, ko, de, fr, ru, pt, es, it)
|
||||
- Size: ~3.4GB
|
||||
|
||||
**Model Source:**
|
||||
- HuggingFace: `Qwen/Qwen3-TTS-12Hz-1.7B-Base`
|
||||
- Auto-downloaded on first use via huggingface-hub
|
||||
- Cached in `/app/voices/hub/`
|
||||
|
||||
**Integration:**
|
||||
- Used for `tts-1-qwen` model (default)
|
||||
- Voice cloning with reference audio + transcript
|
||||
- Pre-configured with Qwen's demo voice
|
||||
|
||||
**Example Config:**
|
||||
```yaml
|
||||
tts-1-qwen:
|
||||
alloy:
|
||||
ref_audio: https://example.com/reference.wav
|
||||
ref_text: "The exact text spoken in the reference audio"
|
||||
language: English
|
||||
```
|
||||
|
||||
**Custom Voice Setup:**
|
||||
1. Record 3+ seconds of clear speech
|
||||
2. Transcribe the audio exactly
|
||||
3. Add to `voice_to_speaker.yaml`:
|
||||
```yaml
|
||||
tts-1-qwen:
|
||||
my_voice:
|
||||
ref_audio: voices/my_voice_sample.wav
|
||||
ref_text: "Hello, this is my voice sample for cloning."
|
||||
language: English
|
||||
```
|
||||
|
||||
**Makefile Targets:**
|
||||
```bash
|
||||
make test # Test Qwen3-TTS (default)
|
||||
make test-qwen # Test Qwen3-TTS explicitly
|
||||
```
|
||||
|
||||
**Hardware Requirements:**
|
||||
- GPU: NVIDIA with 8GB+ VRAM (recommended)
|
||||
- CPU: Works but slower (~10x)
|
||||
- FlashAttention 2 recommended for lower memory
|
||||
|
||||
**Raccoon Priority:** ⭐⭐⭐⭐⭐ (State-of-the-art, actively maintained, Apache 2.0)
|
||||
|
||||
---
|
||||
|
||||
### 1. Piper TTS (disabled by default) ✅
|
||||
|
||||
> 📖 **See [detailed documentation](models/piper-tts.md)** for comprehensive technical specs and integration guide
|
||||
|
||||
|
|
@ -505,8 +597,8 @@ The following models have detailed documentation but are not yet integrated or p
|
|||
|
||||
---
|
||||
|
||||
**Last Updated:** 2025-11-09
|
||||
**Raccoon Status:** 🦝 4 models rescued! Silero and Kokoro TTS integrated successfully
|
||||
**Integration Status:** ✅ Piper (55 voices), XTTS (8 voices), Silero (148 voices), Kokoro (34 voices) | 🎯 Next: Chatterbox, StyleTTS2
|
||||
**API Endpoints:** tts-1, tts-1-hd, tts-1-silero, tts-1-kokoro | /v1/models for discovery
|
||||
**Documentation Status:** 📚 10 models fully documented, 1 comprehensive research overview
|
||||
**Last Updated:** 2026-01-26
|
||||
**Raccoon Status:** 🦝 5 models rescued! Qwen3-TTS is now the default model
|
||||
**Integration Status:** ✅ Qwen3-TTS (default, unlimited voices via cloning) | Disabled: Piper (55), XTTS (8), Silero (148), Kokoro (34)
|
||||
**API Endpoints:** tts-1-qwen (default) | Others available: tts-1, tts-1-hd, tts-1-silero, tts-1-kokoro
|
||||
**Documentation Status:** 📚 11 models fully documented, 1 comprehensive research overview
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue