Fix Kokoro TTS integration - correct KPipeline API
- Removed model_path parameter (not supported by kokoro package) - Removed repo_id parameter (causes KeyError) - Use default KPipeline initialization with only lang_code - Kokoro package handles model download automatically Tested and working: - American English voices (alloy, af_sarah, am_michael, etc.) - British English voices (bm_george, bf_emma, etc.) - Audio generation produces valid MP3 files 🦝 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
parent
f8d46e92d5
commit
9b5caadb8f
3 changed files with 59 additions and 12 deletions
38
README.md
38
README.md
|
|
@ -9,9 +9,11 @@
|
||||||
**Original Notice:** This software was mostly obsolete and no longer updated by the original maintainer.
|
**Original Notice:** This software was mostly obsolete and no longer updated by the original maintainer.
|
||||||
|
|
||||||
**Raccoon Mission:** We're bringing it back to life with:
|
**Raccoon Mission:** We're bringing it back to life with:
|
||||||
- ✅ Working Piper TTS (tts-1) with absolute paths
|
- ✅ Working Piper TTS (tts-1) - 55 voices, fast CPU inference
|
||||||
- ✅ Working XTTS v2 (tts-1-hd) with voice cloning
|
- ✅ Working XTTS v2 (tts-1-hd) - 8 voices with cloning capability
|
||||||
- 🎯 Planning integration of 10+ abandoned TTS engines (Silero, StyleTTS2, Fish Speech, etc.)
|
- ✅ Working Silero TTS (tts-1-silero) - 148 voices, 5 languages, CPU-friendly
|
||||||
|
- ✅ Working Kokoro TTS (tts-1-kokoro) - 34 voices, lightweight decoder
|
||||||
|
- 🎯 Next integrations: StyleTTS2 (best quality), Fish Speech (fast multilingual)
|
||||||
- 📚 Comprehensive documentation in `docs/`
|
- 📚 Comprehensive documentation in `docs/`
|
||||||
- 🛠️ Makefile-driven deployment workflow
|
- 🛠️ Makefile-driven deployment workflow
|
||||||
- 🔒 AGPL v3 - keeps TTS libre forever
|
- 🔒 AGPL v3 - keeps TTS libre forever
|
||||||
|
|
@ -30,10 +32,12 @@ An OpenAI API compatible text to speech server.
|
||||||
Full Compatibility:
|
Full Compatibility:
|
||||||
* `tts-1`: `alloy`, `echo`, `fable`, `onyx`, `nova`, and `shimmer` (configurable)
|
* `tts-1`: `alloy`, `echo`, `fable`, `onyx`, `nova`, and `shimmer` (configurable)
|
||||||
* `tts-1-hd`: `alloy`, `echo`, `fable`, `onyx`, `nova`, and `shimmer` (configurable, uses OpenAI samples by default)
|
* `tts-1-hd`: `alloy`, `echo`, `fable`, `onyx`, `nova`, and `shimmer` (configurable, uses OpenAI samples by default)
|
||||||
|
* `tts-1-silero`: `alloy`, `echo`, `fable`, `onyx`, `nova`, and `shimmer` (148 total voices available)
|
||||||
|
* `tts-1-kokoro`: `alloy`, `echo`, `fable`, `onyx`, `nova`, and `shimmer` (34 total voices available)
|
||||||
* response_format: `mp3`, `opus`, `aac`, `flac`, `wav` and `pcm`
|
* response_format: `mp3`, `opus`, `aac`, `flac`, `wav` and `pcm`
|
||||||
* speed 0.25-4.0 (and more)
|
* speed 0.25-4.0 (and more)
|
||||||
|
|
||||||
Details:
|
Available TTS Engines:
|
||||||
* Model `tts-1` via [piper tts](https://github.com/rhasspy/piper) (very fast, runs on cpu)
|
* Model `tts-1` via [piper tts](https://github.com/rhasspy/piper) (very fast, runs on cpu)
|
||||||
* You can map your own [piper voices](https://rhasspy.github.io/piper-samples/) via the `voice_to_speaker.yaml` configuration file
|
* You can map your own [piper voices](https://rhasspy.github.io/piper-samples/) via the `voice_to_speaker.yaml` configuration file
|
||||||
* Model `tts-1-hd` via [coqui-ai/TTS](https://github.com/coqui-ai/TTS) xtts_v2 voice cloning (fast, but requires around 4GB GPU VRAM)
|
* Model `tts-1-hd` via [coqui-ai/TTS](https://github.com/coqui-ai/TTS) xtts_v2 voice cloning (fast, but requires around 4GB GPU VRAM)
|
||||||
|
|
@ -42,9 +46,35 @@ Details:
|
||||||
* [Custom fine-tuned XTTS model support](#custom-fine-tuned-model-support)
|
* [Custom fine-tuned XTTS model support](#custom-fine-tuned-model-support)
|
||||||
* Configurable [generation parameters](#generation-parameters)
|
* Configurable [generation parameters](#generation-parameters)
|
||||||
* Streamed output while generating
|
* Streamed output while generating
|
||||||
|
* Model `tts-1-silero` via [Silero TTS](https://github.com/snakers4/silero-models) (fast CPU inference, actively maintained)
|
||||||
|
* 148 voices across 5 languages (English, Russian, German, Spanish, French)
|
||||||
|
* 48kHz sample rate, excellent quality/speed ratio
|
||||||
|
* No GPU required, real-time capable on CPU
|
||||||
|
* Model `tts-1-kokoro` via [Kokoro TTS](https://github.com/hexgrad/kokoro) (lightweight decoder-only architecture)
|
||||||
|
* 34 voices (American and British English)
|
||||||
|
* 82M parameters, fast inference
|
||||||
|
* 24kHz sample rate, Apache 2.0 license
|
||||||
* Occasionally, certain words or symbols may sound incorrect, you can fix them with regex via `pre_process_map.yaml`
|
* Occasionally, certain words or symbols may sound incorrect, you can fix them with regex via `pre_process_map.yaml`
|
||||||
* Tested with python 3.9-3.11, piper does not install on python 3.12 yet
|
* Tested with python 3.9-3.11, piper does not install on python 3.12 yet
|
||||||
|
|
||||||
|
## High Priority Integration Targets
|
||||||
|
|
||||||
|
We're actively working on integrating these state-of-the-art TTS engines:
|
||||||
|
|
||||||
|
1. **StyleTTS2** ⭐⭐⭐⭐⭐
|
||||||
|
- Why: State-of-the-art quality, best prosody and naturalness
|
||||||
|
- License: MIT (permissive)
|
||||||
|
- Challenge: Complex dependencies (phonemizer), slower inference
|
||||||
|
- Priority: HIGH - Best quality available
|
||||||
|
|
||||||
|
2. **Fish Speech** ⭐⭐⭐⭐
|
||||||
|
- Why: Fast, modern, active development, good multilingual support
|
||||||
|
- License: Apache 2.0
|
||||||
|
- Challenge: Newer/less proven
|
||||||
|
- Priority: MEDIUM-HIGH - Good balance of quality and speed
|
||||||
|
|
||||||
|
See [docs/MODELS.md](docs/MODELS.md) for the complete integration roadmap and detailed documentation on all supported and planned TTS engines.
|
||||||
|
|
||||||
|
|
||||||
If you find a better voice match for `tts-1` or `tts-1-hd`, please let me know so I can update the defaults.
|
If you find a better voice match for `tts-1` or `tts-1-hd`, please let me know so I can update the defaults.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -122,14 +122,31 @@ uncloseai-speech/
|
||||||
|
|
||||||
### Working
|
### Working
|
||||||
- ✅ Piper TTS (tts-1) - Fast, 100+ voices, absolute paths working
|
- ✅ Piper TTS (tts-1) - Fast, 100+ voices, absolute paths working
|
||||||
- ⚠️ XTTS v2 (tts-1-hd) - High quality, needs speaker samples
|
- ✅ XTTS v2 (tts-1-hd) - High quality voice cloning, multilingual
|
||||||
|
- ✅ Silero TTS (tts-1-silero) - Fast CPU-friendly, 148 voices, 5 languages
|
||||||
|
- ✅ Kokoro TTS (tts-1-kokoro) - Lightweight decoder (82M params), 34 voices
|
||||||
|
|
||||||
### High Priority Integration
|
### High Priority Integration
|
||||||
- 🎯 Silero TTS - Active project, fast, good quality
|
|
||||||
- 🎯 StyleTTS2 - Best quality available
|
|
||||||
- 🎯 Fish Speech - Modern, multilingual
|
|
||||||
|
|
||||||
See `docs/MODELS.md` for complete roadmap.
|
1. **StyleTTS2** ⭐⭐⭐⭐⭐
|
||||||
|
- Why: State-of-the-art quality, best prosody and naturalness
|
||||||
|
- License: MIT (permissive)
|
||||||
|
- Challenge: Complex dependencies (phonemizer), slower inference
|
||||||
|
- Priority: HIGH - Best quality available
|
||||||
|
|
||||||
|
2. **Fish Speech** ⭐⭐⭐⭐
|
||||||
|
- Why: Fast, modern, active development, good multilingual support
|
||||||
|
- License: Apache 2.0
|
||||||
|
- Challenge: Newer/less proven
|
||||||
|
- Priority: MEDIUM-HIGH - Good balance of quality and speed
|
||||||
|
|
||||||
|
3. **Chatterbox** ⭐⭐⭐⭐
|
||||||
|
- Why: Emotion control, 23 languages, zero-shot cloning
|
||||||
|
- License: Apache 2.0
|
||||||
|
- Challenge: Production complexity
|
||||||
|
- Priority: MEDIUM-HIGH - Unique emotion features
|
||||||
|
|
||||||
|
See `docs/MODELS.md` for complete roadmap and detailed model documentation.
|
||||||
|
|
||||||
## Deployment Workflow
|
## Deployment Workflow
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -175,9 +175,8 @@ class kokoro_wrapper():
|
||||||
Kokoro is a lightweight decoder-only TTS model (82M params)
|
Kokoro is a lightweight decoder-only TTS model (82M params)
|
||||||
Output: 24kHz audio
|
Output: 24kHz audio
|
||||||
"""
|
"""
|
||||||
def __init__(self, lang_code='a', model_path='/app/voices/kokoro'):
|
def __init__(self, lang_code='a'):
|
||||||
self.lang_code = lang_code
|
self.lang_code = lang_code
|
||||||
self.model_path = model_path
|
|
||||||
self.sample_rate = 24000 # Kokoro outputs 24kHz
|
self.sample_rate = 24000 # Kokoro outputs 24kHz
|
||||||
|
|
||||||
logger.info(f"Loading Kokoro TTS pipeline for language '{lang_code}'")
|
logger.info(f"Loading Kokoro TTS pipeline for language '{lang_code}'")
|
||||||
|
|
@ -186,7 +185,8 @@ class kokoro_wrapper():
|
||||||
from kokoro import KPipeline
|
from kokoro import KPipeline
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
self.pipeline = KPipeline(lang_code=lang_code, model_path=model_path)
|
# KPipeline will use default repo_id if not specified
|
||||||
|
self.pipeline = KPipeline(lang_code=lang_code)
|
||||||
logger.info(f"Successfully loaded Kokoro pipeline for lang={lang_code}")
|
logger.info(f"Successfully loaded Kokoro pipeline for lang={lang_code}")
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"Failed to load Kokoro model: {e}")
|
logger.error(f"Failed to load Kokoro model: {e}")
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue