Fix Kokoro TTS integration - correct KPipeline API

- Removed model_path parameter (not supported by kokoro package)
- Removed repo_id parameter (causes KeyError)
- Use default KPipeline initialization with only lang_code
- Kokoro package handles model download automatically

Tested and working:
- American English voices (alloy, af_sarah, am_michael, etc.)
- British English voices (bm_george, bf_emma, etc.)
- Audio generation produces valid MP3 files

🦝 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Russell Ballestrini 2025-11-09 14:19:36 -05:00
parent f8d46e92d5
commit 9b5caadb8f
3 changed files with 59 additions and 12 deletions

View file

@ -9,9 +9,11 @@
**Original Notice:** This software was mostly obsolete and no longer updated by the original maintainer. **Original Notice:** This software was mostly obsolete and no longer updated by the original maintainer.
**Raccoon Mission:** We're bringing it back to life with: **Raccoon Mission:** We're bringing it back to life with:
- ✅ Working Piper TTS (tts-1) with absolute paths - ✅ Working Piper TTS (tts-1) - 55 voices, fast CPU inference
- ✅ Working XTTS v2 (tts-1-hd) with voice cloning - ✅ Working XTTS v2 (tts-1-hd) - 8 voices with cloning capability
- 🎯 Planning integration of 10+ abandoned TTS engines (Silero, StyleTTS2, Fish Speech, etc.) - ✅ Working Silero TTS (tts-1-silero) - 148 voices, 5 languages, CPU-friendly
- ✅ Working Kokoro TTS (tts-1-kokoro) - 34 voices, lightweight decoder
- 🎯 Next integrations: StyleTTS2 (best quality), Fish Speech (fast multilingual)
- 📚 Comprehensive documentation in `docs/` - 📚 Comprehensive documentation in `docs/`
- 🛠️ Makefile-driven deployment workflow - 🛠️ Makefile-driven deployment workflow
- 🔒 AGPL v3 - keeps TTS libre forever - 🔒 AGPL v3 - keeps TTS libre forever
@ -30,10 +32,12 @@ An OpenAI API compatible text to speech server.
Full Compatibility: Full Compatibility:
* `tts-1`: `alloy`, `echo`, `fable`, `onyx`, `nova`, and `shimmer` (configurable) * `tts-1`: `alloy`, `echo`, `fable`, `onyx`, `nova`, and `shimmer` (configurable)
* `tts-1-hd`: `alloy`, `echo`, `fable`, `onyx`, `nova`, and `shimmer` (configurable, uses OpenAI samples by default) * `tts-1-hd`: `alloy`, `echo`, `fable`, `onyx`, `nova`, and `shimmer` (configurable, uses OpenAI samples by default)
* `tts-1-silero`: `alloy`, `echo`, `fable`, `onyx`, `nova`, and `shimmer` (148 total voices available)
* `tts-1-kokoro`: `alloy`, `echo`, `fable`, `onyx`, `nova`, and `shimmer` (34 total voices available)
* response_format: `mp3`, `opus`, `aac`, `flac`, `wav` and `pcm` * response_format: `mp3`, `opus`, `aac`, `flac`, `wav` and `pcm`
* speed 0.25-4.0 (and more) * speed 0.25-4.0 (and more)
Details: Available TTS Engines:
* Model `tts-1` via [piper tts](https://github.com/rhasspy/piper) (very fast, runs on cpu) * Model `tts-1` via [piper tts](https://github.com/rhasspy/piper) (very fast, runs on cpu)
* You can map your own [piper voices](https://rhasspy.github.io/piper-samples/) via the `voice_to_speaker.yaml` configuration file * You can map your own [piper voices](https://rhasspy.github.io/piper-samples/) via the `voice_to_speaker.yaml` configuration file
* Model `tts-1-hd` via [coqui-ai/TTS](https://github.com/coqui-ai/TTS) xtts_v2 voice cloning (fast, but requires around 4GB GPU VRAM) * Model `tts-1-hd` via [coqui-ai/TTS](https://github.com/coqui-ai/TTS) xtts_v2 voice cloning (fast, but requires around 4GB GPU VRAM)
@ -42,9 +46,35 @@ Details:
* [Custom fine-tuned XTTS model support](#custom-fine-tuned-model-support) * [Custom fine-tuned XTTS model support](#custom-fine-tuned-model-support)
* Configurable [generation parameters](#generation-parameters) * Configurable [generation parameters](#generation-parameters)
* Streamed output while generating * Streamed output while generating
* Model `tts-1-silero` via [Silero TTS](https://github.com/snakers4/silero-models) (fast CPU inference, actively maintained)
* 148 voices across 5 languages (English, Russian, German, Spanish, French)
* 48kHz sample rate, excellent quality/speed ratio
* No GPU required, real-time capable on CPU
* Model `tts-1-kokoro` via [Kokoro TTS](https://github.com/hexgrad/kokoro) (lightweight decoder-only architecture)
* 34 voices (American and British English)
* 82M parameters, fast inference
* 24kHz sample rate, Apache 2.0 license
* Occasionally, certain words or symbols may sound incorrect, you can fix them with regex via `pre_process_map.yaml` * Occasionally, certain words or symbols may sound incorrect, you can fix them with regex via `pre_process_map.yaml`
* Tested with python 3.9-3.11, piper does not install on python 3.12 yet * Tested with python 3.9-3.11, piper does not install on python 3.12 yet
## High Priority Integration Targets
We're actively working on integrating these state-of-the-art TTS engines:
1. **StyleTTS2** ⭐⭐⭐⭐⭐
- Why: State-of-the-art quality, best prosody and naturalness
- License: MIT (permissive)
- Challenge: Complex dependencies (phonemizer), slower inference
- Priority: HIGH - Best quality available
2. **Fish Speech** ⭐⭐⭐⭐
- Why: Fast, modern, active development, good multilingual support
- License: Apache 2.0
- Challenge: Newer/less proven
- Priority: MEDIUM-HIGH - Good balance of quality and speed
See [docs/MODELS.md](docs/MODELS.md) for the complete integration roadmap and detailed documentation on all supported and planned TTS engines.
If you find a better voice match for `tts-1` or `tts-1-hd`, please let me know so I can update the defaults. If you find a better voice match for `tts-1` or `tts-1-hd`, please let me know so I can update the defaults.

View file

@ -122,14 +122,31 @@ uncloseai-speech/
### Working ### Working
- ✅ Piper TTS (tts-1) - Fast, 100+ voices, absolute paths working - ✅ Piper TTS (tts-1) - Fast, 100+ voices, absolute paths working
- ⚠️ XTTS v2 (tts-1-hd) - High quality, needs speaker samples - ✅ XTTS v2 (tts-1-hd) - High quality voice cloning, multilingual
- ✅ Silero TTS (tts-1-silero) - Fast CPU-friendly, 148 voices, 5 languages
- ✅ Kokoro TTS (tts-1-kokoro) - Lightweight decoder (82M params), 34 voices
### High Priority Integration ### High Priority Integration
- 🎯 Silero TTS - Active project, fast, good quality
- 🎯 StyleTTS2 - Best quality available
- 🎯 Fish Speech - Modern, multilingual
See `docs/MODELS.md` for complete roadmap. 1. **StyleTTS2** ⭐⭐⭐⭐⭐
- Why: State-of-the-art quality, best prosody and naturalness
- License: MIT (permissive)
- Challenge: Complex dependencies (phonemizer), slower inference
- Priority: HIGH - Best quality available
2. **Fish Speech** ⭐⭐⭐⭐
- Why: Fast, modern, active development, good multilingual support
- License: Apache 2.0
- Challenge: Newer/less proven
- Priority: MEDIUM-HIGH - Good balance of quality and speed
3. **Chatterbox** ⭐⭐⭐⭐
- Why: Emotion control, 23 languages, zero-shot cloning
- License: Apache 2.0
- Challenge: Production complexity
- Priority: MEDIUM-HIGH - Unique emotion features
See `docs/MODELS.md` for complete roadmap and detailed model documentation.
## Deployment Workflow ## Deployment Workflow

View file

@ -175,9 +175,8 @@ class kokoro_wrapper():
Kokoro is a lightweight decoder-only TTS model (82M params) Kokoro is a lightweight decoder-only TTS model (82M params)
Output: 24kHz audio Output: 24kHz audio
""" """
def __init__(self, lang_code='a', model_path='/app/voices/kokoro'): def __init__(self, lang_code='a'):
self.lang_code = lang_code self.lang_code = lang_code
self.model_path = model_path
self.sample_rate = 24000 # Kokoro outputs 24kHz self.sample_rate = 24000 # Kokoro outputs 24kHz
logger.info(f"Loading Kokoro TTS pipeline for language '{lang_code}'") logger.info(f"Loading Kokoro TTS pipeline for language '{lang_code}'")
@ -186,7 +185,8 @@ class kokoro_wrapper():
from kokoro import KPipeline from kokoro import KPipeline
import numpy as np import numpy as np
self.pipeline = KPipeline(lang_code=lang_code, model_path=model_path) # KPipeline will use default repo_id if not specified
self.pipeline = KPipeline(lang_code=lang_code)
logger.info(f"Successfully loaded Kokoro pipeline for lang={lang_code}") logger.info(f"Successfully loaded Kokoro pipeline for lang={lang_code}")
except Exception as e: except Exception as e:
logger.error(f"Failed to load Kokoro model: {e}") logger.error(f"Failed to load Kokoro model: {e}")