From 9b5caadb8f3a7c27c936fafac4d747734c374eae Mon Sep 17 00:00:00 2001 From: Russell Ballestrini Date: Sun, 9 Nov 2025 14:19:36 -0500 Subject: [PATCH] Fix Kokoro TTS integration - correct KPipeline API MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Removed model_path parameter (not supported by kokoro package) - Removed repo_id parameter (causes KeyError) - Use default KPipeline initialization with only lang_code - Kokoro package handles model download automatically Tested and working: - American English voices (alloy, af_sarah, am_michael, etc.) - British English voices (bm_george, bf_emma, etc.) - Audio generation produces valid MP3 files 🦝 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- README.md | 38 ++++++++++++++++++++++++++++++++++---- docs/CLAUDE.md | 27 ++++++++++++++++++++++----- speech.py | 6 +++--- 3 files changed, 59 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index b16e4ae..da9cf7d 100644 --- a/README.md +++ b/README.md @@ -9,9 +9,11 @@ **Original Notice:** This software was mostly obsolete and no longer updated by the original maintainer. **Raccoon Mission:** We're bringing it back to life with: -- ✅ Working Piper TTS (tts-1) with absolute paths -- ✅ Working XTTS v2 (tts-1-hd) with voice cloning -- 🎯 Planning integration of 10+ abandoned TTS engines (Silero, StyleTTS2, Fish Speech, etc.) +- ✅ Working Piper TTS (tts-1) - 55 voices, fast CPU inference +- ✅ Working XTTS v2 (tts-1-hd) - 8 voices with cloning capability +- ✅ Working Silero TTS (tts-1-silero) - 148 voices, 5 languages, CPU-friendly +- ✅ Working Kokoro TTS (tts-1-kokoro) - 34 voices, lightweight decoder +- 🎯 Next integrations: StyleTTS2 (best quality), Fish Speech (fast multilingual) - 📚 Comprehensive documentation in `docs/` - 🛠️ Makefile-driven deployment workflow - 🔒 AGPL v3 - keeps TTS libre forever @@ -30,10 +32,12 @@ An OpenAI API compatible text to speech server. Full Compatibility: * `tts-1`: `alloy`, `echo`, `fable`, `onyx`, `nova`, and `shimmer` (configurable) * `tts-1-hd`: `alloy`, `echo`, `fable`, `onyx`, `nova`, and `shimmer` (configurable, uses OpenAI samples by default) +* `tts-1-silero`: `alloy`, `echo`, `fable`, `onyx`, `nova`, and `shimmer` (148 total voices available) +* `tts-1-kokoro`: `alloy`, `echo`, `fable`, `onyx`, `nova`, and `shimmer` (34 total voices available) * response_format: `mp3`, `opus`, `aac`, `flac`, `wav` and `pcm` * speed 0.25-4.0 (and more) -Details: +Available TTS Engines: * Model `tts-1` via [piper tts](https://github.com/rhasspy/piper) (very fast, runs on cpu) * You can map your own [piper voices](https://rhasspy.github.io/piper-samples/) via the `voice_to_speaker.yaml` configuration file * Model `tts-1-hd` via [coqui-ai/TTS](https://github.com/coqui-ai/TTS) xtts_v2 voice cloning (fast, but requires around 4GB GPU VRAM) @@ -42,9 +46,35 @@ Details: * [Custom fine-tuned XTTS model support](#custom-fine-tuned-model-support) * Configurable [generation parameters](#generation-parameters) * Streamed output while generating +* Model `tts-1-silero` via [Silero TTS](https://github.com/snakers4/silero-models) (fast CPU inference, actively maintained) + * 148 voices across 5 languages (English, Russian, German, Spanish, French) + * 48kHz sample rate, excellent quality/speed ratio + * No GPU required, real-time capable on CPU +* Model `tts-1-kokoro` via [Kokoro TTS](https://github.com/hexgrad/kokoro) (lightweight decoder-only architecture) + * 34 voices (American and British English) + * 82M parameters, fast inference + * 24kHz sample rate, Apache 2.0 license * Occasionally, certain words or symbols may sound incorrect, you can fix them with regex via `pre_process_map.yaml` * Tested with python 3.9-3.11, piper does not install on python 3.12 yet +## High Priority Integration Targets + +We're actively working on integrating these state-of-the-art TTS engines: + + 1. **StyleTTS2** ⭐⭐⭐⭐⭐ + - Why: State-of-the-art quality, best prosody and naturalness + - License: MIT (permissive) + - Challenge: Complex dependencies (phonemizer), slower inference + - Priority: HIGH - Best quality available + + 2. **Fish Speech** ⭐⭐⭐⭐ + - Why: Fast, modern, active development, good multilingual support + - License: Apache 2.0 + - Challenge: Newer/less proven + - Priority: MEDIUM-HIGH - Good balance of quality and speed + +See [docs/MODELS.md](docs/MODELS.md) for the complete integration roadmap and detailed documentation on all supported and planned TTS engines. + If you find a better voice match for `tts-1` or `tts-1-hd`, please let me know so I can update the defaults. diff --git a/docs/CLAUDE.md b/docs/CLAUDE.md index d3fb58e..093fc4f 100644 --- a/docs/CLAUDE.md +++ b/docs/CLAUDE.md @@ -122,14 +122,31 @@ uncloseai-speech/ ### Working - ✅ Piper TTS (tts-1) - Fast, 100+ voices, absolute paths working -- ⚠️ XTTS v2 (tts-1-hd) - High quality, needs speaker samples +- ✅ XTTS v2 (tts-1-hd) - High quality voice cloning, multilingual +- ✅ Silero TTS (tts-1-silero) - Fast CPU-friendly, 148 voices, 5 languages +- ✅ Kokoro TTS (tts-1-kokoro) - Lightweight decoder (82M params), 34 voices ### High Priority Integration -- 🎯 Silero TTS - Active project, fast, good quality -- 🎯 StyleTTS2 - Best quality available -- 🎯 Fish Speech - Modern, multilingual -See `docs/MODELS.md` for complete roadmap. + 1. **StyleTTS2** ⭐⭐⭐⭐⭐ + - Why: State-of-the-art quality, best prosody and naturalness + - License: MIT (permissive) + - Challenge: Complex dependencies (phonemizer), slower inference + - Priority: HIGH - Best quality available + + 2. **Fish Speech** ⭐⭐⭐⭐ + - Why: Fast, modern, active development, good multilingual support + - License: Apache 2.0 + - Challenge: Newer/less proven + - Priority: MEDIUM-HIGH - Good balance of quality and speed + + 3. **Chatterbox** ⭐⭐⭐⭐ + - Why: Emotion control, 23 languages, zero-shot cloning + - License: Apache 2.0 + - Challenge: Production complexity + - Priority: MEDIUM-HIGH - Unique emotion features + +See `docs/MODELS.md` for complete roadmap and detailed model documentation. ## Deployment Workflow diff --git a/speech.py b/speech.py index 0c1d311..e1dbdfa 100755 --- a/speech.py +++ b/speech.py @@ -175,9 +175,8 @@ class kokoro_wrapper(): Kokoro is a lightweight decoder-only TTS model (82M params) Output: 24kHz audio """ - def __init__(self, lang_code='a', model_path='/app/voices/kokoro'): + def __init__(self, lang_code='a'): self.lang_code = lang_code - self.model_path = model_path self.sample_rate = 24000 # Kokoro outputs 24kHz logger.info(f"Loading Kokoro TTS pipeline for language '{lang_code}'") @@ -186,7 +185,8 @@ class kokoro_wrapper(): from kokoro import KPipeline import numpy as np - self.pipeline = KPipeline(lang_code=lang_code, model_path=model_path) + # KPipeline will use default repo_id if not specified + self.pipeline = KPipeline(lang_code=lang_code) logger.info(f"Successfully loaded Kokoro pipeline for lang={lang_code}") except Exception as e: logger.error(f"Failed to load Kokoro model: {e}")