Integrate Silero TTS and add infrastructure for Chatterbox/Kokoro

INTEGRATED: Silero TTS (tts-1-silero)
- Added silero_wrapper class to speech.py for PyTorch Hub integration
- CPU-friendly, no GPU required (48kHz sample rate)
- Supports 5 languages: English (117 speakers), Russian, German, Spanish, French
- Loads on-demand via torch.hub from snakers4/silero-models
- Added 6 OpenAI-compatible voice mappings (alloy, echo, fable, etc.)

PREPARED: Chatterbox & Kokoro TTS
- Added dependencies to requirements.txt:
  * git+https://github.com/resemble-ai/chatterbox.git
  * transformers>=4.35.0 (for Kokoro)
  * huggingface-hub[cli] (for model downloads)
- Created Makefile targets for downloading models
- Created test targets for all three new engines

Makefile Enhancements:
- make voices-silero: Download Silero models (en, ru, de, es, fr)
- make test-silero: Test Silero TTS endpoint
- make voices-chatterbox: Download Chatterbox models via HF CLI
- make test-chatterbox: Test Chatterbox with emotion control
- make voices-kokoro: Download Kokoro models via HF CLI
- make test-kokoro: Test Kokoro fast synthesis

speech.py Changes:
- Added silero_wrapper class with tts() method
- Added tts-1-silero model handler in generate_speech()
- Registered tts-1-silero model in app
- Added PCM media type for Silero (48000 Hz)
- Global state: silero_model, silero_speakers dict

Configuration:
- Updated voice_to_speaker.default.yaml with tts-1-silero section
- Mapped all 6 OpenAI voices to Silero speakers (en_0 through en_5)

Documentation:
- Updated docs/MODELS.md: Silero marked as ✅ INTEGRATED
- Updated roadmap: Phase 1 task 3 completed
- Updated status footer: 3 models rescued
- Added integration examples and Makefile commands

Next Steps:
- Test Silero integration in Docker
- Implement Chatterbox emotion control engine
- Implement Kokoro fast decoder engine
This commit is contained in:
Claude 2025-11-09 15:31:47 +00:00 • committed by Russell Ballestrini
parent 2d1e1b344f
commit 848c2c6cb5
5 changed files with 327 additions and 288 deletions

View file

@ -14,7 +14,7 @@ REMOTE_USER ?= $(USER)
REMOTE_PATH ?= ~/uncloseai-speech
CONTAINER_NAME ?= uncloseai-speech-server-1
.PHONY: help deploy sync restart logs test clean stop start voices voices-piper voices-xtts push-all
.PHONY: help deploy sync restart logs test clean stop start voices voices-piper voices-xtts voices-kokoro test-kokoro voices-silero test-silero voices-chatterbox test-chatterbox push-all
help:
@echo "🦝 Raccoon TTS Mission - Development Commands"
@ -31,6 +31,12 @@ help:
@echo " make voices - Download all voices (Piper + XTTS)"
@echo " make voices-piper - Download Piper voices only"
@echo " make voices-xtts - Download XTTS voices and samples"
@echo " make voices-kokoro - Download Kokoro models"
@echo " make test-kokoro - Test Kokoro fast TTS"
@echo " make voices-silero - Download Silero models (en, ru, de, es, fr)"
@echo " make test-silero - Test Silero TTS endpoint"
@echo " make voices-chatterbox - Download Chatterbox models"
@echo " make test-chatterbox - Test Chatterbox TTS with emotion control"
@echo ""
@echo "Container:"
@echo " make start - Start Docker container"
@ -105,6 +111,23 @@ voices-xtts:
ssh $(REMOTE_USER)@$(REMOTE_HOST) "docker exec $(CONTAINER_NAME) bash -c 'cd /app && ./scripts/download_samples.sh'"
@echo "✅ XTTS speaker samples downloaded!"
voices-kokoro:
@echo "🎤 Downloading Kokoro TTS models..."
ssh $(REMOTE_USER)@$(REMOTE_HOST) "docker exec $(CONTAINER_NAME) bash -c '\
mkdir -p /app/voices/kokoro && \
cd /app/voices/kokoro && \
huggingface-cli download hexgrad/kokoro-82m --local-dir .'"
@echo "✅ Kokoro models downloaded!"
test-kokoro:
@echo "🧪 Testing Kokoro fast synthesis..."
curl -X POST http://$(REMOTE_HOST):8000/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{"model":"tts-1-kokoro","voice":"alloy","input":"Testing Kokoro fast decoder synthesis"}' \
-o /tmp/kokoro_test.mp3
@echo "✅ Test complete! Playing audio..."
@firefox /tmp/kokoro_test.mp3 || mpv /tmp/kokoro_test.mp3 || echo "Install firefox or mpv to play audio"
test-xtts:
@echo "🧪 Testing XTTS HD endpoint (this may take 1-2 minutes on first run)..."
curl -X POST http://$(REMOTE_HOST):8000/v1/audio/speech \
@ -119,3 +142,39 @@ push-all:
git push origin main
git push github main
@echo "✅ Pushed to origin and github!"
voices-silero:
@echo "🎤 Downloading Silero TTS models..."
ssh $(REMOTE_USER)@$(REMOTE_HOST) "docker exec $(CONTAINER_NAME) bash -c '\
cd /app/voices && \
python3 -c \"import torch; \
for lang in [\"\"en\"\", \"\"ru\"\", \"\"de\"\", \"\"es\"\", \"\"fr\"\"]: \
model, *_ = torch.hub.load(repo_or_dir=\"\"snakers4/silero-models\"\", model=\"\"silero_tts\"\", language=lang, speaker=\"\"v4_\"\"+lang if lang==\"\"en\"\" else \"\"v3_\"\"+lang); \
print(f\"\"Downloaded Silero {lang}\"\")\"'"
@echo "✅ Silero models downloaded!"
test-silero:
@echo "🧪 Testing Silero endpoint..."
curl -X POST http://$(REMOTE_HOST):8000/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{"model":"tts-1-silero","voice":"alloy","input":"Testing Silero fast synthesis"}' \
-o /tmp/silero_test.mp3
@echo "✅ Test complete! Playing audio..."
@firefox /tmp/silero_test.mp3 || mpv /tmp/silero_test.mp3 || echo "Install firefox or mpv to play audio"
voices-chatterbox:
@echo "🎤 Downloading Chatterbox models..."
ssh $(REMOTE_USER)@$(REMOTE_HOST) "docker exec $(CONTAINER_NAME) bash -c '\
mkdir -p /app/voices/chatterbox && \
cd /app/voices/chatterbox && \
huggingface-cli download resemble-ai/chatterbox --local-dir .'"
@echo "✅ Chatterbox models downloaded!"
test-chatterbox:
@echo "🧪 Testing Chatterbox with emotion control..."
curl -X POST http://$(REMOTE_HOST):8000/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{"model":"tts-1-chatter","voice":"alloy","input":"Testing emotional speech synthesis"}' \
-o /tmp/chatterbox_test.mp3
@echo "✅ Test complete! Playing audio..."
@firefox /tmp/chatterbox_test.mp3 || mpv /tmp/chatterbox_test.mp3 || echo "Install firefox or mpv to play audio"