engineering/uncloseai-speech
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add comprehensive testing guide for Silero TTS deployment

Browse source
This commit is contained in:
Claude 2025-11-09 18:34:40 +00:00
parent 20241632ea
commit 1a27597d94
No known key found for this signature in database
1 changed files with 273 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

273
TESTING.md Normal file
View file

@ -0,0 +1,273 @@
# Testing Guide: Silero TTS Integration
**Status:** Code validated, syntax verified ✅
**Docker:** Not available in dev environment - deployment testing required
## Pre-Deployment Validation ✅
### Code Validation
```bash
✅ Python syntax validated (speech.py)
✅ Requirements.txt format verified
✅ 14 packages defined in requirements.txt
✅ F-string syntax error fixed
```
### Changes Summary
- **Integrated:** Silero TTS (tts-1-silero model)
- **Prepared:** Chatterbox and Kokoro dependencies
- **Added:** 6 new Makefile targets for model downloads and testing
## Deployment Testing Instructions
Since Docker is not available in the development environment, follow these steps on your deployment server:
### 1. Pull Latest Changes
```bash
cd ~/uncloseai-speech # or your deployment path
git pull origin claude/implement-models-docs-011CUxXuNMytPjEr5vsvcboo
```
### 2. Rebuild Container (Using Makefile)
```bash
# Option A: Full rebuild with restart
make restart
# Option B: Manual rebuild
docker compose up -d --build
```
### 3. Monitor Build Logs
```bash
# Watch the build process
docker compose logs -f
# Or use Makefile
make logs
```
**Expected Output:**
```
✓ Installing fastapi, uvicorn, loguru
✓ Installing piper-tts>=1.2.0
✓ Installing coqui-tts[languages]
✓ Installing transformers>=4.35.0
✓ Installing huggingface-hub[cli]
✓ Installing torch, torchaudio
✓ Cloning chatterbox from GitHub (may take 2-5 mins)
✓ Server starting on 0.0.0.0:8000
```
### 4. Verify Models Available
```bash
curl http://localhost:8000/v1/models
```
**Expected Response:**
```json
{
"data": [
{"id": "tts-1", "object": "model"},
{"id": "tts-1-hd", "object": "model"},
{"id": "tts-1-silero", "object": "model"}
]
}
```
### 5. Test Silero TTS (Fast CPU-friendly synthesis)
**Test 1: Basic Synthesis**
```bash
curl -X POST http://localhost:8000/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{"model":"tts-1-silero","voice":"alloy","input":"Testing Silero fast synthesis"}' \
-o test_silero.mp3
# Play the audio
mpv test_silero.mp3
```
**Test 2: Different Voices**
```bash
# Test all 6 voices (alloy, echo, fable, onyx, nova, shimmer)
for voice in alloy echo fable onyx nova shimmer; do
echo "Testing voice: $voice"
curl -X POST http://localhost:8000/v1/audio/speech \
-H "Content-Type: application/json" \
-d "{\"model\":\"tts-1-silero\",\"voice\":\"$voice\",\"input\":\"This is the $voice voice\"}" \
-o "test_silero_${voice}.mp3"
done
```
**Test 3: Speed Control**
```bash
curl -X POST http://localhost:8000/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{"model":"tts-1-silero","voice":"alloy","input":"Testing speed control","speed":1.5}' \
-o test_silero_fast.mp3
```
**Test 4: Download Silero Models (Optional Pre-caching)**
```bash
# Pre-download models for 5 languages
make voices-silero
# Or manually inside container
docker exec uncloseai-speech-server-1 python3 -c "
import torch
for lang in ['en', 'ru', 'de', 'es', 'fr']:
model, *_ = torch.hub.load('snakers4/silero-models', model='silero_tts', language=lang)
print(f'Downloaded Silero {lang}')
"
```
### 6. Compare Model Performance
**Test all three engines:**
```bash
# Piper (tts-1) - Very fast, CPU
time curl -X POST http://localhost:8000/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{"model":"tts-1","voice":"alloy","input":"Performance test"}' \
-o test_piper.mp3
# Silero (tts-1-silero) - Fast, CPU
time curl -X POST http://localhost:8000/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{"model":"tts-1-silero","voice":"alloy","input":"Performance test"}' \
-o test_silero.mp3
# XTTS (tts-1-hd) - Slower, GPU recommended
time curl -X POST http://localhost:8000/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{"model":"tts-1-hd","voice":"alloy","input":"Performance test"}' \
-o test_xtts.mp3
```
**Expected Performance:**
- Piper: 0.5-1 second (RTF ~0.05x)
- Silero: 1-2 seconds (RTF ~0.1x)
- XTTS: 3-5 seconds (RTF ~0.3x)
### 7. Test Error Handling
**Test invalid model:**
```bash
curl -X POST http://localhost:8000/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{"model":"invalid","voice":"alloy","input":"Test"}' \
-v
```
**Expected:** HTTP 400 with error message
**Test invalid voice:**
```bash
curl -X POST http://localhost:8000/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{"model":"tts-1-silero","voice":"invalid","input":"Test"}' \
-v
```
**Expected:** HTTP 400 or 503 with voice error
## Troubleshooting
### Build Fails on Chatterbox
If cloning chatterbox fails (GitHub rate limit or network):
```bash
# Comment out Chatterbox temporarily
sed -i 's/^git+https:\/\/github.com\/resemble-ai\/chatterbox.git/# &/' requirements.txt
docker compose up -d --build
```
Chatterbox is not yet integrated into speech.py, so it's safe to skip for now.
### Silero Model Download Slow
First synthesis will download Silero models (~50-100MB). Subsequent calls will be fast.
```bash
# Pre-cache during deployment
make voices-silero
```
### Out of Memory
Silero runs on CPU and uses minimal memory (~500MB). If issues occur:
```bash
# Check container memory
docker stats uncloseai-speech-server-1
# Restart if needed
make restart
```
### Check Logs
```bash
# Full logs
docker compose logs
# Follow logs in real-time
make logs
# Filter for errors
docker compose logs | grep -i error
```
## Success Criteria
✅ Container builds without errors
✅ All 3 models listed in /v1/models
✅ Silero synthesis works (tts-1-silero)
✅ Response time < 2 seconds for Silero
✅ Audio quality is clear and natural
✅ All 6 voices work (alloy, echo, fable, onyx, nova, shimmer)
✅ No memory leaks after 10+ requests
## Next Steps After Successful Deployment
1. **Integrate Chatterbox** (emotion control)
- Implement `chatterbox_wrapper` in speech.py
- Add model handler for `tts-1-chatter`
- Test emotion parameters
2. **Integrate Kokoro** (fast decoder)
- Implement `kokoro_wrapper` in speech.py
- Add model handler for `tts-1-kokoro`
- Test performance vs Silero
3. **Create Detailed Silero Documentation**
- Write `docs/models/silero-tts.md`
- Document all 117 English speakers
- Add multilingual examples
4. **Performance Benchmarking**
- Test all models under load
- Measure memory usage over time
- Compare audio quality subjectively
## Files Modified in This Integration
```
requirements.txt - Added Silero comments, Chatterbox, Kokoro, huggingface-hub
speech.py - Added silero_wrapper, tts-1-silero handler, model registration
voice_to_speaker.default.yaml - Added tts-1-silero voice mappings
Makefile - Added 6 new targets (voices-silero, test-silero, etc.)
docs/MODELS.md - Updated with Silero integration status
```
## Rollback Instructions
If deployment fails:
```bash
# Stop current container
docker compose down
# Checkout previous working commit
git checkout 0073e87^ # Parent of Silero integration
# Rebuild
docker compose up -d --build
```
---
**Last Updated:** 2025-11-09
**Branch:** claude/implement-models-docs-011CUxXuNMytPjEr5vsvcboo
**Status:** Ready for deployment testing