📚 Update docs with hydration results and multiprocess fixes

**README.md:**
- Added Raccoon Mission Updates section (2025-11-09)
- Documented 235/245 voices working (95.9% success rate)
- Listed all major fixes: multiprocess architecture, voice auto-detection,
  cache initialization, args initialization
- Added Makefile targets documentation (hydrate, load-test)

**docs/CLAUDE.md:**
- Updated TTS Engine Status with hydration percentages
- Added Testing Philosophy section with hydrate and load-test targets
- Documented Known Issues and Solutions:
  * Worker processes not loading caches - lifespan solution
  * Worker processes AttributeError on args - DefaultArgs solution
- Included code examples for both fixes

These updates capture the complete journey from 0% to 95.9% voice
hydration success and document the multiprocess worker architecture fixes.

🦝 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Russell Ballestrini 2025-11-09 18:27:48 -05:00
parent be374409d6
commit 187121558e
2 changed files with 85 additions and 5 deletions

View file

@ -79,6 +79,21 @@ If you find a better voice match for `tts-1` or `tts-1-hd`, please let me know s
## Recent Changes
**Raccoon Mission Updates, 2025-11-09**
* 🦝 **Production-ready multiprocess architecture** - 4 uvicorn workers for true concurrency, bypassing Python's GIL
* 🦝 **Voice auto-detection** - `model` parameter now optional, automatically selects correct engine from voice name
* 🦝 **Voice cache initialization fix** - All worker processes now properly initialize voice-to-model lookup cache
* 🦝 **Args initialization fix** - Worker processes now have access to server configuration via DefaultArgs class
* 🦝 **235/245 voices working** (95.9% hydration success rate):
- Piper: 55/55 voices (100%)
- XTTS: 6/8 voices (75%)
- Silero: 142/148 voices (95.9%)
- Kokoro: 32/34 voices (94.1%)
* 🦝 **Extended `/v1/voices` endpoint** - Returns all available voices with engine metadata
* 🦝 **Makefile targets** - `make hydrate` (test all voices), `make load-test` (concurrent stress test)
* 🦝 **Comprehensive docs** - See `docs/CLAUDE.md`, `docs/MODELS.md`, `docs/MIRRORS.md`, `docs/AUDIT.md`
Version 0.18.2, 2024-08-16
* Fix docker building for amd64, refactor github actions again, free up more disk space

View file

@ -120,11 +120,11 @@ uncloseai-speech/
## TTS Engine Status
### Working
- ✅ Piper TTS (tts-1) - Fast, 100+ voices, absolute paths working
- ✅ XTTS v2 (tts-1-hd) - High quality voice cloning, multilingual
- ✅ Silero TTS (tts-1-silero) - Fast CPU-friendly, 148 voices, 5 languages
- ✅ Kokoro TTS (tts-1-kokoro) - Lightweight decoder (82M params), 34 voices
### Working (95.9% Hydration Success - 235/245 voices)
- ✅ Piper TTS (tts-1) - 55/55 voices (100%), fast CPU inference, absolute paths working
- ✅ XTTS v2 (tts-1-hd) - 6/8 voices (75%), high quality voice cloning, multilingual
- ✅ Silero TTS (tts-1-silero) - 142/148 voices (95.9%), fast CPU-friendly, 5 languages, auto-downloads from torch.hub
- ✅ Kokoro TTS (tts-1-kokoro) - 32/34 voices (94.1%), lightweight decoder (82M params)
### High Priority Integration
@ -177,6 +177,20 @@ Container:
2. Fresh deploy (`make deploy`)
3. Voice download (`make voices`)
4. API test (`make test`, `make test-xtts`)
5. Hydration test (`make hydrate`) - Tests ALL 245 voices sequentially
6. Load test (`make load-test`) - 100 concurrent random requests
**Hydration Testing:**
- `make hydrate` - Tests every voice across all 4 engines
- Sequential (one at a time) to avoid overwhelming server
- Reports success/failure with file sizes
- Output saved to `/tmp/hydrate_test/`
**Load Testing:**
- `make load-test` - 100 concurrent requests with random voices
- 10 parallel workers via `xargs -P10`
- Tests multiprocess worker architecture
- Reports timing, throughput, and success rate
**Never assume** - if you changed something, test from scratch.
@ -213,6 +227,57 @@ Container:
4. Check if container was rebuilt (`make deploy` does this)
5. Verify voices downloaded (`ls` in container via `make logs` approach)
## Known Issues and Solutions (2025-11-09)
### Issue: Worker Processes Not Loading Caches
**Symptom:** Voice auto-detection fails with "Voice 'X' not found in any model" despite voices being configured in `voice_to_speaker.yaml`.
**Root Cause:** When using `uvicorn.run("speech:app", workers=4)`, worker processes spawn as fresh imports and don't execute the `if __name__ == "__main__"` block where caches were initialized.
**Solution:** Move cache initialization to FastAPI's `lifespan` context manager, which runs during startup in EACH worker process.
```python
@contextlib.asynccontextmanager
async def lifespan(app):
# Startup: Initialize voice caches in each worker process
global voice_to_model_cache, voices_cache
# Build caches from YAML...
# (see speech.py:23-81 for full implementation)
yield
# Shutdown: Cleanup...
```
### Issue: Worker Processes Get AttributeError on args
**Symptom:** `AttributeError: 'NoneType' object has no attribute 'xtts_device'` when workers try to access command-line arguments.
**Root Cause:** `args` was parsed in `if __name__ == "__main__"` which only runs in parent process, not in worker processes.
**Solution:** Create DefaultArgs class with sensible defaults that gets used by workers, while parent process overrides with actual command-line args.
```python
# Default args for worker processes (will be overridden in __main__)
class DefaultArgs:
xtts_device = 'cpu'
use_deepspeed = False
unload_timer = None
# ... etc
args = DefaultArgs()
```
Then in `__main__`:
```python
if __name__ == "__main__":
parser = argparse.ArgumentParser(...)
args = parser.parse_args() # Overrides DefaultArgs
# ... rest of startup
```
## Future Refactoring (Planned)
- Move `speech.py`, `openedai.py`, `audio_reader.py``src/`