F5-TTS cloning quality depends on ref_text matching the prosody of ref_audio
(commas, periods, casing). Previous ref_texts were LibriSpeech ground-truth
labels: ALL CAPS, no punctuation — wrong signal for a flow-matching TTS
conditioned on text. Whisper hears what F5 will hear.
- scripts/whisper_refs.py — transcribe all wavs, rewrite
voice_to_speaker.default.yaml + cloned-voices/voices_metadata.json
in place. Also writes cloned-voices/whisper_refs.json sidecar.
- Makefile: whisper-refs target. Idempotent, rerun whenever
cloned-voices/ changes.
Run on a GPU host (4090/3090). ~30s for 40 short clips on a 4090.
Runs scripts/download_diverse_voices.py which pulls speaker samples
from LibriSpeech and assigns permanent human names via voice registry.
Works with venv or system python. Also adds Voices section to help.
- Add CHANGELOG.md with full version history (moved from README)
- Update all documentation to use lowercase 'uncloseai-speech' project name
- Update organization references to lowercase 'uncloseai' (not 'UncloseAI')
- Add Brand Identity section to docs/CLAUDE.md with naming guidelines
- Update speech.py argparse description to match branding
- Update README.md headers and sections with consistent naming
- Update all model documentation with consistent branding
Files updated:
- CHANGELOG.md (new file)
- README.md (changelog reference, server options, multilingual section)
- speech.py (--workers argument, branding in argparse)
- Makefile (header comment)
- docs/CLAUDE.md (Brand Identity section)
- docs/MODELS.md
- docs/MIRRORS.md
- docs/AUDIT.md
- docs/models/coqui-tts.md
- docs/research/tts-models-overview.md
Branding standard:
- Project: uncloseai-speech (lowercase, hyphenated)
- Organization: uncloseai (lowercase, one word)
🦝 Generated with Claude Code
Makefile targets:
- make hydrate: Sequential testing of ALL voices (227 total)
- make load-test: 100 concurrent requests with random voices/models
Load test results (with multiprocess workers):
- 100 requests in 4 seconds (25 req/s)
- 10 concurrent requests at a time
- 100% success rate (no crashes!)
- 15% voices returned full audio (voices downloaded)
- 85% returned stub MP3s (voices not yet downloaded)
Key insight: Server handles concurrent load perfectly with 4 workers
- No deadlocks
- No timeouts
- Graceful handling even when voice files missing
TODO: Run 'make voices' to download all Piper voices for full test
ROOT CAUSE: Python GIL prevents true concurrent execution
- asyncio.to_thread() still bound by GIL and limited thread pool
- Under load: 115+ threads exhausted default pool, server deadlocked
- ML models loading concurrently overwhelmed single-process server
SOLUTION:
1. Added uvicorn workers=4 for true multiprocess concurrency
- Each worker = separate Python process with own GIL
- Models loaded independently per worker
- 4x capacity for concurrent requests
2. Added semaphores for model loading safety
- silero_load_semaphore: Only 1 Silero load at a time per worker
- kokoro_load_semaphore: Only 1 Kokoro load at a time per worker
- Double-check pattern prevents race conditions
3. Increased timeout_keep_alive=300s for long model loads
IMPACT:
- Can now handle 100+ concurrent requests without deadlock
- Each worker independently serves requests during model loads
- Graceful degradation under extreme load
- Ready for production traffic
Alternative considered: Elixir/Phoenix with BEAM VM
- Would give millions of lightweight processes
- Better for massive scale (1000+ concurrent)
- Keep on roadmap for future if needed
Raccoon wisdom: Sometimes the solution is more processes, not more threads!
INTEGRATED: Silero TTS (tts-1-silero)
- Added silero_wrapper class to speech.py for PyTorch Hub integration
- CPU-friendly, no GPU required (48kHz sample rate)
- Supports 5 languages: English (117 speakers), Russian, German, Spanish, French
- Loads on-demand via torch.hub from snakers4/silero-models
- Added 6 OpenAI-compatible voice mappings (alloy, echo, fable, etc.)
PREPARED: Chatterbox & Kokoro TTS
- Added dependencies to requirements.txt:
* git+https://github.com/resemble-ai/chatterbox.git
* transformers>=4.35.0 (for Kokoro)
* huggingface-hub[cli] (for model downloads)
- Created Makefile targets for downloading models
- Created test targets for all three new engines
Makefile Enhancements:
- make voices-silero: Download Silero models (en, ru, de, es, fr)
- make test-silero: Test Silero TTS endpoint
- make voices-chatterbox: Download Chatterbox models via HF CLI
- make test-chatterbox: Test Chatterbox with emotion control
- make voices-kokoro: Download Kokoro models via HF CLI
- make test-kokoro: Test Kokoro fast synthesis
speech.py Changes:
- Added silero_wrapper class with tts() method
- Added tts-1-silero model handler in generate_speech()
- Registered tts-1-silero model in app
- Added PCM media type for Silero (48000 Hz)
- Global state: silero_model, silero_speakers dict
Configuration:
- Updated voice_to_speaker.default.yaml with tts-1-silero section
- Mapped all 6 OpenAI voices to Silero speakers (en_0 through en_5)
Documentation:
- Updated docs/MODELS.md: Silero marked as ✅ INTEGRATED
- Updated roadmap: Phase 1 task 3 completed
- Updated status footer: 3 models rescued
- Added integration examples and Makefile commands
Next Steps:
- Test Silero integration in Docker
- Implement Chatterbox emotion control engine
- Implement Kokoro fast decoder engine
- Add github remote: git@github.com:russellballestrini/openedai-speech.git
- Add 'make push-all' to push to both origin and github
- Document in make help
This ensures the raccoon mission code is mirrored on GitHub
for visibility and resilience.
🦝 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Makefile improvements:
- Add voices-xtts target to download speaker samples
- Add test-xtts target for testing HD model
- Split voices into voices-piper and voices-xtts
- Update help text with all new targets
speech.py:
- Fix threading import scope issue for XTTS
- Remove redundant 'import threading' inside Piper block
docs/CLAUDE.md:
- Complete guide for Claude Code contributors
- Makefile-first development philosophy
- Never create dirs manually, always use Makefile
- Documentation requirements and testing philosophy
- Common mistakes to avoid
- Raccoon mission values and principles
This ensures consistent, repeatable deployments and makes it easy
to add new TTS engines following the same pattern.
🦝 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add step in sync target to copy sample.env to speech.env if missing
- Ensures Makefile works from scratch without manual intervention
- Tested full deployment cycle: deploy -> voices -> test
- Successfully creates ~/uncloseai-speech directory
- Downloads voices with absolute paths
- Generates working TTS audio
Raccoon mission: Makefile is now fully self-sufficient!
🦝 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Renamed all references from openedai-speech to uncloseai-speech across
the entire codebase, including:
- Project name in README and documentation
- Docker image names in compose files
- Makefile deployment paths and container names
- Configuration examples in vars.sh.example
This establishes our raccoon mission fork as UncloseAI Speech, a unified
TTS system supporting multiple engines (Piper, XTTS, etc.) with OpenAI
API compatibility.
🦝 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit resolves the "download entire voices" issue by properly handling
absolute paths in Piper model configuration and improves the deployment system.
Key changes:
- speech.py: Detect absolute paths and omit --data-dir/--download-dir flags
when using absolute model paths, allowing Piper to load models directly
- speech.py: Add debug logging and stderr capture for Piper subprocess
- voice_to_speaker.default.yaml: Use absolute paths for all Piper models
- Makefile: Load deployment config from vars.sh for better security
- Makefile: Change restart to rebuild container ensuring code updates apply
- Add vars.sh.example template for deployment configuration
- .gitignore: Add vars.sh to prevent committing deployment secrets
Tested successfully with en_US-libritts_r-medium model using absolute path:
/app/voices/en/en_US/libritts_r/medium/en_US-libritts_r-medium.onnx
🦝 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
SOLUTION FOUND! Piper requires ABSOLUTE paths to model files, not relative paths.
Changes:
- Update voice_to_speaker.default.yaml to use /app/voices/... absolute paths
- Update Makefile voices target to set absolute paths via sed
- Tested: Piper works with /app/voices/en/en_US/libritts_r/medium/en_US-libritts_r-medium.onnx
Root cause: PyPI piper-tts package doesn't auto-download voices or support
voice name resolution. Must use full absolute file paths.
Co-Authored-By: Claude <noreply@anthropic.com>
- Add comprehensive Makefile for deployment to ai.foxhop.net
- Switch from abandoned rhasspy/piper to OHF-Voice/piper1-gpl v1.3.0
- Add voice download automation with correct directory structure
- Includes deploy, sync, test, logs, and voice management commands
Co-Authored-By: Claude <noreply@anthropic.com>
