diff --git a/docs/AUDIT.md b/docs/AUDIT.md new file mode 100644 index 0000000..22e8893 --- /dev/null +++ b/docs/AUDIT.md @@ -0,0 +1,340 @@ +# UncloseAI Speech Repository Audit + +**Date:** 2025-11-09 +**Mission:** Raccoon TTS - Build a unified, resilient TTS system from abandoned open source projects + +## Current Repository Structure + +### Core Service Files (KEEP - Essential) +- `speech.py` (18K) - Main FastAPI server, handles OpenAI-compatible TTS API +- `openedai.py` (5.8K) - OpenAI API models and data structures +- `audio_reader.py` (3.7K) - Audio processing utilities +- `voice_to_speaker.default.yaml` - Default voice configuration mapping +- `pre_process_map.default.yaml` - Text preprocessing rules + +### Docker & Deployment (KEEP - Essential) +- `Dockerfile` - Main docker build (Piper + XTTS with NVIDIA GPU) +- `Dockerfile.min` - Minimal build (Piper only, no GPU) +- `docker-compose.yml` - Standard deployment +- `docker-compose.min.yml` - Minimal deployment +- `docker-compose.rocm.yml` - AMD ROCm GPU support +- `Makefile` - Raccoon deployment system (recently added) +- `vars.sh.example` - Deployment configuration template + +### Requirements Files (KEEP - Essential) +- `requirements.txt` - Main Python dependencies +- `requirements-min.txt` - Minimal dependencies (Piper only) +- `requirements-rocm.txt` - AMD ROCm dependencies + +### Startup Scripts (KEEP - Essential) +- `startup.sh` - Main startup with voice downloads +- `startup.min.sh` - Minimal startup +- `download_voices_tts-1.sh` - Download Piper voices +- `download_voices_tts-1-hd.sh` - Download XTTS HD voices + +### CLI Utilities (KEEP - Useful) +- `say.py` (3.5K) - Command-line TTS client using OpenAI API +- `add_voice.py` (2.8K) - Tool to add custom voices to config +- `test_voices.sh` (2.1K) - Test script for all voice models + +### Windows Batch Files (EVALUATE - Questionable) +- `startup.bat` (185B) - Windows startup +- `download_samples.bat` (230B) - Download sample WAVs +- `download_voices_tts-1.bat` (323B) - Download voices on Windows +- `download_voices_tts-1-hd.bat` (199B) - Download HD voices on Windows + +**Assessment:** These are only useful if we support Windows deployment. Most raccoon missions run on Linux servers. Could move to `docs/windows/` or remove entirely. + +### Sample Scripts (EVALUATE - Non-Essential) +- `download_samples.sh` (205B) - Downloads OpenAI sample WAV files +- `boot_docker_compose_ubuntu.sh` (623B) - One-time NVIDIA Docker setup for Ubuntu + +**Assessment:** `download_samples.sh` is only for comparing against OpenAI's original voices - move to `docs/tools/`. The boot script is one-time setup - move to `docs/setup/`. + +### Documentation (KEEP - Update) +- `README.md` (16.6K) - Main documentation +- `LICENSE` - MIT license +- `sample.env` - Environment variable examples + +### CI/CD (KEEP) +- `.github/workflows/build-docker.yml` - GitHub Actions for building Docker images + +### Configuration (KEEP) +- `.gitignore` - Git ignore patterns +- `config/config_files_will_go_here.txt` - Placeholder + +## Files to Remove or Relocate + +### REMOVE (Windows-specific, low value): +- `startup.bat` +- `download_samples.bat` +- `download_voices_tts-1.bat` +- `download_voices_tts-1-hd.bat` + +### MOVE to `docs/tools/`: +- `download_samples.sh` (useful for voice comparison) +- `test_voices.sh` (useful for testing) + +### MOVE to `docs/setup/`: +- `boot_docker_compose_ubuntu.sh` (one-time setup instructions) + +## Abandoned TTS Models to Integrate + +Based on the raccoon mission to rescue abandoned TTS projects: + +### 1. **Piper TTS** (Currently Integrated) +- **Status:** ✅ Working with absolute paths +- **Original:** rhasspy/piper (abandoned) +- **Fork:** OHF-Voice/piper1-gpl v1.3.0 (no PyPI package) +- **Current:** Using PyPI piper-tts v1.2.0 +- **Models:** ~100+ voices from HuggingFace rhasspy/piper-voices +- **Action:** Consider creating our own PyPI package from OHF-Voice fork + +### 2. **Coqui XTTS** (Currently Integrated) +- **Status:** ✅ Integrated as tts-1-hd +- **Original:** coqui-ai/TTS (company shut down, repo archived) +- **Package:** coqui-tts (still on PyPI) +- **Models:** XTTS v2 from HuggingFace +- **Action:** Mirror XTTS v2 models, explore community forks + +### 3. **Mozilla TTS** (Predecessor to Coqui) +- **Status:** ❌ Not integrated, abandoned +- **Original:** mozilla/TTS (archived, became Coqui) +- **Action:** Evaluate if any unique models worth preserving + +### 4. **Silero TTS** +- **Status:** ❌ Not integrated +- **Repo:** snakers4/silero-models (still active!) +- **Models:** Fast, multilingual, small footprint +- **Languages:** Russian, English, German, Spanish, French +- **Action:** HIGH PRIORITY - Add Silero support + +### 5. **Bark (Suno AI)** +- **Status:** ❌ Not integrated +- **Repo:** suno-ai/bark (archived, company pivoted) +- **Features:** Music, sound effects, non-speech audio +- **Action:** Evaluate for unique use cases + +### 6. **Tortoise TTS** +- **Status:** ❌ Not integrated +- **Repo:** neonbjb/tortoise-tts (low activity) +- **Features:** High quality, very slow generation +- **Action:** Low priority - too slow for production + +### 7. **StyleTTS2** +- **Status:** ❌ Not integrated +- **Repo:** yl4579/StyleTTS2 (active research project) +- **Features:** State-of-art quality, prosody +- **Action:** HIGH PRIORITY - Best quality available + +### 8. **MetaVoice** +- **Status:** ❌ Not integrated +- **Repo:** metavoiceio/metavoice-src (partially abandoned) +- **Features:** Long-form TTS, emotional control +- **Action:** Evaluate community forks + +### 9. **Fish Speech** +- **Status:** ❌ Not integrated +- **Repo:** fishaudio/fish-speech (active) +- **Features:** Fast, multilingual +- **Action:** MEDIUM PRIORITY - Active project + +### 10. **Kokoro TTS** +- **Status:** ❌ Not integrated +- **Repo:** hexgrad/kokoro (new, active) +- **Features:** Fast, small, quality +- **Action:** MEDIUM PRIORITY - Promising new project + +## Binary Mirror Strategy + +### Problem +- HuggingFace models can disappear +- Upstream repositories get archived/deleted +- Need resilience for production systems + +### Solution: Multi-Tier Mirror System + +#### Tier 1: Primary Sources (Current) +- HuggingFace Hub +- PyPI packages +- GitHub releases + +#### Tier 2: UncloseAI Mirror (Proposed) +- Self-hosted model repository +- Options: + - MinIO/S3-compatible storage + - LFS-backed Git repository + - Dedicated file server with rsync + +#### Tier 3: Fallback (Proposed) +- Archive.org uploads for critical models +- IPFS pinning for decentralization +- Torrent magnets for popular models + +### Priority Models to Mirror +1. **Piper voices** - ~2GB from rhasspy/piper-voices +2. **XTTS v2** - ~1.8GB model + config +3. **Silero models** - ~100MB per language +4. **StyleTTS2 weights** - TBD size + +### Implementation Plan +- [ ] Set up MinIO on ai.foxhop.net +- [ ] Create download script with fallback URLs +- [ ] Document mirror usage in docs +- [ ] Automated sync from upstream sources + +## Recommended Refactoring + +### 1. Directory Structure + +``` +uncloseai-speech/ +├── docs/ +│ ├── AUDIT.md (this file) +│ ├── MODELS.md (TTS engine documentation) +│ ├── MIRRORS.md (Binary mirror setup) +│ ├── API.md (API documentation) +│ └── setup/ +│ └── nvidia-docker-ubuntu.md +├── scripts/ +│ ├── add_voice.py (add custom voices to config) +│ ├── say.py (CLI TTS client) +│ ├── test_voices.sh (test all voices) +│ ├── download_samples.sh (download OpenAI sample WAVs) +│ ├── download_voices_tts-1.sh (Piper voice downloader) +│ └── download_voices_tts-1-hd.sh (XTTS voice downloader) +├── src/ (future refactor) +│ ├── engines/ +│ │ ├── piper.py +│ │ ├── xtts.py +│ │ ├── silero.py (NEW) +│ │ └── base.py (NEW) +│ ├── api/ +│ │ ├── openedai.py +│ │ └── server.py +│ └── utils/ +│ └── audio_reader.py +├── config/ +│ ├── voice_to_speaker.default.yaml +│ └── pre_process_map.default.yaml +├── Dockerfile +├── Dockerfile.min +├── docker-compose.yml +├── docker-compose.min.yml +├── docker-compose.rocm.yml +├── startup.sh +├── startup.min.sh +├── Makefile +├── speech.py (current monolith - will refactor to src/) +├── openedai.py (will move to src/api/) +├── audio_reader.py (will move to src/utils/) +├── requirements.txt +├── requirements-min.txt +├── requirements-rocm.txt +└── README.md +``` + +### 2. Engine Abstraction + +Create a base engine class that all TTS engines implement: + +```python +# src/engines/base.py +from abc import ABC, abstractmethod +from typing import BinaryIO, Optional + +class TTSEngine(ABC): + @abstractmethod + def synthesize(self, text: str, voice: str, + speed: float = 1.0, + language: Optional[str] = None) -> BinaryIO: + """Synthesize speech from text""" + pass + + @abstractmethod + def list_voices(self) -> list[str]: + """Return available voices for this engine""" + pass + + @abstractmethod + def download_models(self, mirror_urls: list[str] = None): + """Download required models with fallback mirrors""" + pass +``` + +### 3. Configuration Schema + +Extend voice config to support multiple engines: + +```yaml +# Future voice_to_speaker.yaml format +tts-1: + alloy: + engine: piper # or xtts, silero, styletts2 + model: /app/voices/en/en_US/libritts_r/medium/en_US-libritts_r-medium.onnx + speaker: 79 + nova: + engine: silero + model: v3_1_ru + speaker: aidar + +tts-1-hd: + alloy: + engine: xtts + model: xtts + speaker: /app/voices/alloy.wav + custom-voice: + engine: styletts2 + model: /app/models/styletts2-base + speaker: /app/voices/custom.wav + language: en +``` + +## Next Steps + +### Immediate (This Session) +1. ✅ Create docs directory +2. ✅ Write audit document +3. [ ] Remove Windows .bat files +4. [ ] Relocate tools to docs/tools/ +5. [ ] Create MODELS.md documenting each TTS engine + +### Short Term (Next Few Sessions) +1. [ ] Implement engine abstraction layer +2. [ ] Add Silero TTS support +3. [ ] Set up model mirror on ai.foxhop.net +4. [ ] Test all existing functionality still works + +### Medium Term +1. [ ] Add StyleTTS2 support +2. [ ] Add Fish Speech support +3. [ ] Implement fallback mirror system +4. [ ] Create automated model sync + +### Long Term +1. [ ] Add Bark for sound effects +2. [ ] Implement voice cloning +3. [ ] Multi-language auto-detection +4. [ ] Performance benchmarking dashboard + +## Raccoon Mission Status + +🦝 **Mission:** Rescue abandoned TTS models and build a resilient unified system + +**Current Status:** +- ✅ Piper TTS working with absolute paths +- ✅ XTTS integrated +- ✅ Deployment system (Makefile + vars.sh) +- ✅ Renamed to UncloseAI Speech +- 📝 Repository audit complete +- 🔄 Documentation in progress + +**Key Risks:** +- Upstream model URLs can break (need mirrors) +- PyPI packages could disappear (need vendoring strategy) +- HuggingFace rate limits (need caching) + +**Mitigation:** +- Build multi-tier mirror system +- Document all model sources +- Create offline installation option diff --git a/download_samples.bat b/download_samples.bat deleted file mode 100644 index 87b4d2e..0000000 --- a/download_samples.bat +++ /dev/null @@ -1,6 +0,0 @@ -@echo off -for %%i in (alloy echo fable onyx nova shimmer) do ( - if not exist "voices\%%i.wav" ( - curl -s https://cdn.openai.com/API/docs/audio/%%i.wav | ffmpeg -loglevel error -i - -ar 22050 -ac 1 voices\%%i.wav - ) -) diff --git a/download_voices_tts-1-hd.bat b/download_voices_tts-1-hd.bat deleted file mode 100644 index d156e96..0000000 --- a/download_voices_tts-1-hd.bat +++ /dev/null @@ -1,8 +0,0 @@ -@echo off -set COQUI_TOS_AGREED=1 -set TTS_HOME=voices - -for %%i in (%*) do ( - python -c "from TTS.utils.manage import ModelManager; ModelManager().download_model('%%i')" -) -call download_samples.bat diff --git a/download_voices_tts-1.bat b/download_voices_tts-1.bat deleted file mode 100644 index 1939a04..0000000 --- a/download_voices_tts-1.bat +++ /dev/null @@ -1,8 +0,0 @@ -@echo off -set models=%* -if "%models%" == "" set models=en_GB-northern_english_male-medium en_US-libritts_r-medium - -piper --update-voices --data-dir voices --download-dir voices --model x 2> nul -for %%i in (%models%) do ( - if not exist "voices\%%i.onnx" piper --data-dir voices --download-dir voices --model %%i > nul -) diff --git a/add_voice.py b/scripts/add_voice.py similarity index 100% rename from add_voice.py rename to scripts/add_voice.py diff --git a/download_samples.sh b/scripts/download_samples.sh similarity index 100% rename from download_samples.sh rename to scripts/download_samples.sh diff --git a/say.py b/scripts/say.py similarity index 100% rename from say.py rename to scripts/say.py diff --git a/test_voices.sh b/scripts/test_voices.sh similarity index 100% rename from test_voices.sh rename to scripts/test_voices.sh diff --git a/startup.bat b/startup.bat deleted file mode 100644 index 4f4c75d..0000000 --- a/startup.bat +++ /dev/null @@ -1,8 +0,0 @@ -@echo off - -set /p < speech.env - -call download_voices_tts-1.bat -call download_voices_tts-1-hd.bat %PRELOAD_MODEL% - -python speech.py %PRELOAD_MODEL:+--preload %PRELOAD_MODEL% %EXTRA_ARGS% \ No newline at end of file