From 4a019cf89758fd7fd9eccc41a3dc4df06e4599fc Mon Sep 17 00:00:00 2001 From: "russell@unturf.com" Date: Mon, 26 Jan 2026 10:39:23 -0500 Subject: [PATCH] Add detailed AGPL v3 license obligations documentation Explains source code requirements for network service operators, practical compliance methods, and Raccoon Mission rationale. Co-Authored-By: Claude Opus 4.5 --- CLAUDE.md | 72 +++++-- README.md | 612 ++++++++++++++++++++++++++---------------------------- 2 files changed, 338 insertions(+), 346 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 0b240e2..742d7c1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -163,31 +163,22 @@ uncloseai-speech/ ## TTS Engine Status -### Production Ready (95.9% success rate across 245 voices) -- ✅ Piper TTS (tts-1) - 55 voices, fast CPU inference -- ✅ XTTS v2 (tts-1-hd) - Voice cloning, multilingual -- ✅ Silero TTS (tts-1-silero) - 142 voices, 5 languages, auto-downloads -- ✅ Kokoro TTS (tts-1-kokoro) - 32 voices, lightweight (82M params) +### Default Model (Qwen3-TTS) +- ✅ **Qwen3-TTS (tts-1-qwen)** - DEFAULT - 1.7B params, 10 languages, voice cloning, 97ms latency -### High Priority Integration +### Other Engines (disabled by default, enable in voice_to_speaker.yaml) +- Piper TTS (tts-1) - 55 voices, fast CPU inference +- XTTS v2 (tts-1-hd) - Voice cloning, multilingual +- Silero TTS (tts-1-silero) - 142 voices, 5 languages, auto-downloads +- Kokoro TTS (tts-1-kokoro) - 32 voices, lightweight (82M params) - 1. **StyleTTS2** ⭐⭐⭐⭐⭐ - - Why: State-of-the-art quality, best prosody and naturalness - - License: MIT (permissive) - - Challenge: Complex dependencies (phonemizer), slower inference - - Priority: HIGH - Best quality available - - 2. **Fish Speech** ⭐⭐⭐⭐ - - Why: Fast, modern, active development, good multilingual support - - License: Apache 2.0 - - Challenge: Newer/less proven - - Priority: MEDIUM-HIGH - Good balance of quality and speed - - 3. **Chatterbox** ⭐⭐⭐⭐ - - Why: Emotion control, 23 languages, zero-shot cloning - - License: Apache 2.0 - - Challenge: Production complexity - - Priority: MEDIUM-HIGH - Unique emotion features +### Why Qwen3-TTS is Default +- State-of-the-art quality with voice cloning +- Actively maintained by Alibaba +- Apache 2.0 license (commercial-friendly) +- 10 languages: Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, Italian +- Fast first-packet latency (97ms) +- Easy voice cloning from 3-second samples See `docs/MODELS.md` for complete roadmap and detailed model documentation. @@ -257,6 +248,41 @@ tmux send-keys -t 0:1 'cd /home/fox/git/uncloseai-speech && docker compose up -- 4. **Liberation** - Keep TTS libre (AGPL v3) 5. **Unification** - All TTS engines, one API +## AGPL v3 Compliance + +**This project is AGPL v3 licensed.** The key obligation: anyone who uses this TTS service over a network must be able to access the source code. + +### What This Means + +Unlike regular GPL, AGPL closes the "SaaS loophole". If you run uncloseai-speech as a service (even without distributing binaries), users have the right to request source code. + +### Requirements for Operators + +When running uncloseai-speech as a network service, you must provide: +- Complete source code of the running version +- Any modifications you've made +- Build instructions + +### How to Comply + +1. **Link in API response** - Add source URL to `/v1/models` or root endpoint +2. **Link in documentation** - Include repository URL in service docs +3. **Host source code** - Keep your fork in a public git repository + +### Example Implementation + +```python +# In API responses +"source_code": "https://github.com/uncloseai/uncloseai-speech" +``` + +### Why AGPL? + +- Ensures forks remain open source +- Community improvements flow back to the project +- Prevents proprietary TTS services from using our work without sharing back +- Aligns with Raccoon Mission: **Keep TTS libre** + ## Common Mistakes to Avoid ❌ DON'T create directories with raw ssh diff --git a/README.md b/README.md index 098cdc2..a82e4c4 100644 --- a/README.md +++ b/README.md @@ -1,421 +1,387 @@ # uncloseai-speech -🦝 **Raccoon Mission Fork:** Rescuing abandoned TTS models and building a unified, resilient text-to-speech system. +OpenAI-compatible text-to-speech API server with state-of-the-art voice cloning. -**Repository:** https://git.unturf.com/engineering/unturf/uncloseai-speech (GitLab) +**Default Engine:** [Qwen3-TTS](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base) - 1.7B parameters, 10 languages, 97ms latency -**Original Notice:** The original `openedai-speech` project (GitHub) was archived and no longer maintained. This is the active fork. - -**Raccoon Mission:** We're bringing it back to life with: -- ✅ Working Piper TTS (tts-1) - 55 voices, fast CPU inference -- ✅ Working XTTS v2 (tts-1-hd) - 8 voices with cloning capability -- ✅ Working Silero TTS (tts-1-silero) - 148 voices, 5 languages, CPU-friendly -- ✅ Working Kokoro TTS (tts-1-kokoro) - 34 voices, lightweight decoder -- 🎯 Next integrations: StyleTTS2 (best quality), Fish Speech (fast multilingual) -- 📚 Comprehensive documentation in `docs/` -- 🛠️ Makefile-driven deployment workflow -- 🔒 AGPL v3 - keeps TTS libre forever - -See `docs/MODELS.md` for the complete roadmap and `docs/CLAUDE.md` for contribution guidelines. - ----- - -An OpenAI API compatible text to speech server. - -* Compatible with the OpenAI audio/speech API -* Serves the [/v1/audio/speech endpoint](https://platform.openai.com/docs/api-reference/audio/createSpeech) -* Not affiliated with OpenAI in any way, does not require an OpenAI API Key -* A free, private, text-to-speech server with custom voice cloning - -Full Compatibility: -* `tts-1`: `alloy`, `echo`, `fable`, `onyx`, `nova`, and `shimmer` (configurable, 100+ Piper voices available) -* `tts-1-hd`: `alloy`, `echo`, `fable`, `onyx`, `nova`, and `shimmer` (configurable, uses OpenAI samples by default) -* `tts-1-silero`: 148 voices with native names (`en_0`, `en_1`, etc.) across 5 languages (English, Russian, German, Spanish, French) -* `tts-1-kokoro`: `alloy`, `echo`, `fable`, `onyx`, `nova`, `shimmer` (OpenAI-themed voices) + 34 native voices -* `model` parameter is optional - voice auto-detection automatically selects the correct engine -* response_format: `mp3`, `opus`, `aac`, `flac`, `wav` and `pcm` -* speed 0.25-4.0 (and more) - -Available TTS Engines: -* Model `tts-1` via [piper tts](https://github.com/rhasspy/piper) (very fast, runs on cpu) - * You can map your own [piper voices](https://rhasspy.github.io/piper-samples/) via the `voice_to_speaker.yaml` configuration file -* Model `tts-1-hd` via [coqui-ai/TTS](https://github.com/coqui-ai/TTS) xtts_v2 voice cloning (fast, but requires around 4GB GPU VRAM) - * Custom cloned voices can be used for tts-1-hd, See: [Custom Voices Howto](#custom-voices-howto) - * 🌐 [Multilingual](#multilingual) support with XTTS voices, the language is automatically detected if not set - * [Custom fine-tuned XTTS model support](#custom-fine-tuned-model-support) - * Configurable [generation parameters](#generation-parameters) - * Streamed output while generating -* Model `tts-1-silero` via [Silero TTS](https://github.com/snakers4/silero-models) (fast CPU inference, actively maintained) - * 148 voices across 5 languages (English, Russian, German, Spanish, French) - * 48kHz sample rate, excellent quality/speed ratio - * No GPU required, real-time capable on CPU -* Model `tts-1-kokoro` via [Kokoro TTS](https://github.com/hexgrad/kokoro) (lightweight decoder-only architecture) - * 34 voices (American and British English) - * 82M parameters, fast inference - * 24kHz sample rate, Apache 2.0 license -* Occasionally, certain words or symbols may sound incorrect, you can fix them with regex via `pre_process_map.yaml` -* Tested with python 3.9-3.11, piper does not install on python 3.12 yet - -## High Priority Integration Targets - -We're actively working on integrating these state-of-the-art TTS engines: - - 1. **StyleTTS2** ⭐⭐⭐⭐⭐ - - Why: State-of-the-art quality, best prosody and naturalness - - License: MIT (permissive) - - Challenge: Complex dependencies (phonemizer), slower inference - - Priority: HIGH - Best quality available - - 2. **Fish Speech** ⭐⭐⭐⭐ - - Why: Fast, modern, active development, good multilingual support - - License: Apache 2.0 - - Challenge: Newer/less proven - - Priority: MEDIUM-HIGH - Good balance of quality and speed - -See [docs/MODELS.md](docs/MODELS.md) for the complete integration roadmap and detailed documentation on all supported and planned TTS engines. - - -If you find a better voice match for `tts-1` or `tts-1-hd`, please let me know so I can update the defaults. - -## Recent Changes - -See [CHANGELOG.md](CHANGELOG.md) for full version history. - -## Installation instructions - -### Recommended: Makefile-based workflow - -The project includes a comprehensive Makefile for deployment and development. See available commands: +## Quick Start ```bash -make help +git clone https://github.com/uncloseai/uncloseai-speech.git +cd uncloseai-speech + +# Option 1: Docker with GPU (recommended) +make local + +# Option 2: Docker CPU only +make local-cpu + +# Option 3: Python venv (no Docker) +make venv && make venv-run ``` -#### Quick Start with Makefile - -1. **Create deployment configuration** (if deploying to remote server): +Test the API: ```bash -cp vars.sh.example vars.sh -# Edit vars.sh with your server details +curl http://localhost:8000/v1/audio/speech \ + -H "Content-Type: application/json" \ + -d '{"input":"Hello from Qwen TTS!","voice":"alloy"}' \ + -o test.mp3 ``` -2. **Deploy to remote server**: +## Requirements + +| Setup | GPU | RAM | Disk | Notes | +|-------|-----|-----|------|-------| +| Docker + GPU | NVIDIA 8GB+ VRAM | 8GB | 5GB | Recommended, fastest | +| Docker + CPU | None | 16GB | 5GB | ~10x slower | +| Python venv | Optional | 16GB | 5GB | Direct install | + +### GPU Setup (NVIDIA) + +Install [nvidia-container-toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html): + ```bash -make deploy # Sync files, rebuild container, restart -make voices # Download Piper + XTTS voice models -make test # Test the API -make logs # View live logs +# Ubuntu/Debian +curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg +curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \ + sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \ + sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list +sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit +sudo nvidia-ctk runtime configure --runtime=docker +sudo systemctl restart docker ``` -3. **Local development**: +Verify GPU access: ```bash -make local-deploy # Deploy locally with docker compose +docker run --rm --gpus all nvidia/cuda:12.0-base nvidia-smi ``` -See `docs/CLAUDE.md` for detailed Makefile usage and development workflow. +## Installation -### Alternative: Manual Docker setup +### Docker with GPU -#### Create a `speech.env` environment file - -Copy the `sample.env` to `speech.env` (customize if needed) ```bash cp sample.env speech.env +make local +# Or: docker compose up -d --build ``` -**Defaults:** +### Docker CPU Only + ```bash -TTS_HOME=voices -HF_HOME=voices -#PRELOAD_MODEL=xtts -#PRELOAD_MODEL=xtts_v2.0.2 -#EXTRA_ARGS=--log-level DEBUG --unload-timer 300 -#USE_ROCM=1 +cp sample.env speech.env +make local-cpu +# Or: docker compose -f docker-compose.cpu.yml up -d --build ``` -#### Docker Images +### Python Virtual Environment -**Nvidia GPU (cuda)** -```shell -docker compose up -``` +```bash +# Create and activate venv +make venv -**AMD GPU (ROCm support)** -```shell -docker compose -f docker-compose.rocm.yml up -``` +# Run the server +make venv-run -**ARM64 (Apple M-series, Raspberry Pi)** -> XTTS only has CPU support here and will be very slow, you can use the Nvidia image for XTTS with CPU (slow), or use the piper only image (recommended) - -**CPU only, No GPU (piper only)** -> For a minimal docker image with only piper support (<1GB vs. 8GB). -```shell -docker compose -f docker-compose.min.yml up -``` - -### Alternative: Manual Python installation - -```shell -# install curl and ffmpeg -sudo apt install curl ffmpeg -# Create & activate a new virtual environment (optional but recommended) -python -m venv .venv +# Or manually: +python3 -m venv .venv source .venv/bin/activate -# Install the Python requirements -# - use requirements-rocm.txt for AMD GPU (ROCm support) -# - use requirements-min.txt for piper only (CPU only) -pip install -U -r requirements.txt -# run the server -bash startup.sh +pip install -r requirements.txt +python speech.py ``` -> On first run, the voice models will be downloaded automatically. This might take a while depending on your network connection. +### AMD GPU (ROCm) -## Server Options - -```shell -usage: speech.py [-h] [--xtts_device XTTS_DEVICE] [--preload PRELOAD] [--unload-timer UNLOAD_TIMER] - [--use-deepspeed] [--no-cache-speaker] [-W WORKERS] [-P PORT] [-H HOST] - [-L {DEBUG,INFO,WARNING,ERROR,CRITICAL}] - -uncloseai-speech API Server - -options: - -h, --help show this help message and exit - --xtts_device XTTS_DEVICE - Set the device for the xtts model. The special value of 'none' will use piper for all models. (default: cuda) - --preload PRELOAD Preload a model (Ex. 'xtts' or 'xtts_v2.0.2'). By default it's loaded on first use. (default: None) - --unload-timer UNLOAD_TIMER - Idle unload timer for the XTTS model in seconds, Ex. 900 for 15 minutes (default: None) - --use-deepspeed Use deepspeed with xtts (this option is unsupported) (default: False) - --no-cache-speaker Don't use the speaker wav embeddings cache (default: False) - -W WORKERS, --workers WORKERS - Number of uvicorn worker processes for concurrent request handling (default: 4) - -P PORT, --port PORT Server tcp port (default: 8000) - -H HOST, --host HOST Host to listen on, Ex. 0.0.0.0 (default: 0.0.0.0) - -L {DEBUG,INFO,WARNING,ERROR,CRITICAL}, --log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL} - Set the log level (default: INFO) +```bash +docker compose -f docker-compose.rocm.yml up -d --build ``` +## API Reference -## Sample Usage +### Generate Speech -You can use it like this: +```bash +POST /v1/audio/speech +``` -```shell -curl http://localhost:8000/v1/audio/speech -H "Content-Type: application/json" -d '{ - "model": "tts-1", - "input": "The quick brown fox jumped over the lazy dog.", +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `input` | string | required | Text to synthesize | +| `voice` | string | `alloy` | Voice name | +| `model` | string | `tts-1-qwen` | Model ID | +| `response_format` | string | `mp3` | `mp3`, `opus`, `aac`, `flac`, `wav`, `pcm` | +| `speed` | float | `1.0` | Speed multiplier (0.25-4.0) | + +**Example:** +```bash +curl http://localhost:8000/v1/audio/speech \ + -H "Content-Type: application/json" \ + -d '{ + "model": "tts-1-qwen", "voice": "alloy", + "input": "The quick brown fox jumped over the lazy dog.", "response_format": "mp3", "speed": 1.0 - }' > speech.mp3 + }' -o speech.mp3 ``` -Or just like this: +### List Models -```shell -curl -s http://localhost:8000/v1/audio/speech -H "Content-Type: application/json" -d '{ - "input": "The quick brown fox jumped over the lazy dog."}' > speech.mp3 +```bash +GET /v1/models ``` -Or like this example from the [OpenAI Text to speech guide](https://platform.openai.com/docs/guides/text-to-speech): +### List Voices + +```bash +GET /v1/voices +``` + +Returns all voices with metadata including engine, sample rate, and language support. + +## Python SDK Usage ```python import openai client = openai.OpenAI( - # This part is not needed if you set these environment variables before import openai - # export OPENAI_API_KEY=sk-11111111111 - # export OPENAI_BASE_URL=http://localhost:8000/v1 - api_key = "sk-111111111", - base_url = "http://localhost:8000/v1", + api_key="not-needed", + base_url="http://localhost:8000/v1", ) +# Basic usage with client.audio.speech.with_streaming_response.create( - model="tts-1", - voice="alloy", - input="Today is a wonderful day to build something people love!" + model="tts-1-qwen", + voice="alloy", + input="Hello world!" ) as response: - response.stream_to_file("speech.mp3") + response.stream_to_file("speech.mp3") + +# With options +with client.audio.speech.with_streaming_response.create( + model="tts-1-qwen", + voice="nova", + input="This is faster speech.", + response_format="opus", + speed=1.2 +) as response: + response.stream_to_file("speech.opus") ``` -Also see the `say.py` sample application for an example of how to use the openai-python API. +## Voice Cloning -```shell -# play the audio, requires 'pip install playsound' -python say.py -t "The quick brown fox jumped over the lazy dog." -p -# save to a file in flac format -python say.py -t "The quick brown fox jumped over the lazy dog." -m tts-1-hd -v onyx -f flac -o fox.flac +Qwen3-TTS clones any voice from a 3+ second audio sample. + +### 1. Prepare Reference Audio + +- **Length:** 3-30 seconds (6-10 optimal) +- **Quality:** Clear speech, minimal noise +- **Format:** WAV, MP3, or URL + +### 2. Configure Voice + +Edit `config/voice_to_speaker.yaml`: + +```yaml +tts-1-qwen: + my_voice: + ref_audio: voices/my_sample.wav # Local file or URL + ref_text: "Exact transcript of the audio." + language: English ``` -You can also try the included `audio_reader.py` for listening to longer text and streamed input. +### 3. Use the Voice -Example usage: ```bash -python audio_reader.py -s 2 < LICENSE # read the software license - fast +curl http://localhost:8000/v1/audio/speech \ + -H "Content-Type: application/json" \ + -d '{"voice":"my_voice","input":"Hello in my cloned voice!"}' \ + -o output.mp3 ``` -## OpenAI API Documentation and Guide +### Supported Languages -* [OpenAI Text to speech guide](https://platform.openai.com/docs/guides/text-to-speech) -* [OpenAI API Reference](https://platform.openai.com/docs/api-reference/audio/createSpeech) +Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, Italian +## Default Voices -## Custom Voices Howto +| Voice | Description | +|-------|-------------| +| `alloy` | Neutral, balanced | +| `echo` | Warm, conversational | +| `fable` | Expressive, storytelling | +| `onyx` | Deep, authoritative | +| `nova` | Friendly, upbeat | +| `shimmer` | Soft, gentle | -### Piper +All voices use Qwen3-TTS voice cloning with pre-configured reference audio. - 1. Select the piper voice and model from the [piper samples](https://rhasspy.github.io/piper-samples/) - 2. Update the `config/voice_to_speaker.yaml` with a new section for the voice, for example: -```yaml -... -tts-1: - ryan: - model: voices/en_US-ryan-high.onnx - speaker: # default speaker -``` - 3. New models will be downloaded as needed, of you can download them in advance with `download_voices_tts-1.sh`. For example: -```shell -bash download_voices_tts-1.sh en_US-ryan-high +## Configuration + +### Environment Variables + +Edit `speech.env`: + +```bash +TTS_HOME=voices # Model cache directory +HF_HOME=voices # HuggingFace cache +EXTRA_ARGS=--log-level INFO # Additional server args ``` -### Coqui XTTS v2 +### Server Arguments -Coqui XTTS v2 voice cloning can work with as little as 6 seconds of clear audio. To create a custom voice clone, you must prepare a WAV file sample of the voice. - -#### Guidelines for preparing good sample files for Coqui XTTS v2 -* Mono (single channel) 22050 Hz WAV file -* 6-30 seconds long - longer isn't always better (I've had some good results with as little as 4 seconds) -* low noise (no hiss or hum) -* No partial words, breathing, laughing, music or backgrounds sounds -* An even speaking pace with a variety of words is best, like in interviews or audiobooks. -* Audio longer than 30 seconds will be silently truncated. - -You can use FFmpeg to prepare your audio files, here are some examples: - -```shell -# convert a multi-channel audio file to mono, set sample rate to 22050 hz, trim to 6 seconds, and output as WAV file. -ffmpeg -i input.mp3 -ac 1 -ar 22050 -t 6 -y me.wav -# use a simple noise filter to clean up audio, and select a start time start for sampling. -ffmpeg -i input.wav -af "highpass=f=200, lowpass=f=3000" -ac 1 -ar 22050 -ss 00:13:26.2 -t 6 -y me.wav -# A more complex noise reduction setup, including volume adjustment -ffmpeg -i input.mkv -af "highpass=f=200, lowpass=f=3000, volume=5, afftdn=nf=25" -ac 1 -ar 22050 -ss 00:13:26.2 -t 6 -y me.wav +``` +--xtts_device DEVICE Device: cuda, cpu, none (default: auto-detect) +--workers N Worker processes (default: 4) +--port PORT Listen port (default: 8000) +--host HOST Bind address (default: 0.0.0.0) +--log-level LEVEL DEBUG, INFO, WARNING, ERROR, CRITICAL ``` -Once your WAV file is prepared, save it in the `/voices/` directory and update the `config/voice_to_speaker.yaml` file with the new file name. +## Makefile Commands -For example: +```bash +make help # Show all commands -```yaml -... -tts-1-hd: - me: - model: xtts - speaker: voices/me.wav # this could be you +# Local Development +make local # Docker with GPU +make local-cpu # Docker CPU only +make venv # Create Python venv +make venv-run # Run in venv + +# Testing +make test # Test API +make logs # View logs + +# Remote Deployment +make deploy # Sync + restart remote +make sync # Sync files only +make restart # Restart container + +# Container +make start # Start container +make stop # Stop container +make clean # Remove container ``` -You can also use a sub folder for multiple audio samples to combine small samples or to mix different samples together. +## Other TTS Engines -For example: +These engines are disabled by default. Enable by uncommenting in `config/voice_to_speaker.yaml` and `requirements.txt`. -```yaml -... -tts-1-hd: - mixed: - model: xtts - speaker: voices/mixed +| Model | Engine | Voices | Speed | Notes | +|-------|--------|--------|-------|-------| +| `tts-1` | Piper | 55 | Fast | CPU optimized | +| `tts-1-hd` | XTTS v2 | 8 | Medium | Voice cloning | +| `tts-1-silero` | Silero | 148 | Fast | 5 languages | +| `tts-1-kokoro` | Kokoro | 34 | Fast | 82M params | + +See [docs/MODELS.md](docs/MODELS.md) for details. + +## Troubleshooting + +### Model Download Fails + +```bash +# Check logs +docker logs uncloseai-speech-server-1 + +# Manual download +docker exec -it uncloseai-speech-server-1 \ + huggingface-cli download Qwen/Qwen3-TTS-12Hz-1.7B-Base ``` -Where the `voices/mixed/` folder contains multiple wav files. The total audio length is still limited to 30 seconds. +### Out of GPU Memory -## Multilingual +Qwen3-TTS needs ~6GB VRAM. Options: -uncloseai-speech supports multiple languages across different TTS engines: +1. Add to `speech.env`: `EXTRA_ARGS=--xtts_device cpu` +2. Reduce workers: `EXTRA_ARGS=--workers 1` +3. Use CPU-only: `make local-cpu` -### XTTS (tts-1-hd) - 17 Languages +### Slow Generation -Multilingual cloning support was added in version 0.11.0. Coqui XTTSv2 has support for multiple languages: English (`en`), Spanish (`es`), French (`fr`), German (`de`), Italian (`it`), Portuguese (`pt`), Polish (`pl`), Turkish (`tr`), Russian (`ru`), Dutch (`nl`), Czech (`cs`), Arabic (`ar`), Chinese (`zh-cn`), Hungarian (`hu`), Korean (`ko`), Japanese (`ja`), and Hindi (`hi`). When not set, an attempt will be made to automatically detect the language, falling back to English (`en`). +- GPU: ~1-2 seconds per sentence +- CPU: ~10-20 seconds per sentence -Unfortunately the OpenAI API does not support language parameters, but you can create your own custom speaker voice and set the language for that. +For faster CPU inference, enable Piper or Silero engines. -1) Create the WAV file for your speaker, as in [Custom Voices Howto](#custom-voices-howto) -2) Add the voice to `config/voice_to_speaker.yaml` and include the correct Coqui `language` code for the speaker. For example: +### Voice Quality Issues -```yaml - xunjiang: - model: xtts - speaker: voices/xunjiang.wav - language: zh-cn +- Use 6-10 seconds of clear reference audio +- Ensure transcript exactly matches audio +- Avoid background noise +- Match language setting to audio language + +## Architecture + +``` +┌─────────────────────────────────────────────┐ +│ Client │ +│ (OpenAI SDK / curl) │ +└─────────────────┬───────────────────────────┘ + │ HTTP POST /v1/audio/speech + ▼ +┌─────────────────────────────────────────────┐ +│ FastAPI Server │ +│ (speech.py, port 8000) │ +├─────────────────────────────────────────────┤ +│ │ +│ ┌─────────────┐ Voice Config │ +│ │ Qwen3-TTS │◄─────────────────────────┐ │ +│ │ (default) │ config/voice_to_speaker │ │ +│ └──────┬──────┘ │ │ +│ │ │ │ +│ ▼ │ │ +│ ┌─────────────┐ │ │ +│ │ FFmpeg │ Audio encoding │ │ +│ │ (mp3/opus) │ │ │ +│ └──────┬──────┘ │ │ +│ │ │ │ +└─────────┼──────────────────────────────────┘ + │ + ▼ Audio stream + Client ``` -3) Don't remove high unicode characters in your `config/pre_process_map.yaml`! If you have these lines, you will need to remove them. For example: +## License -Remove: -```yaml -- - '[\U0001F600-\U0001F64F\U0001F300-\U0001F5FF\U0001F680-\U0001F6FF\U0001F700-\U0001F77F\U0001F780-\U0001F7FF\U0001F800-\U0001F8FF\U0001F900-\U0001F9FF\U0001FA00-\U0001FA6F\U0001FA70-\U0001FAFF\U00002702-\U000027B0\U000024C2-\U0001F251]+' - - '' +**AGPL v3** - This software is licensed under the [GNU Affero General Public License v3](https://www.gnu.org/licenses/agpl-3.0.html). + +### Key AGPL v3 Obligations + +1. **Network use triggers copyleft** - Unlike regular GPL, AGPL closes the "SaaS loophole". If you run uncloseai-speech as a service (even without distributing binaries), users have the right to request source code. + +2. **What you must provide:** + - Complete source code of the running version + - Any modifications you've made + - Build instructions + +3. **How to comply:** + - Link to your source repository in API responses or docs + - Offer source code download from the same server + - Keep your modifications in a public git repo + +### Practical Implementation + +For a service at `ai.foxhop.net`, you could: + +```python +# Add to API response headers or /source endpoint +"source_code": "https://github.com/uncloseai/uncloseai-speech" ``` -These lines were added to the `config/pre_process_map.yaml` config file by default before version 0.11.0. +Or include it in your API's `/models` or root endpoint response. -4) Your new multi-lingual speaker voice is ready to use! +### Why AGPL for TTS? -### Silero (tts-1-silero) - 5 Languages +From the Raccoon Mission values: +- **Liberation** - Keeps TTS libre +- **Resilience** - Ensures forks remain open +- **Unification** - Community improvements flow back -Silero TTS provides native multilingual support with 148 voices across 5 languages: -- **English (en)** - 117 voices -- **Russian (ru)** - 10 voices -- **German (de)** - 5 voices -- **Spanish (es)** - 3 voices -- **French (fr)** - 5 voices +## Links -Each language has multiple speaker variations. Voices are automatically configured in `config/voice_to_speaker.yaml`. Simply select the appropriate voice (e.g., `en_0`, `ru_0`, `de_0`) and the correct language model will be loaded automatically. - -### Kokoro (tts-1-kokoro) - English Only - -Kokoro TTS currently supports only American and British English with 34 high-quality voices. The model uses a lightweight decoder-only architecture (82M parameters) optimized for English speech synthesis. - -### Piper (tts-1) - 50+ Languages - -For Piper TTS, simply download language-specific voice models from [piper samples](https://rhasspy.github.io/piper-samples/). Piper supports 50+ languages with hundreds of voice options. Add the voice to `config/voice_to_speaker.yaml` as shown in [Custom Voices Howto](#custom-voices-howto). - - -## Custom Fine-Tuned Model Support - -Adding a custom xtts model is simple. Here is an example of how to add a custom fine-tuned 'halo' XTTS model. - -1) Save the model folder under `voices/` (all 4 files are required, including the vocab.json from the model) -``` -uncloseai-speech$ ls voices/halo/ -config.json vocab.json model.pth sample.wav -``` -2) Add the custom voice entry under the `tts-1-hd` section of `config/voice_to_speaker.yaml`: -```yaml -tts-1-hd: -... - halo: - model: halo # This name is required to be unique - speaker: voices/halo/sample.wav # voice sample is required - model_path: voices/halo -``` -3) The model will be loaded when you access the voice for the first time (`--preload` doesn't work with custom models yet) - -## Generation Parameters - -The generation of XTTSv2 voices can be fine tuned with the following options (defaults included below): - -```yaml -tts-1-hd: - alloy: - model: xtts - speaker: voices/alloy.wav - enable_text_splitting: True - length_penalty: 1.0 - repetition_penalty: 10 - speed: 1.0 - temperature: 0.75 - top_k: 50 - top_p: 0.85 -``` +- [Documentation](docs/MODELS.md) +- [Contributing](CLAUDE.md) +- [Qwen3-TTS Model](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base) +- [OpenAI TTS API Reference](https://platform.openai.com/docs/api-reference/audio/createSpeech)