uncloseai-speech/README.md

# uncloseai-speech

OpenAI-compatible text-to-speech API server with state-of-the-art voice cloning.

**Default Engines:**
- [Qwen3-TTS](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base) - 1.7B parameters, 10 languages, 97ms latency (`tts-1-qwen`)
- [F5-TTS](https://huggingface.co/SWivid/F5-TTS) - 336M parameters, flow-matching zero-shot voice cloning, lower VRAM (`tts-1-f5`)

## Quick Start

```bash
git clone https://github.com/uncloseai/uncloseai-speech.git
cd uncloseai-speech

# Option 1: Docker with GPU (recommended)
make local

# Option 2: Docker CPU only
make local-cpu

# Option 3: Python venv (no Docker)
make venv && make venv-run
```

Test the API:
```bash
curl http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"input":"Hello from Qwen TTS!","voice":"alloy"}' \
  -o test.mp3
```

## Requirements

| Setup | GPU | RAM | Disk | Notes |
|-------|-----|-----|------|-------|
| Docker + GPU | NVIDIA 8GB+ VRAM | 8GB | 5GB | Recommended, fastest |
| Docker + CPU | None | 16GB | 5GB | ~10x slower |
| Python venv | Optional | 16GB | 5GB | Direct install |

### GPU Setup (NVIDIA)

Install [nvidia-container-toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html):

```bash
# Ubuntu/Debian
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
```

Verify GPU access:
```bash
docker run --rm --gpus all nvidia/cuda:12.0-base nvidia-smi
```

## Installation

### Docker with GPU

```bash
cp sample.env speech.env
make local
# Or: docker compose up -d --build
```

### Docker CPU Only

```bash
cp sample.env speech.env
make local-cpu
# Or: docker compose -f docker-compose.cpu.yml up -d --build
```

### Python Virtual Environment

```bash
# Create and activate venv
make venv

# Run the server
make venv-run

# Or manually:
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python speech.py
```

### AMD GPU (ROCm)

```bash
docker compose -f docker-compose.rocm.yml up -d --build
```

## API Reference

### Generate Speech

```bash
POST /v1/audio/speech
```

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `input` | string | required | Text to synthesize |
| `voice` | string | `alloy` | Voice name |
| `model` | string | `tts-1-qwen` | Model ID |
| `response_format` | string | `mp3` | `mp3`, `opus`, `aac`, `flac`, `wav`, `pcm` |
| `speed` | float | `1.0` | Speed multiplier (0.25-4.0) |

**Example:**
```bash
curl http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "tts-1-qwen",
    "voice": "alloy",
    "input": "The quick brown fox jumped over the lazy dog.",
    "response_format": "mp3",
    "speed": 1.0
  }' -o speech.mp3
```

### List Models

```bash
GET /v1/models
```

### List Voices

```bash
GET /v1/voices
```

Returns all voices with metadata including engine, sample rate, and language support.

## Python SDK Usage

```python
import openai

client = openai.OpenAI(
    api_key="not-needed",
    base_url="http://localhost:8000/v1",
)

# Basic usage
with client.audio.speech.with_streaming_response.create(
    model="tts-1-qwen",
    voice="alloy",
    input="Hello world!"
) as response:
    response.stream_to_file("speech.mp3")

# With options
with client.audio.speech.with_streaming_response.create(
    model="tts-1-qwen",
    voice="nova",
    input="This is faster speech.",
    response_format="opus",
    speed=1.2
) as response:
    response.stream_to_file("speech.opus")
```

## Voice Cloning

Qwen3-TTS clones any voice from a 3+ second audio sample.

### 1. Prepare Reference Audio

- **Length:** 3-30 seconds (6-10 optimal)
- **Quality:** Clear speech, minimal noise
- **Format:** WAV, MP3, or URL

### 2. Configure Voice

Edit `config/voice_to_speaker.yaml`:

```yaml
tts-1-qwen:
  my_voice:
    ref_audio: voices/my_sample.wav      # Local file or URL
    ref_text: "Exact transcript of the audio."
    language: English
```

### 3. Use the Voice

```bash
curl http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"voice":"my_voice","input":"Hello in my cloned voice!"}' \
  -o output.mp3
```

### Supported Languages

Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, Italian

## Default Voices

| Voice | Description |
|-------|-------------|
| `alloy` | Neutral, balanced |
| `echo` | Warm, conversational |
| `fable` | Expressive, storytelling |
| `onyx` | Deep, authoritative |
| `nova` | Friendly, upbeat |
| `shimmer` | Soft, gentle |

All voices use Qwen3-TTS voice cloning with pre-configured reference audio.

## Configuration

### Environment Variables

Edit `speech.env`:

```bash
TTS_HOME=voices              # Model cache directory
HF_HOME=voices               # HuggingFace cache
EXTRA_ARGS=--log-level INFO  # Additional server args
```

### Server Arguments

```
--xtts_device DEVICE   Device: cuda, cpu, none (default: auto-detect)
--workers N            Worker processes (default: 4)
--port PORT            Listen port (default: 8000)
--host HOST            Bind address (default: 0.0.0.0)
--log-level LEVEL      DEBUG, INFO, WARNING, ERROR, CRITICAL
```

## Makefile Commands

```bash
make help          # Show all commands

# Local Development
make local         # Docker with GPU
make local-cpu     # Docker CPU only
make venv          # Create Python venv
make venv-run      # Run in venv

# Testing
make test          # Test API
make logs          # View logs

# Remote Deployment
make deploy        # Sync + restart remote
make sync          # Sync files only
make restart       # Restart container

# Container
make start         # Start container
make stop          # Stop container
make clean         # Remove container
```

## Engines

**Enabled by default:**

| Model | Engine | Voices | Speed | Notes |
|-------|--------|--------|-------|-------|
| `tts-1-qwen` | Qwen3-TTS | 40 | Fast | Voice cloning, 10 languages, 1.7B params |
| `tts-1-f5` | F5-TTS | 40 | Faster | Voice cloning, flow-matching, 336M params, lower VRAM |

**Disabled by default** — enable by uncommenting in `config/voice_to_speaker.yaml` and `requirements.txt`:

| Model | Engine | Voices | Speed | Notes |
|-------|--------|--------|-------|-------|
| `tts-1` | Piper | 55 | Fast | CPU optimized |
| `tts-1-hd` | XTTS v2 | 8 | Medium | Voice cloning |
| `tts-1-silero` | Silero | 148 | Fast | 5 languages |
| `tts-1-kokoro` | Kokoro | 34 | Fast | 82M params |

See [docs/MODELS.md](docs/MODELS.md) for details.

## Troubleshooting

### Model Download Fails

```bash
# Check logs
docker logs uncloseai-speech-server-1

# Manual download
docker exec -it uncloseai-speech-server-1 \
  huggingface-cli download Qwen/Qwen3-TTS-12Hz-1.7B-Base
```

### Out of GPU Memory

Qwen3-TTS needs ~6GB VRAM. Options:

1. Add to `speech.env`: `EXTRA_ARGS=--xtts_device cpu`
2. Reduce workers: `EXTRA_ARGS=--workers 1`
3. Use CPU-only: `make local-cpu`

### Slow Generation

- GPU: ~1-2 seconds per sentence
- CPU: ~10-20 seconds per sentence

For faster CPU inference, enable Piper or Silero engines.

### Voice Quality Issues

- Use 6-10 seconds of clear reference audio
- Ensure transcript exactly matches audio
- Avoid background noise
- Match language setting to audio language

## Architecture

```
┌─────────────────────────────────────────────┐
│                  Client                      │
│            (OpenAI SDK / curl)              │
└─────────────────┬───────────────────────────┘
                  │ HTTP POST /v1/audio/speech
                  ▼
┌─────────────────────────────────────────────┐
│              FastAPI Server                  │
│           (speech.py, port 8000)            │
├─────────────────────────────────────────────┤
│                                             │
│  ┌─────────────┐  Voice Config              │
│  │ Qwen3-TTS   │◄─────────────────────────┐ │
│  │  (default)  │  config/voice_to_speaker │ │
│  └──────┬──────┘                          │ │
│         │                                  │ │
│         ▼                                  │ │
│  ┌─────────────┐                          │ │
│  │   FFmpeg    │  Audio encoding          │ │
│  │  (mp3/opus) │                          │ │
│  └──────┬──────┘                          │ │
│         │                                  │ │
└─────────┼──────────────────────────────────┘
          │
          ▼ Audio stream
       Client
```

## License

**AGPL v3** - This software is licensed under the [GNU Affero General Public License v3](https://www.gnu.org/licenses/agpl-3.0.html).

### Key AGPL v3 Obligations

1. **Network use triggers copyleft** - Unlike regular GPL, AGPL closes the "SaaS loophole". If you run uncloseai-speech as a service (even without distributing binaries), users have the right to request source code.

2. **What you must provide:**
   - Complete source code of the running version
   - Any modifications you've made
   - Build instructions

3. **How to comply:**
   - Link to your source repository in API responses or docs
   - Offer source code download from the same server
   - Keep your modifications in a public git repo

### Practical Implementation

For a service at `ai.foxhop.net`, you could:

```python
# Add to API response headers or /source endpoint
"source_code": "https://github.com/uncloseai/uncloseai-speech"
```

Or include it in your API's `/models` or root endpoint response.

### Why AGPL for TTS?

From the Raccoon Mission values:
- **Liberation** - Keeps TTS libre
- **Resilience** - Ensures forks remain open
- **Unification** - Community improvements flow back

## Links

- [Documentation](docs/MODELS.md)
- [Contributing](CLAUDE.md)
- [Qwen3-TTS Model](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base)
- [OpenAI TTS API Reference](https://platform.openai.com/docs/api-reference/audio/createSpeech)