uncloseai-speech/docs/CLAUDE.md

Instructions for Claude Code

Project: uncloseai-speech - Raccoon Mission TTS System License: AGPL v3 (must provide source code to network service users)

Brand Identity

CRITICAL: Always use consistent naming across all files.

Project Name

• ✅ Correct: uncloseai-speech (lowercase, hyphenated)
• ❌ Wrong: "UncloseAI Speech", "Uncloseai Speech", "UncloseAI-Speech"

Organization Name

• ✅ Correct: uncloseai (lowercase, one word)
• ❌ Wrong: "UncloseAI", "Unclose AI", "UnClose AI"

Usage Guidelines

• In code: Use uncloseai-speech for project references
• In documentation: Use uncloseai-speech for project name
• In comments: Use uncloseai-speech consistently
• Repository URLs: uncloseai-speech (lowercase, hyphenated)
• Docker images: uncloseai-speech (lowercase, hyphenated)
• API responses: Use "owned_by": "uncloseai" (lowercase, one word)

Examples

# Correct
description='uncloseai-speech API Server'
owned_by = "uncloseai"

# Wrong
description='UncloseAI Speech API Server'
owned_by = "UncloseAI"

# Correct
# uncloseai-speech

**Raccoon Mission:** Rescue abandoned TTS models and integrate them into uncloseai-speech

# Wrong
# UncloseAI Speech

**Raccoon Mission:** Rescue abandoned TTS models and integrate them into UncloseAI Speech

Core Principles

1. Makefile-First Development

ALWAYS prefer Makefile targets over manual commands.

• ✅ DO: make deploy, make voices, make test
• ❌ DON'T: Manual ssh commands, docker commands, curl commands

When adding new functionality:

1. Add it to the Makefile first
2. Document it in make help
3. Test it works from scratch
4. Only then modify other files if needed

Makefile is the source of truth for all deployment and development tasks.

2. Work Locally, Deploy Remotely

• Local development: /home/fox/git/openedai-speech/
• Remote server: Configured in vars.sh (gitignored)
• Never create remote directories manually - let Makefile handle it
• Always test from scratch - make clean then make deploy

3. Configuration Management

• vars.sh - Deployment secrets (gitignored, never commit)
• vars.sh.example - Template for users (commit this)
• sample.env - Default environment (commit this)
• speech.env - Runtime environment (created automatically by Makefile)

Never view or log secrets - source them and use them.

4. Documentation Requirements

When adding features, update ALL relevant docs:

• Makefile help text
• docs/MODELS.md for new TTS engines
• docs/MIRRORS.md for binary downloads
• docs/AUDIT.md for file changes
• This file (docs/CLAUDE.md) for new patterns

Common Tasks

Full Deployment from Scratch

# 1. Clean everything
make clean

# 2. Deploy (syncs files, creates env, builds container)
make deploy

# 3. Download voices (Piper + XTTS samples)
make voices

# 4. Test
make test
make test-xtts

Adding a New TTS Engine

1. Document it in docs/MODELS.md first
2. Add download target to Makefile (e.g., voices-silero)
3. Implement engine wrapper in speech.py or src/engines/
4. Add test target (e.g., test-silero)
5. Update make voices to include it
6. Test full cycle: make clean && make deploy && make voices

Debugging Issues

make logs          # Tail live logs
make logs | grep ERROR  # Filter errors

Never use raw docker/ssh commands - extend Makefile if needed.

File Organization

Scripts vs Docs

• scripts/ - Executable utilities (add_voice.py, download_samples.sh, etc.)
• docs/ - Documentation ONLY (no executable code)
• Dockerfiles, startup.sh - Root level (build artifacts)
• Makefile - Root level (primary interface)

Never put executable scripts in docs/ directory.

Current Structure (as of 2025-11-09)

uncloseai-speech/
├── Makefile              # PRIMARY INTERFACE - always update first
├── vars.sh               # Secrets (gitignored)
├── vars.sh.example       # Template
├── speech.py             # Main server (will refactor to src/)
├── openedai.py           # API models
├── voice_to_speaker.default.yaml  # Voice config
├── docs/
│   ├── CLAUDE.md         # This file
│   ├── AUDIT.md          # Repository audit
│   ├── MODELS.md         # TTS engines
│   └── MIRRORS.md        # Binary mirror strategy
├── scripts/
│   ├── add_voice.py
│   ├── say.py
│   ├── test_voices.sh
│   └── download_samples.sh
├── Dockerfile
├── docker-compose.yml
└── startup.sh

TTS Engine Status

Production Ready (95.9% success rate across 245 voices)

• ✅ Piper TTS (tts-1) - 55 voices, fast CPU inference
• ✅ XTTS v2 (tts-1-hd) - Voice cloning, multilingual
• ✅ Silero TTS (tts-1-silero) - 142 voices, 5 languages, auto-downloads
• ✅ Kokoro TTS (tts-1-kokoro) - 32 voices, lightweight (82M params)

High Priority Integration

1. StyleTTS2 ⭐⭐⭐⭐⭐

   • Why: State-of-the-art quality, best prosody and naturalness
   • License: MIT (permissive)
   • Challenge: Complex dependencies (phonemizer), slower inference
   • Priority: HIGH - Best quality available

2. Fish Speech ⭐⭐⭐⭐

   • Why: Fast, modern, active development, good multilingual support
   • License: Apache 2.0
   • Challenge: Newer/less proven
   • Priority: MEDIUM-HIGH - Good balance of quality and speed

3. Chatterbox ⭐⭐⭐⭐

   • Why: Emotion control, 23 languages, zero-shot cloning
   • License: Apache 2.0
   • Challenge: Production complexity
   • Priority: MEDIUM-HIGH - Unique emotion features

See docs/MODELS.md for complete roadmap and detailed model documentation.

Deployment Workflow

Local:
  /home/fox/git/openedai-speech/

  ↓ make deploy (rsync)

Remote (ai.foxhop.net):
  ~/uncloseai-speech/

  ↓ docker compose up --build

Container:
  /app/
  ├── speech.py
  ├── voices/
  │   └── en/en_US/libritts_r/medium/*.onnx
  └── config/
      └── voice_to_speaker.yaml

Testing Philosophy

Full stack testing workflow:

1. make clean - Clean state
2. make deploy - Fresh deploy
3. make voices - Download voice models
4. make test - Basic API test
5. make hydrate - Test all 245 voices (sequential, safe)
6. make load-test - 100 concurrent requests (stress test)

Never assume - always test from scratch after changes.

Raccoon Mission Values

1. Resilience - Assume upstream dies, plan mirrors
2. Simplicity - Makefile > manual commands
3. Documentation - Write docs before code
4. Liberation - Keep TTS libre (AGPL v3)
5. Unification - All TTS engines, one API

Common Mistakes to Avoid

❌ DON'T create directories with raw ssh ✅ DO add Makefile target for deployment

❌ DON'T assume container has changes after rsync ✅ DO rebuild with make deploy (runs docker compose up --build)

❌ DON'T put scripts in docs/ ✅ DO put scripts in scripts/, reference from docs

❌ DON'T hardcode paths/hosts ✅ DO use vars.sh variables

❌ DON'T forget to test from scratch ✅ DO run make clean && make deploy && make voices

When Things Break

1. Check make logs for errors
2. Verify Makefile was updated
3. Test from clean state
4. Check if container was rebuilt (make deploy does this)
5. Verify voices downloaded (ls in container via make logs approach)

Multiprocess Architecture (uvicorn workers=4)

Key pattern: Worker processes spawn as fresh imports, don't run __main__ block.

Solutions implemented:

1. Caches - Initialize in lifespan context manager (runs per worker)
2. Args - Use DefaultArgs class at module level, override in __main__

See speech.py:23-113 for implementation.

Future Refactoring (Planned)

• Move speech.py, openedai.py, audio_reader.py → src/
• Create engine abstraction layer in src/engines/
• Unified voice config with engine selection
• Binary mirror implementation (MinIO on ai.foxhop.net)

See docs/AUDIT.md for detailed refactoring plan.

---

Remember: Makefile first, documentation second, code third. Test from scratch every time.

🦝 Raccoon Mission: Keep TTS libre, rescue abandoned models, unify all engines.