uncloseai-speech/CLAUDE.md

Instructions for Claude Code

Project: uncloseai-speech - Raccoon Mission TTS System License: AGPL v3 (must provide source code to network service users)

Brand Identity

CRITICAL: Always use consistent naming across all files.

Project Name

• Correct: uncloseai-speech (lowercase, hyphenated)
• Wrong: "UncloseAI Speech", "Uncloseai Speech", "UncloseAI-Speech"

Organization Name

• Correct: uncloseai (lowercase, one word)
• Wrong: "UncloseAI", "Unclose machine learning", "UnClose machine learning"

Usage Guidelines

• In code: Use uncloseai-speech for project references
• In documentation: Use uncloseai-speech for project name
• In comments: Use uncloseai-speech consistently
• Repository URLs: uncloseai-speech (lowercase, hyphenated)
• Docker images: uncloseai-speech (lowercase, hyphenated)
• API responses: Use "owned_by": "uncloseai" (lowercase, one word)

Core Principles

1. Makefile-First Development

ALWAYS prefer Makefile targets over manual commands.

• DO: make deploy, make voices, make test
• DON'T: Manual docker commands, curl commands

Makefile is our source of truth for all deployment and development tasks.

2. Remote Access: tmux-hosts and tmux ONLY

CRITICAL: NEVER use ssh, scp, or rsync to access remote servers.

Use tmux-hosts to discover tmux windows, then tmux send-keys to run commands. Window numbers shift; never hardcode them. Always discover first.

tmux-hosts
# example output (yours WILL differ — verify every session):
# 0:0          3090-ai.foxhop.net
# 0:2          ai.foxhop.net           # 4090, current speech prod

tmux send-keys -t 0:2 'command here' Enter
tmux capture-pane -t 0:2 -p | tail -20

• NEVER use ssh user@host "command" — use tmux send-keys
• NEVER use rsync or scp — use git push + tmux send-keys '... git pull ...' Enter
• ALWAYS rerun tmux-hosts to confirm window numbers before sending keys

3. Git-Based Deployment

We use git, not rsync/scp. All code syncs via git push/pull.

# 1. local: commit and push
git add files && git commit -m "message" && git push

# 2. remote: pull + restart speech.py (see Production Deployment below for the relaunch)
tmux send-keys -t <prod-window> 'cd /mnt/data/git/uncloseai-speech && sudo -u fox git pull' Enter

4. All Commands Run Locally

Our Makefile assumes it runs on our server directly. No remote execution. When you need to run a make target on a remote host, use tmux:

tmux send-keys -t <prod-window> 'cd /mnt/data/git/uncloseai-speech && make <target>' Enter

5. Configuration Management

• sample.env - Default environment (commit this)
• speech.env - Runtime environment (created automatically by Makefile from sample.env)

6. Git Commit Guidelines

• Never add machine learning attribution - Do not use Co-Authored-By: Claude or similar in commit messages
• Write clear, concise commit messages describing what changed and why
• Use imperative mood ("Add feature" not "Added feature")

Production Deployment

Prod runs as a bare python process on a 4090 host (ai.foxhop.net) — no Docker. Caddy on 80/443 reverse-proxies https://speech.ai.unturf.com to local port 8000.

Host	ai.foxhop.net (4090). Find current tmux window with tmux-hosts.
Run user	fox
Repo	/mnt/data/git/uncloseai-speech (symlinked from /home/fox/git/uncloseai-speech)
Venv	/mnt/data/f5-sidecar/venv/
Logs	/mnt/data/f5-sidecar/logs/speech.log
Launch cmd	python speech.py --workers 1 --port 8000 --log-level INFO
Frontend	Caddy → https://speech.ai.unturf.com
Git remote	ssh://git@git.unturf.com:2222/engineering/unturf/uncloseai-speech.git
Git user on server	fox

Docker compose files (docker-compose.yml, Dockerfile) are kept for parity with self-hosted deploys, but our prod does NOT use them.

Deploy a code/config change

# 1. local
git push

# 2. remote (find window via tmux-hosts, then):
tmux send-keys -t <4090-window> '\
  cd /mnt/data/git/uncloseai-speech && \
  sudo -u fox git pull && \
  PID=$(sudo ss -tlnp | awk "/:8000 / {print \$NF}" | sed "s/.*pid=\([0-9]*\),.*/\1/") && \
  echo "killing PID $PID" && sudo -u fox kill $PID && \
  until ! sudo ss -tln | grep -q :8000; do sleep 1; done && \
  cd /mnt/data/git/uncloseai-speech && \
  sudo -u fox bash -c "nohup env PYTHONUNBUFFERED=1 /mnt/data/f5-sidecar/venv/bin/python speech.py --workers 1 --port 8000 --log-level INFO > /mnt/data/f5-sidecar/logs/speech.log 2>&1 &" && \
  until sudo ss -tln | grep -q :8000; do sleep 2; done && echo PORT_BOUND \
' Enter

Audio yaml / cloned-voices/ changes don't need a code change — same restart picks them up (speech.py loads yaml at startup).

Quick checks

# is prod up?
curl -sS -o /dev/null -w "%{http_code}\n" https://speech.ai.unturf.com/v1/voices

# what PID is bound to :8000?
tmux send-keys -t <4090-window> 'sudo ss -tlnp | grep :8000' Enter

# tail prod log
tmux send-keys -t <4090-window> 'tail -50 /mnt/data/f5-sidecar/logs/speech.log' Enter

# GPU state
tmux send-keys -t <4090-window> 'nvidia-smi' Enter

Voice Configuration

40 distinct gendered voices from LibriSpeech test-clean (public domain). Roster lives in cloned-voices/voices_metadata.json.

Voice WAV files in cloned-voices/ are read directly by speech.py (no container mount — prod runs bare). Engine→voice mapping in voice_to_speaker.default.yaml.

F5-TTS ref_text values are generated by make whisper-refs (whisper-large-v3 over the actual wav files) — they match what the audio sounds like, not LibriSpeech ground-truth labels. Regenerate any time cloned-voices/ changes.

Fixing Mispronunciations

When a voice mispronounces a word (e.g., F5-TTS said "Provenance" wrong), patch via text pre-processing — our engines read what we feed them, so a phonetic respelling at the input layer is our fastest fix. No model retraining.

Where:

• config/pre_process_map.yaml — live, tracked. What preprocess() reads.
• pre_process_map.default.yaml — seed for fresh installs. Edit both, keep them in sync.

Pattern: word-boundary, case-insensitive, respell phonetically. Hyphens act as syllable hints. Example:

# F5-TTS mispronounces "Provenance" — respell phonetically (per fox)
- - (?i)\bProvenance\b
  - prahvanans

Workflow:

1. Edit both yaml files locally with our respelling.
2. git add both, commit, git push.
3. On prod, git pull — no restart needed. preprocess() re-reads our yaml on every request (speech.py:646), so toggles are live.

   tmux send-keys -t <4090-window> \
     'cd /mnt/data/git/uncloseai-speech && sudo -u fox git pull' Enter
   

4. Smoke test via our prod curl (see Testing below). Save output to a temp mp3 and listen.
5. If it still sounds wrong, iterate our respelling (e.g., Prov-uh-dence → Prav-uh-dence → prahvadence). Repeat 1–4 — fox is the ear, ask for the target phonetics.

A/B trick: commit both an active and a # commented-out variant so we can toggle on prod without redeploys. Strip stale stubs after fox confirms a respelling.

TTS Engine Status

tts-1-f5 (F5-TTS) is what prod serves today. tts-1-qwen (Qwen3-TTS) is also wired up. Other engines (Piper, XTTS, Silero, Kokoro, Chatterbox) live in voice_to_speaker.default.yaml but are disabled by default. See docs/MODELS.md for the full roadmap.

Testing

# end-to-end via the prod URL (always-correct)
curl -sS -X POST https://speech.ai.unturf.com/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"model":"tts-1-f5","voice":"aria","input":"Smoke test."}' \
  -o /tmp/test.mp3 -w "http=%{http_code} size=%{size_download}\n"

# on the prod host directly (via tmux), bypassing Caddy
tmux send-keys -t <4090-window> 'cd /mnt/data/git/uncloseai-speech && make test-f5' Enter

When Things Break

# tail prod log
tmux send-keys -t <4090-window> 'tail -100 /mnt/data/f5-sidecar/logs/speech.log' Enter

# GPU memory
tmux send-keys -t <4090-window> 'nvidia-smi' Enter

# kill + relaunch (see "Deploy a code/config change" above for the full sequence)

Common Mistakes to Avoid

• DON'T use ssh user@host "command" — use tmux send-keys
• DON'T use rsync or scp — use git push + tmux git pull
• DON'T hardcode tmux window numbers — rediscover via tmux-hosts every session
• DON'T put scripts in docs/ — put them in scripts/
• DON'T forget to push before deploying
• DON'T assume prod uses docker — it's bare python; the compose files are for self-hosted parity only

AGPL v3 Compliance

This project is AGPL v3 licensed. Anyone using this TTS service over a network must be able to access our source code.

Multiprocess Architecture

Worker processes spawn as fresh imports, don't run __main__ block.

1. Caches - Initialize in lifespan context manager (runs per worker)
2. Args - Use DefaultArgs class at module level, override in __main__

See speech.py:23-113 for implementation.

---

Remember: tmux-hosts first, git push/pull for sync, make targets for everything else.

Style

• Never use "AI" — always say "machine learning." We grow machine learning, not "AI." This term is forbidden in all permacomputer discourse, marketing, & documentation.
• Prefer "our" for shared things; "a" when something is one of many; avoid "the" — it implies fixed, singular ownership. Most teams and systems are fluid and ever-changing, like water.