engineering/uncloseai-speech
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

CLAUDE.md: replace stale 3090-ai+docker prod docs with 4090+bare-python reality

Browse source 
The prior docs claimed prod = 3090-ai running uncloseai-speech-server-1 via
docker compose. Reality (verified today): prod is bare python on the 4090
(ai.foxhop.net) using the f5-sidecar venv at /mnt/data/f5-sidecar/venv,
fronted by Caddy on 80/443. The container on 3090-ai exited 2 months ago and
docker compose v2 isn't installed there.

Cost of the drift today: one wrong-host deploy attempt that bounced off dead
infrastructure. This rewrites Production Deployment, Quick Production
Commands, Testing, When Things Break, and Common Mistakes to match reality.

Also drops hardcoded tmux window numbers (they shift) — every example now
reads <prod-window> with a reminder to rediscover via tmux-hosts.
This commit is contained in:
russell@unturf.com 2026-05-24 16:02:08 -04:00
parent 4ac192585d
commit 72a9553047
No known key found for this signature in database
1 changed files with 86 additions and 96 deletions
Download patch file Download diff file Expand all files Collapse all files

182
CLAUDE.md
View file

@ -39,45 +39,41 @@
**CRITICAL: NEVER use ssh, scp, or rsync to access remote servers.** **CRITICAL: NEVER use ssh, scp, or rsync to access remote servers.**
Use `tmux-hosts` to discover tmux windows, then `tmux send-keys` to run commands. Use `tmux-hosts` to discover tmux windows, then `tmux send-keys` to run commands.
Window numbers shift; never hardcode them. Always discover first.
```bash ```bash
# Discover available hosts
tmux-hosts tmux-hosts
# Output: # example output (yours WILL differ — verify every session):
# 0:0 ai.foxhop.net # 0:0 3090-ai.foxhop.net
# 0:1 3090-ai.foxhop.net # 0:2 ai.foxhop.net # 4090, current speech prod
# Run a command on 3090-ai tmux send-keys -t 0:2 'command here' Enter
tmux send-keys -t 0:1 'command here' Enter tmux capture-pane -t 0:2 -p | tail -20
# Read output
tmux capture-pane -t 0:1 -p | tail -20
``` ```
- **NEVER** use `ssh user@host "command"` -- use `tmux send-keys -t 0:1` - **NEVER** use `ssh user@host "command"` — use `tmux send-keys`
- **NEVER** use `rsync` or `scp` -- use `git push` then `tmux send-keys -t 0:1 'git pull' Enter` - **NEVER** use `rsync` or `scp` — use `git push` + `tmux send-keys '... git pull ...' Enter`
- **ALWAYS** discover our correct window with `tmux-hosts` first - **ALWAYS** rerun `tmux-hosts` to confirm window numbers before sending keys
### 3. Git-Based Deployment ### 3. Git-Based Deployment
**We use git, not rsync/scp.** All code syncs via git push/pull. **We use git, not rsync/scp.** All code syncs via git push/pull.
```bash ```bash
# Deploy workflow: # 1. local: commit and push
# 1. Commit and push locally
git add files && git commit -m "message" && git push git add files && git commit -m "message" && git push
# 2. Pull and rebuild on server via tmux # 2. remote: pull + restart speech.py (see Production Deployment below for the relaunch)
tmux send-keys -t 0:1 'cd ~/git/uncloseai-speech && git pull && sudo docker compose up -d --build' Enter tmux send-keys -t <prod-window> 'cd /mnt/data/git/uncloseai-speech && sudo -u fox git pull' Enter
``` ```
### 4. All Commands Run Locally ### 4. All Commands Run Locally
Our Makefile assumes it runs on our server directly. No remote execution. Our Makefile assumes it runs on our server directly. No remote execution.
When you need to run make targets on our server, use tmux: When you need to run a make target on a remote host, use tmux:
```bash ```bash
tmux send-keys -t 0:1 'cd ~/git/uncloseai-speech && make deploy' Enter tmux send-keys -t <prod-window> 'cd /mnt/data/git/uncloseai-speech && make <target>' Enter
``` ```
### 5. Configuration Management ### 5. Configuration Management
@ -93,119 +89,113 @@ tmux send-keys -t 0:1 'cd ~/git/uncloseai-speech && make deploy' Enter
## Production Deployment ## Production Deployment
**Production Server:** `3090-ai.foxhop.net` (tmux window `0:1`) Prod runs as a **bare python process** on a 4090 host (`ai.foxhop.net`) — no Docker.
- **URL:** https://speech.ai.unturf.com Caddy on 80/443 reverse-proxies https://speech.ai.unturf.com to local port 8000.
- **Repo Location:** `/home/fox/git/uncloseai-speech`
- **Container:** `uncloseai-speech-server-1`
- **Git remote:** `ssh://git@git.unturf.com:2222/engineering/unturf/uncloseai-speech.git`
- **Git user on server:** `timehexon <timehexon@unturf.com>`
### Deployment Workflow | | |
|---|---|
| Host | `ai.foxhop.net` (4090). Find current tmux window with `tmux-hosts`. |
| Run user | `fox` |
| Repo | `/mnt/data/git/uncloseai-speech` (symlinked from `/home/fox/git/uncloseai-speech`) |
| Venv | `/mnt/data/f5-sidecar/venv/` |
| Logs | `/mnt/data/f5-sidecar/logs/speech.log` |
| Launch cmd | `python speech.py --workers 1 --port 8000 --log-level INFO` |
| Frontend | Caddy → `https://speech.ai.unturf.com` |
| Git remote | `ssh://git@git.unturf.com:2222/engineering/unturf/uncloseai-speech.git` |
| Git user on server | `fox` |
``` Docker compose files (`docker-compose.yml`, `Dockerfile`) are kept for parity with
Local: self-hosted deploys, but our prod does NOT use them.
/home/fox/git/uncloseai-speech/
git commit && git push ### Deploy a code/config change
Remote (3090-ai, tmux 0:1):
/home/fox/git/uncloseai-speech/
git pull && sudo docker compose up -d --build
Container:
/app/
├── speech.py
├── cloned-voices/ (mounted from host)
├── voices/ (mounted from host)
└── config/
└── voice_to_speaker.yaml
```
### Quick Production Commands
```bash ```bash
# Discover tmux windows # 1. local
tmux-hosts
# Deploy changes
git push git push
tmux send-keys -t 0:1 'cd ~/git/uncloseai-speech && git pull && sudo docker compose up -d --build' Enter
# Check container status # 2. remote (find window via tmux-hosts, then):
tmux send-keys -t 0:1 'sudo docker ps | grep speech' Enter tmux send-keys -t <4090-window> '\
cd /mnt/data/git/uncloseai-speech && \
sudo -u fox git pull && \
PID=$(sudo ss -tlnp | awk "/:8000 / {print \$NF}" | sed "s/.*pid=\([0-9]*\),.*/\1/") && \
echo "killing PID $PID" && sudo -u fox kill $PID && \
until ! sudo ss -tln | grep -q :8000; do sleep 1; done && \
cd /mnt/data/git/uncloseai-speech && \
sudo -u fox bash -c "nohup env PYTHONUNBUFFERED=1 /mnt/data/f5-sidecar/venv/bin/python speech.py --workers 1 --port 8000 --log-level INFO > /mnt/data/f5-sidecar/logs/speech.log 2>&1 &" && \
until sudo ss -tln | grep -q :8000; do sleep 2; done && echo PORT_BOUND \
' Enter
```
# View logs Audio yaml / `cloned-voices/` changes don't need a code change — same restart picks them up
tmux send-keys -t 0:1 'sudo docker logs --tail 50 uncloseai-speech-server-1' Enter (speech.py loads yaml at startup).
# Restart container ### Quick checks
tmux send-keys -t 0:1 'cd ~/git/uncloseai-speech && sudo docker compose restart' Enter
# Read output from tmux ```bash
tmux capture-pane -t 0:1 -p | tail -20 # is prod up?
curl -sS -o /dev/null -w "%{http_code}\n" https://speech.ai.unturf.com/v1/voices
# what PID is bound to :8000?
tmux send-keys -t <4090-window> 'sudo ss -tlnp | grep :8000' Enter
# tail prod log
tmux send-keys -t <4090-window> 'tail -50 /mnt/data/f5-sidecar/logs/speech.log' Enter
# GPU state
tmux send-keys -t <4090-window> 'nvidia-smi' Enter
``` ```
## Voice Configuration ## Voice Configuration
21 distinct gendered voices from LibriSpeech test-clean (public domain): 40 distinct gendered voices from LibriSpeech test-clean (public domain).
- **Female (11):** aria, clara, elena, grace, hazel, iris, luna, maya, ruby, sage, sofia Roster lives in `cloned-voices/voices_metadata.json`.
- **Male (10):** atlas, caleb, felix, hugo, jasper, kai, leo, marcus, owen, theo
Voice WAV files are in `cloned-voices/`, mounted into our container. Voice WAV files in `cloned-voices/` are read directly by `speech.py` (no container mount —
Config is in `voice_to_speaker.default.yaml`. prod runs bare). Engine→voice mapping in `voice_to_speaker.default.yaml`.
F5-TTS `ref_text` values are generated by `make whisper-refs` (whisper-large-v3 over
the actual wav files) — they match what the audio sounds like, not LibriSpeech ground-truth
labels. Regenerate any time `cloned-voices/` changes.
## TTS Engine Status ## TTS Engine Status
### Default Model (Qwen3-TTS) `tts-1-f5` (F5-TTS) is what prod serves today. `tts-1-qwen` (Qwen3-TTS) is also wired up.
- **Qwen3-TTS (tts-1-qwen)** - DEFAULT - 1.7B params, 10 languages, voice cloning Other engines (Piper, XTTS, Silero, Kokoro, Chatterbox) live in `voice_to_speaker.default.yaml`
but are disabled by default. See `docs/MODELS.md` for the full roadmap.
### Other Engines (disabled by default, enable in voice_to_speaker.yaml)
- Piper TTS (tts-1) - 55 voices, fast CPU inference
- XTTS v2 (tts-1-hd) - Voice cloning, multilingual
- Silero TTS (tts-1-silero) - 142 voices, 5 languages, auto-downloads
- Kokoro TTS (tts-1-kokoro) - 32 voices, lightweight (82M params)
See `docs/MODELS.md` for complete roadmap.
## Testing ## Testing
```bash ```bash
# Run on server via tmux # end-to-end via the prod URL (always-correct)
tmux send-keys -t 0:1 'cd ~/git/uncloseai-speech && make test' Enter curl -sS -X POST https://speech.ai.unturf.com/v1/audio/speech \
# Or test from any machine with curl
curl -X POST http://3090-ai.foxhop.net:8000/v1/audio/speech \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{"model":"tts-1-qwen","voice":"aria","input":"Test"}' \ -d '{"model":"tts-1-f5","voice":"aria","input":"Smoke test."}' \
-o /tmp/test.mp3 -o /tmp/test.mp3 -w "http=%{http_code} size=%{size_download}\n"
# on the prod host directly (via tmux), bypassing Caddy
tmux send-keys -t <4090-window> 'cd /mnt/data/git/uncloseai-speech && make test-f5' Enter
``` ```
## When Things Break ## When Things Break
```bash ```bash
# Check logs # tail prod log
tmux send-keys -t 0:1 'sudo docker logs --tail 50 uncloseai-speech-server-1' Enter tmux send-keys -t <4090-window> 'tail -100 /mnt/data/f5-sidecar/logs/speech.log' Enter
# Check GPU memory # GPU memory
tmux send-keys -t 0:1 'nvidia-smi' Enter tmux send-keys -t <4090-window> 'nvidia-smi' Enter
# Restart container # kill + relaunch (see "Deploy a code/config change" above for the full sequence)
tmux send-keys -t 0:1 'cd ~/git/uncloseai-speech && sudo docker compose restart' Enter
# Full rebuild
tmux send-keys -t 0:1 'cd ~/git/uncloseai-speech && git pull && sudo docker compose up -d --build' Enter
# Read tmux output
tmux capture-pane -t 0:1 -p | tail -20
``` ```
## Common Mistakes to Avoid ## Common Mistakes to Avoid
- DON'T use `ssh user@host "command"` -- use `tmux send-keys -t 0:1` - DON'T use `ssh user@host "command"` — use `tmux send-keys`
- DON'T use `rsync` or `scp` -- use `git push` + `tmux send-keys -t 0:1 'git pull' Enter` - DON'T use `rsync` or `scp` — use `git push` + tmux `git pull`
- DON'T hardcode tmux window numbers -- use `tmux-hosts` to discover them - DON'T hardcode tmux window numbers — rediscover via `tmux-hosts` every session
- DON'T put scripts in docs/ -- put them in scripts/ - DON'T put scripts in docs/ — put them in scripts/
- DON'T forget to push before deploying - DON'T forget to push before deploying
- DON'T assume prod uses docker — it's bare python; the compose files are for self-hosted parity only
## AGPL v3 Compliance ## AGPL v3 Compliance