Simplify CLAUDE.md: remove SSH/rsync references, local-only workflow
This commit is contained in:
parent
c3ac2def48
commit
3f3918fed2
1 changed files with 128 additions and 253 deletions
381
CLAUDE.md
381
CLAUDE.md
|
|
@ -8,12 +8,12 @@
|
|||
**CRITICAL: Always use consistent naming across all files.**
|
||||
|
||||
### Project Name
|
||||
- ✅ **Correct:** `uncloseai-speech` (lowercase, hyphenated)
|
||||
- ❌ **Wrong:** "UncloseAI Speech", "Uncloseai Speech", "UncloseAI-Speech"
|
||||
- **Correct:** `uncloseai-speech` (lowercase, hyphenated)
|
||||
- **Wrong:** "UncloseAI Speech", "Uncloseai Speech", "UncloseAI-Speech"
|
||||
|
||||
### Organization Name
|
||||
- ✅ **Correct:** `uncloseai` (lowercase, one word)
|
||||
- ❌ **Wrong:** "UncloseAI", "Unclose AI", "UnClose AI"
|
||||
- **Correct:** `uncloseai` (lowercase, one word)
|
||||
- **Wrong:** "UncloseAI", "Unclose AI", "UnClose AI"
|
||||
|
||||
### Usage Guidelines
|
||||
- **In code:** Use `uncloseai-speech` for project references
|
||||
|
|
@ -23,204 +23,101 @@
|
|||
- **Docker images:** `uncloseai-speech` (lowercase, hyphenated)
|
||||
- **API responses:** Use `"owned_by": "uncloseai"` (lowercase, one word)
|
||||
|
||||
### Examples
|
||||
```python
|
||||
# Correct
|
||||
description='uncloseai-speech API Server'
|
||||
owned_by = "uncloseai"
|
||||
|
||||
# Wrong
|
||||
description='UncloseAI Speech API Server'
|
||||
owned_by = "UncloseAI"
|
||||
```
|
||||
|
||||
```markdown
|
||||
# Correct
|
||||
# uncloseai-speech
|
||||
|
||||
**Raccoon Mission:** Rescue abandoned TTS models and integrate them into uncloseai-speech
|
||||
|
||||
# Wrong
|
||||
# UncloseAI Speech
|
||||
|
||||
**Raccoon Mission:** Rescue abandoned TTS models and integrate them into UncloseAI Speech
|
||||
```
|
||||
|
||||
## Core Principles
|
||||
|
||||
### 1. Makefile-First Development
|
||||
|
||||
**ALWAYS prefer Makefile targets over manual commands.**
|
||||
|
||||
- ✅ DO: `make deploy`, `make voices`, `make test`
|
||||
- ❌ DON'T: Manual ssh commands, docker commands, curl commands
|
||||
|
||||
**When adding new functionality:**
|
||||
1. Add it to the Makefile first
|
||||
2. Document it in `make help`
|
||||
3. Test it works from scratch
|
||||
4. Only then modify other files if needed
|
||||
- DO: `make deploy`, `make voices`, `make test`
|
||||
- DON'T: Manual docker commands, curl commands
|
||||
|
||||
**Makefile is the source of truth** for all deployment and development tasks.
|
||||
|
||||
### 2. Work Locally, Deploy Remotely
|
||||
### 2. Remote Access: tmux-hosts and tmux ONLY
|
||||
|
||||
- **Local development:** `/home/fox/git/uncloseai-speech/`
|
||||
- **Remote server:** `ai.foxhop.net` (configured in `vars.sh`, gitignored)
|
||||
- **Never create remote directories manually** - let Makefile handle it
|
||||
- **Always test from scratch** - `make clean` then `make deploy`
|
||||
**CRITICAL: NEVER use ssh, scp, or rsync to access remote servers.**
|
||||
|
||||
### 3. Configuration Management
|
||||
Use `tmux-hosts` to discover tmux windows, then `tmux send-keys` to run commands.
|
||||
|
||||
```bash
|
||||
# Discover available hosts
|
||||
tmux-hosts
|
||||
# Output:
|
||||
# 0:0 ai.foxhop.net
|
||||
# 0:1 3090-ai.foxhop.net
|
||||
|
||||
# Run a command on 3090-ai
|
||||
tmux send-keys -t 0:1 'command here' Enter
|
||||
|
||||
# Read output
|
||||
tmux capture-pane -t 0:1 -p | tail -20
|
||||
```
|
||||
|
||||
- **NEVER** use `ssh user@host "command"` -- use `tmux send-keys -t 0:1`
|
||||
- **NEVER** use `rsync` or `scp` -- use `git push` then `tmux send-keys -t 0:1 'git pull' Enter`
|
||||
- **ALWAYS** discover the correct window with `tmux-hosts` first
|
||||
|
||||
### 3. Git-Based Deployment
|
||||
|
||||
**We use git, not rsync/scp.** All code syncs via git push/pull.
|
||||
|
||||
```bash
|
||||
# Deploy workflow:
|
||||
# 1. Commit and push locally
|
||||
git add files && git commit -m "message" && git push
|
||||
|
||||
# 2. Pull and rebuild on server via tmux
|
||||
tmux send-keys -t 0:1 'cd ~/git/uncloseai-speech && git pull && sudo docker compose up -d --build' Enter
|
||||
```
|
||||
|
||||
### 4. All Commands Run Locally
|
||||
|
||||
The Makefile assumes it runs on the server directly. No remote execution.
|
||||
When you need to run make targets on the server, use tmux:
|
||||
|
||||
```bash
|
||||
tmux send-keys -t 0:1 'cd ~/git/uncloseai-speech && make deploy' Enter
|
||||
```
|
||||
|
||||
### 5. Configuration Management
|
||||
|
||||
- `vars.sh` - Deployment secrets (gitignored, never commit)
|
||||
- `vars.sh.example` - Template for users (commit this)
|
||||
- `sample.env` - Default environment (commit this)
|
||||
- `speech.env` - Runtime environment (created automatically by Makefile)
|
||||
- `speech.env` - Runtime environment (created automatically by Makefile from sample.env)
|
||||
|
||||
**Key environment variables:**
|
||||
- `WORKERS` - Number of uvicorn workers (default: 1 for GPU models like Qwen3-TTS)
|
||||
- Use 1 for GPU-bound models to avoid VRAM duplication across workers
|
||||
- Increase for CPU-bound models like Piper (e.g., WORKERS=4)
|
||||
|
||||
**Never view or log secrets** - source them and use them.
|
||||
|
||||
### 4. Git Commit Guidelines
|
||||
### 6. Git Commit Guidelines
|
||||
|
||||
- **Never add AI attribution** - Do not use `Co-Authored-By: Claude` or similar in commit messages
|
||||
- Write clear, concise commit messages describing what changed and why
|
||||
- Use imperative mood ("Add feature" not "Added feature")
|
||||
|
||||
### 5. Documentation Requirements
|
||||
|
||||
When adding features, update ALL relevant docs:
|
||||
- `Makefile` help text
|
||||
- `docs/MODELS.md` for new TTS engines
|
||||
- `docs/MIRRORS.md` for binary downloads
|
||||
- `docs/AUDIT.md` for file changes
|
||||
- This file (`docs/CLAUDE.md`) for new patterns
|
||||
|
||||
## Common Tasks
|
||||
|
||||
### Full Deployment from Scratch
|
||||
|
||||
```bash
|
||||
# 1. Clean everything
|
||||
make clean
|
||||
|
||||
# 2. Deploy (commit, push, pull on server, rebuild container)
|
||||
git push && tmux send-keys -t 0:1 'cd ~/uncloseai-speech && git pull && sudo docker-compose up --build -d' Enter
|
||||
|
||||
# 3. Download voices (Piper + XTTS samples)
|
||||
make voices
|
||||
|
||||
# 4. Test
|
||||
make test
|
||||
make test-xtts
|
||||
```
|
||||
|
||||
### Adding a New TTS Engine
|
||||
|
||||
1. Document it in `docs/MODELS.md` first
|
||||
2. Add download target to Makefile (e.g., `voices-silero`)
|
||||
3. Implement engine wrapper in `speech.py` or `src/engines/`
|
||||
4. Add test target (e.g., `test-silero`)
|
||||
5. Update `make voices` to include it
|
||||
6. Test full cycle: commit, push, pull on server, rebuild, test
|
||||
|
||||
### Debugging Issues
|
||||
|
||||
```bash
|
||||
make logs # Tail live logs
|
||||
make logs | grep ERROR # Filter errors
|
||||
```
|
||||
|
||||
Never use raw docker/ssh commands - extend Makefile if needed.
|
||||
|
||||
## File Organization
|
||||
|
||||
### Scripts vs Docs
|
||||
|
||||
- `scripts/` - Executable utilities (add_voice.py, download_samples.sh, etc.)
|
||||
- `docs/` - Documentation ONLY (no executable code)
|
||||
- Dockerfiles, startup.sh - Root level (build artifacts)
|
||||
- Makefile - Root level (primary interface)
|
||||
|
||||
**Never put executable scripts in docs/ directory.**
|
||||
|
||||
### Current Structure (as of 2025-11-09)
|
||||
|
||||
```
|
||||
uncloseai-speech/
|
||||
├── Makefile # PRIMARY INTERFACE - always update first
|
||||
├── vars.sh # Secrets (gitignored)
|
||||
├── vars.sh.example # Template
|
||||
├── speech.py # Main server (will refactor to src/)
|
||||
├── openedai.py # API models
|
||||
├── voice_to_speaker.default.yaml # Voice config
|
||||
├── docs/
|
||||
│ ├── CLAUDE.md # This file
|
||||
│ ├── AUDIT.md # Repository audit
|
||||
│ ├── MODELS.md # TTS engines
|
||||
│ └── MIRRORS.md # Binary mirror strategy
|
||||
├── scripts/
|
||||
│ ├── add_voice.py
|
||||
│ ├── say.py
|
||||
│ ├── test_voices.sh
|
||||
│ └── download_samples.sh
|
||||
├── Dockerfile
|
||||
├── docker-compose.yml
|
||||
└── startup.sh
|
||||
```
|
||||
|
||||
## TTS Engine Status
|
||||
|
||||
### Default Model (Qwen3-TTS)
|
||||
- ✅ **Qwen3-TTS (tts-1-qwen)** - DEFAULT - 1.7B params, 10 languages, voice cloning, 97ms latency
|
||||
|
||||
### Other Engines (disabled by default, enable in voice_to_speaker.yaml)
|
||||
- Piper TTS (tts-1) - 55 voices, fast CPU inference
|
||||
- XTTS v2 (tts-1-hd) - Voice cloning, multilingual
|
||||
- Silero TTS (tts-1-silero) - 142 voices, 5 languages, auto-downloads
|
||||
- Kokoro TTS (tts-1-kokoro) - 32 voices, lightweight (82M params)
|
||||
|
||||
### Why Qwen3-TTS is Default
|
||||
- State-of-the-art quality with voice cloning
|
||||
- Actively maintained by Alibaba
|
||||
- Apache 2.0 license (commercial-friendly)
|
||||
- 10 languages: Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, Italian
|
||||
- Fast first-packet latency (97ms)
|
||||
- Easy voice cloning from 3-second samples
|
||||
|
||||
See `docs/MODELS.md` for complete roadmap and detailed model documentation.
|
||||
|
||||
## Production Deployment
|
||||
|
||||
**Production Server:** `3090-ai.foxhop.net`
|
||||
- **URL:** https://speech.ai.unturf.com (proxied via ai.foxhop.net)
|
||||
- **Repo Location:** `/home/fox/uncloseai-speech`
|
||||
- **Container:** `uncloseai-speech_server_1` (image: `uncloseai-speech:local`)
|
||||
- **tmux access:** Use `tmux-hosts` to discover window mappings, then `tmux send-keys -t 0:1 'command' Enter`
|
||||
- **Current mapping:** window `0:1` is `3090-ai.foxhop.net` (verify with `tmux-hosts`)
|
||||
**Production Server:** `3090-ai.foxhop.net` (tmux window `0:1`)
|
||||
- **URL:** https://speech.ai.unturf.com
|
||||
- **Repo Location:** `/home/fox/git/uncloseai-speech`
|
||||
- **Container:** `uncloseai-speech-server-1`
|
||||
- **Git remote:** `ssh://git@git.unturf.com:2222/engineering/unturf/uncloseai-speech.git`
|
||||
- **Git user on server:** `timehexon <timehexon@unturf.com>`
|
||||
|
||||
### Deployment Workflow
|
||||
|
||||
**We use git, not rsync.** Commit locally, push, pull on server, rebuild.
|
||||
|
||||
```
|
||||
Local:
|
||||
/home/fox/git/uncloseai-speech/
|
||||
|
||||
↓ git commit && git push
|
||||
git commit && git push
|
||||
|
||||
Remote (3090-ai.foxhop.net):
|
||||
/home/fox/uncloseai-speech/
|
||||
Remote (3090-ai, tmux 0:1):
|
||||
/home/fox/git/uncloseai-speech/
|
||||
|
||||
↓ git pull && sudo docker-compose up --build -d
|
||||
git pull && sudo docker compose up -d --build
|
||||
|
||||
Container:
|
||||
/app/
|
||||
├── speech.py
|
||||
├── voices/samples/ (cloned-voices mounted)
|
||||
├── cloned-voices/ (mounted from host)
|
||||
├── voices/ (mounted from host)
|
||||
└── config/
|
||||
└── voice_to_speaker.yaml
|
||||
```
|
||||
|
|
@ -228,122 +125,100 @@ Container:
|
|||
### Quick Production Commands
|
||||
|
||||
```bash
|
||||
# Use tmux-hosts to see available tmux windows
|
||||
# Discover tmux windows
|
||||
tmux-hosts
|
||||
|
||||
# Push changes and deploy
|
||||
git add -A && git commit -m "message" && git push
|
||||
tmux send-keys -t 0:1 'cd ~/uncloseai-speech && git pull && sudo docker-compose up --build -d' Enter
|
||||
# Deploy changes
|
||||
git push
|
||||
tmux send-keys -t 0:1 'cd ~/git/uncloseai-speech && git pull && sudo docker compose up -d --build' Enter
|
||||
|
||||
# Check container status
|
||||
tmux send-keys -t 0:1 'sudo docker ps | grep speech' Enter
|
||||
|
||||
# View logs
|
||||
tmux send-keys -t 0:1 'sudo docker logs --tail 50 uncloseai-speech_server_1' Enter
|
||||
tmux send-keys -t 0:1 'sudo docker logs --tail 50 uncloseai-speech-server-1' Enter
|
||||
|
||||
# Restart container
|
||||
tmux send-keys -t 0:1 'cd ~/uncloseai-speech && sudo docker-compose restart' Enter
|
||||
tmux send-keys -t 0:1 'cd ~/git/uncloseai-speech && sudo docker compose restart' Enter
|
||||
|
||||
# Read output from tmux
|
||||
tmux capture-pane -t 0:1 -p | tail -20
|
||||
```
|
||||
|
||||
## Testing Philosophy
|
||||
## Voice Configuration
|
||||
|
||||
**Full stack testing workflow:**
|
||||
1. Commit and push changes
|
||||
2. Pull on server and rebuild: `git pull && sudo docker-compose up --build -d`
|
||||
3. Wait for model to load (check logs)
|
||||
4. Test via curl or browser
|
||||
21 distinct gendered voices from LibriSpeech test-clean (public domain):
|
||||
- **Female (11):** aria, clara, elena, grace, hazel, iris, luna, maya, ruby, sage, sofia
|
||||
- **Male (10):** atlas, caleb, felix, hugo, jasper, kai, leo, marcus, owen, theo
|
||||
|
||||
**Never assume** - always test from scratch after changes.
|
||||
Voice WAV files are in `cloned-voices/`, mounted into the container.
|
||||
Config is in `voice_to_speaker.default.yaml`.
|
||||
|
||||
## Raccoon Mission Values
|
||||
## TTS Engine Status
|
||||
|
||||
1. **Resilience** - Assume upstream dies, plan mirrors
|
||||
2. **Simplicity** - Makefile > manual commands
|
||||
3. **Documentation** - Write docs before code
|
||||
4. **Liberation** - Keep TTS libre (AGPL v3)
|
||||
5. **Unification** - All TTS engines, one API
|
||||
### Default Model (Qwen3-TTS)
|
||||
- **Qwen3-TTS (tts-1-qwen)** - DEFAULT - 1.7B params, 10 languages, voice cloning
|
||||
|
||||
## AGPL v3 Compliance
|
||||
### Other Engines (disabled by default, enable in voice_to_speaker.yaml)
|
||||
- Piper TTS (tts-1) - 55 voices, fast CPU inference
|
||||
- XTTS v2 (tts-1-hd) - Voice cloning, multilingual
|
||||
- Silero TTS (tts-1-silero) - 142 voices, 5 languages, auto-downloads
|
||||
- Kokoro TTS (tts-1-kokoro) - 32 voices, lightweight (82M params)
|
||||
|
||||
**This project is AGPL v3 licensed.** The key obligation: anyone who uses this TTS service over a network must be able to access the source code.
|
||||
See `docs/MODELS.md` for complete roadmap.
|
||||
|
||||
### What This Means
|
||||
## Testing
|
||||
|
||||
Unlike regular GPL, AGPL closes the "SaaS loophole". If you run uncloseai-speech as a service (even without distributing binaries), users have the right to request source code.
|
||||
```bash
|
||||
# Run on server via tmux
|
||||
tmux send-keys -t 0:1 'cd ~/git/uncloseai-speech && make test' Enter
|
||||
|
||||
### Requirements for Operators
|
||||
|
||||
When running uncloseai-speech as a network service, you must provide:
|
||||
- Complete source code of the running version
|
||||
- Any modifications you've made
|
||||
- Build instructions
|
||||
|
||||
### How to Comply
|
||||
|
||||
1. **Link in API response** - Add source URL to `/v1/models` or root endpoint
|
||||
2. **Link in documentation** - Include repository URL in service docs
|
||||
3. **Host source code** - Keep your fork in a public git repository
|
||||
|
||||
### Example Implementation
|
||||
|
||||
```python
|
||||
# In API responses
|
||||
"source_code": "https://github.com/uncloseai/uncloseai-speech"
|
||||
# Or test from any machine with curl
|
||||
curl -X POST http://3090-ai.foxhop.net:8000/v1/audio/speech \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model":"tts-1-qwen","voice":"aria","input":"Test"}' \
|
||||
-o /tmp/test.mp3
|
||||
```
|
||||
|
||||
### Why AGPL?
|
||||
|
||||
- Ensures forks remain open source
|
||||
- Community improvements flow back to the project
|
||||
- Prevents proprietary TTS services from using our work without sharing back
|
||||
- Aligns with Raccoon Mission: **Keep TTS libre**
|
||||
|
||||
## Common Mistakes to Avoid
|
||||
|
||||
❌ DON'T create directories with raw ssh
|
||||
✅ DO add Makefile target for deployment
|
||||
|
||||
❌ DON'T use rsync - we use git
|
||||
✅ DO commit, push, pull on server, then rebuild with docker-compose
|
||||
|
||||
❌ DON'T put scripts in docs/
|
||||
✅ DO put scripts in scripts/, reference from docs
|
||||
|
||||
❌ DON'T hardcode paths/hosts
|
||||
✅ DO use vars.sh variables
|
||||
|
||||
❌ DON'T forget to commit and push before deploying
|
||||
✅ DO commit, push, pull on server, rebuild container
|
||||
|
||||
## When Things Break
|
||||
|
||||
1. Check logs: `tmux send-keys -t 0:1 'sudo docker logs --tail 50 uncloseai-speech_server_1' Enter`
|
||||
2. Check GPU memory: `tmux send-keys -t 0:1 'nvidia-smi' Enter`
|
||||
3. Restart container: `tmux send-keys -t 0:1 'sudo docker-compose restart' Enter`
|
||||
4. Rebuild: `git pull && sudo docker-compose up --build -d`
|
||||
5. Check voice files: `tmux send-keys -t 0:1 'ls voices/samples/' Enter`
|
||||
```bash
|
||||
# Check logs
|
||||
tmux send-keys -t 0:1 'sudo docker logs --tail 50 uncloseai-speech-server-1' Enter
|
||||
|
||||
## Multiprocess Architecture (uvicorn workers=4)
|
||||
# Check GPU memory
|
||||
tmux send-keys -t 0:1 'nvidia-smi' Enter
|
||||
|
||||
**Key pattern:** Worker processes spawn as fresh imports, don't run `__main__` block.
|
||||
# Restart container
|
||||
tmux send-keys -t 0:1 'cd ~/git/uncloseai-speech && sudo docker compose restart' Enter
|
||||
|
||||
**Solutions implemented:**
|
||||
# Full rebuild
|
||||
tmux send-keys -t 0:1 'cd ~/git/uncloseai-speech && git pull && sudo docker compose up -d --build' Enter
|
||||
|
||||
# Read tmux output
|
||||
tmux capture-pane -t 0:1 -p | tail -20
|
||||
```
|
||||
|
||||
## Common Mistakes to Avoid
|
||||
|
||||
- DON'T use `ssh user@host "command"` -- use `tmux send-keys -t 0:1`
|
||||
- DON'T use `rsync` or `scp` -- use `git push` + `tmux send-keys -t 0:1 'git pull' Enter`
|
||||
- DON'T hardcode tmux window numbers -- use `tmux-hosts` to discover them
|
||||
- DON'T put scripts in docs/ -- put them in scripts/
|
||||
- DON'T forget to push before deploying
|
||||
|
||||
## AGPL v3 Compliance
|
||||
|
||||
**This project is AGPL v3 licensed.** Anyone using this TTS service over a network must be able to access the source code.
|
||||
|
||||
## Multiprocess Architecture
|
||||
|
||||
Worker processes spawn as fresh imports, don't run `__main__` block.
|
||||
1. **Caches** - Initialize in `lifespan` context manager (runs per worker)
|
||||
2. **Args** - Use `DefaultArgs` class at module level, override in `__main__`
|
||||
|
||||
See `speech.py:23-113` for implementation.
|
||||
|
||||
## Future Refactoring (Planned)
|
||||
|
||||
- Move `speech.py`, `openedai.py`, `audio_reader.py` → `src/`
|
||||
- Create engine abstraction layer in `src/engines/`
|
||||
- Unified voice config with engine selection
|
||||
- Binary mirror implementation (MinIO on ai.foxhop.net)
|
||||
|
||||
See `docs/AUDIT.md` for detailed refactoring plan.
|
||||
|
||||
---
|
||||
|
||||
**Remember:** Makefile first, documentation second, code third. Test from scratch every time.
|
||||
|
||||
🦝 **Raccoon Mission:** Keep TTS libre, rescue abandoned models, unify all engines.
|
||||
**Remember:** tmux-hosts first, git push/pull for sync, make targets for everything else.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue