engineering/uncloseai-speech
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add make whisper-refs — re-transcribe cloned-voices/*.wav with whisper-large-v3

Browse source 
F5-TTS cloning quality depends on ref_text matching the prosody of ref_audio
(commas, periods, casing). Previous ref_texts were LibriSpeech ground-truth
labels: ALL CAPS, no punctuation — wrong signal for a flow-matching TTS
conditioned on text. Whisper hears what F5 will hear.

- scripts/whisper_refs.py — transcribe all wavs, rewrite
  voice_to_speaker.default.yaml + cloned-voices/voices_metadata.json
  in place. Also writes cloned-voices/whisper_refs.json sidecar.
- Makefile: whisper-refs target. Idempotent, rerun whenever
  cloned-voices/ changes.

Run on a GPU host (4090/3090). ~30s for 40 short clips on a 4090.
This commit is contained in:
russell@unturf.com 2026-05-24 11:56:15 -04:00
parent 3a76ba6748
commit f25731ca08
No known key found for this signature in database
2 changed files with 152 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

12
Makefile
View file

@ -3,7 +3,7 @@
CONTAINER_NAME ?= uncloseai-speech-server-1
.PHONY: help deploy restart logs test clean stop start voices voices-qwen voices-f5 voices-piper voices-xtts voices-kokoro test-kokoro voices-silero test-silero voices-chatterbox test-chatterbox push-all hydrate load-test test-qwen test-f5 venv venv-run local local-cpu
.PHONY: help deploy restart logs test clean stop start voices voices-qwen voices-f5 voices-piper voices-xtts voices-kokoro test-kokoro voices-silero test-silero voices-chatterbox test-chatterbox push-all hydrate load-test test-qwen test-f5 venv venv-run local local-cpu whisper-refs
help:
@echo "Raccoon TTS Mission - Development Commands"
@ -31,6 +31,7 @@ help:
@echo "Voices:"
@echo " make voices-qwen - Download Qwen3-TTS cloned voice samples"
@echo " make voices-f5 - Prepare F5-TTS voices (reuses Qwen samples)"
@echo " make whisper-refs - Re-transcribe cloned-voices/*.wav with whisper-large-v3 (GPU)"
@echo " make voices-piper - Download Piper voices"
@echo " make voices-xtts - Download XTTS voices"
@echo " make voices-kokoro - Download Kokoro models"
@ -186,6 +187,15 @@ voices-f5: voices-qwen
@echo "F5-TTS reuses the same cloned-voices/ samples as Qwen3-TTS"
@echo " Model (~1.5GB: F5-TTS_v1 + Vocos) downloads automatically on first use"
whisper-refs:
@echo "Transcribing cloned-voices/*.wav with whisper-large-v3 (GPU)..."
@if [ -d ".venv" ]; then \
.venv/bin/python scripts/whisper_refs.py; \
else \
python3 scripts/whisper_refs.py; \
fi
@echo "Done. Review diff: git diff voice_to_speaker.default.yaml cloned-voices/voices_metadata.json"
voices-all: voices-qwen voices-f5 voices-piper voices-xtts voices-silero
@echo "All voices downloaded!"