Commit graph

2 commits

Author SHA1 Message Date
c5247cd993
whisper_refs: normalize transcripts (sentence-case, strip quote artifacts)
normalize_text() handles three Whisper quirks that produced messy F5
ref_texts:

  1. Lowercase output with no terminal period — capitalize first letter,
     append "." if missing.
  2. Hallucinated "' clusters Whisper inserts when it interprets a
     fragment as quoted dialogue (cora, ivan, atlas, hope had these).
     Strip everywhere; never legitimate English punctuation.
  3. Trailing apostrophe-then-period (.'.) from earlier rounds where a
     closing-quoted line got an extra "." appended — collapse to single
     terminal. Function is now idempotent.

Adds --from-cache flag: skip ASR, re-apply normalize from cached
whisper_refs.json. No GPU needed, useful after tuning the normalizer.
Lazy-imports torch so --from-cache works on any host.

Affects 9 of 40 voices: clara, grace, hazel, iris, felix, hugo
(lowercase fix); cora, ivan, atlas, hope (quote-cluster fix).
2026-05-24 16:02:18 -04:00
f25731ca08
Add make whisper-refs — re-transcribe cloned-voices/*.wav with whisper-large-v3
F5-TTS cloning quality depends on ref_text matching the prosody of ref_audio
(commas, periods, casing). Previous ref_texts were LibriSpeech ground-truth
labels: ALL CAPS, no punctuation — wrong signal for a flow-matching TTS
conditioned on text. Whisper hears what F5 will hear.

- scripts/whisper_refs.py — transcribe all wavs, rewrite
  voice_to_speaker.default.yaml + cloned-voices/voices_metadata.json
  in place. Also writes cloned-voices/whisper_refs.json sidecar.
- Makefile: whisper-refs target. Idempotent, rerun whenever
  cloned-voices/ changes.

Run on a GPU host (4090/3090). ~30s for 40 short clips on a 4090.
2026-05-24 11:56:15 -04:00