uncloseai-speech/docs/VOICES.md

Voice Corpus Documentation

Overview

uncloseai-speech uses voice samples from LibriSpeech test-clean for Qwen3-TTS voice cloning. LibriSpeech is a public domain corpus of read English speech from LibriVox audiobook recordings.

• Source: OpenSLR LibriSpeech
• HuggingFace: openslr/librispeech_asr
• License: Public domain (LibriVox recordings)
• Audio: 16kHz WAV, single speaker per file

Voice Registry System

Voice-to-speaker assignments are managed by voice_registry.json -- an append-only, idempotent registry that permanently locks each speaker to a voice name.

How It Works

1. Registry file (voice_registry.json): Committed to git, contains all locked assignments
2. Name pools: 50 female + 50 male names, assigned in order as new speakers are added
3. Deterministic assignment: New speakers sorted by ID (ascending), names assigned in pool order
4. Append-only: Once a speaker is assigned a name, that assignment never changes
5. Multi-corpus: Registry tracks which corpus each speaker came from

Adding a New Corpus

To add speakers from a new corpus:

1. Add the corpus config to voice_registry.json under corpora:

   "corpora": {
     "librispeech-test-clean": { ... },
     "librispeech-dev-clean": {
       "dataset": "openslr/librispeech_asr",
       "config": "clean",
       "split": "validation",
       "description": "LibriSpeech dev-clean, 40 speakers"
     }
   }
   

2. Run the download script with the new corpus:

   python scripts/download_diverse_voices.py --corpora librispeech-test-clean librispeech-dev-clean
   

3. New speakers get the next available names from the pool. Existing assignments are untouched.

4. Commit the updated voice_registry.json to lock the new assignments.

Test-Clean Corpus

The test-clean split contains 40 speakers (~8 minutes each, ~350MB total):

Female Speakers (20)

ID	LibriVox Name	Minutes
121	Nikolle Doolin	8.01
237	rachelellen	8.02
367	Kathleen Dang	6.74
1221	Dianne	8.07
1284	Daniel Anaya	8.16
1580	TinyPines	8.07
1995	(unknown)	~8
2094	amycsj	8.09
2961	Leni	8.07
3570	sarac	8.05
3575	supergirl	8.06
3729	Heather Hogan	8.03
4446	Jen Maxwell	8.00
4507	Rachel Nelson-Smith	8.05
4970	airandwaters	8.15
4992	Joyce Martin	8.21
5142	Mary Ballard-Johansson	8.07
5570	Ulf Bjorklund	8.28
5683	Rachael Lapidis	8.01
6829	LadyBug	8.24

Male Speakers (20)

ID	LibriVox Name	Minutes
61	Paul-Gabriel Wiener	8.08
260	Brad Bush	8.05
672	Taylor Burton-Edward	8.27
908	Sam Stinson	8.05
1089	Peter Bobbe	8.05
1188	Duncan Murrell	8.20
1320	number6	8.02
2300	Mitchell L Leopard	8.19
2830	Tim Perkins	8.04
4077	Nathan Markham	8.14
5105	elongman	8.12
5639	(unknown)	~8
6930	Nolan Fout	8.00
7021	(unknown)	~8
7127	(unknown)	~8
7176	(unknown)	~8
7729	(unknown)	~8
8230	(unknown)	~8
8455	(unknown)	~8
8463	(unknown)	~8

Note: Names marked (unknown) were not in the SPEAKERS.TXT mirror we fetched.

Voice Names

These are our own names -- LibriSpeech only provides LibriVox usernames (like "supergirl", "LadyBug", "number6"), not character-style voice names.

Names are assigned from pools in voice_registry.json and locked permanently. The first 21 voices (original set) are:

Female voices: aria, clara, elena, grace, hazel, iris, luna, maya, ruby, sage, sofia

Male voices: atlas, caleb, felix, hugo, jasper, kai, leo, marcus, owen, theo

When all 40 test-clean speakers are registered, the remaining 19 get the next names from the pools (amber, brooke, cora, ... for female; archer, blake, cole, ... for male).

How Voices Are Selected

The script picks the best sample per speaker based on:

• Duration: prefers 5-10 seconds (ideal for voice cloning reference)
• Completeness: bonus for sentences ending with a period
• Length: penalty for very long text (>300 chars)

Gender Verification

Speaker genders are fetched at runtime from the official LibriSpeech SPEAKERS.TXT (via GitHub mirror). This ensures female names always map to female speakers and male names always map to male speakers. No hardcoded gender assumptions.

Source: https://raw.githubusercontent.com/oscarknagg/voicemap/master/data/LibriSpeech/SPEAKERS.TXT

Other LibriSpeech Splits

Additional splits can be added as new corpora in the registry:

• train-clean-100: ~250 speakers, 100 hours
• train-clean-360: ~920 speakers, 360 hours
• train-other-500: ~1160 speakers, 500 hours (noisier)
• test-other: 33 speakers (noisier conditions)
• dev-clean: 40 speakers (validation set)
• dev-other: 33 speakers

Using larger splits would give hundreds or thousands of distinct voices, but test-clean provides the highest quality recordings.

File Layout

voice_registry.json           # Idempotent voice-to-speaker assignments (committed)
cloned-voices/
├── aria.wav                  # Female voice sample (~7s, 16kHz)
├── atlas.wav                 # Male voice sample
├── ...                       # (40 voices when fully expanded)
└── voices_metadata.json      # Speaker IDs, genders, transcripts, durations

The cloned-voices/ directory is mounted into the Docker container at /app/cloned-voices/. Voice config is in voice_to_speaker.default.yaml.