uncloseai-speech/docs/VOICES.md

# Voice Corpus Documentation

## Overview

uncloseai-speech uses voice samples from **LibriSpeech test-clean** for Qwen3-TTS voice cloning.
LibriSpeech is a public domain corpus of read English speech from LibriVox audiobook recordings.

- **Source:** [OpenSLR LibriSpeech](https://www.openslr.org/12)
- **HuggingFace:** [openslr/librispeech_asr](https://huggingface.co/datasets/openslr/librispeech_asr)
- **License:** Public domain (LibriVox recordings)
- **Audio:** 16kHz WAV, single speaker per file

## Voice Registry System

Voice-to-speaker assignments are managed by `voice_registry.json` -- an append-only,
idempotent registry that permanently locks each speaker to a voice name.

### How It Works

1. **Registry file** (`voice_registry.json`): Committed to git, contains all locked assignments
2. **Name pools**: 50 female + 50 male names, assigned in order as new speakers are added
3. **Deterministic assignment**: New speakers sorted by ID (ascending), names assigned in pool order
4. **Append-only**: Once a speaker is assigned a name, that assignment never changes
5. **Multi-corpus**: Registry tracks which corpus each speaker came from

### Adding a New Corpus

To add speakers from a new corpus:

1. Add the corpus config to `voice_registry.json` under `corpora`:
   ```json
   "corpora": {
     "librispeech-test-clean": { ... },
     "librispeech-dev-clean": {
       "dataset": "openslr/librispeech_asr",
       "config": "clean",
       "split": "validation",
       "description": "LibriSpeech dev-clean, 40 speakers"
     }
   }
   ```

2. Run the download script with the new corpus:
   ```bash
   python scripts/download_diverse_voices.py --corpora librispeech-test-clean librispeech-dev-clean
   ```

3. New speakers get the next available names from the pool. Existing assignments are untouched.

4. Commit the updated `voice_registry.json` to lock the new assignments.

## Test-Clean Corpus

The `test-clean` split contains **40 speakers** (~8 minutes each, ~350MB total):

### Female Speakers (20)

| ID | LibriVox Name | Minutes |
|----|---------------|---------|
| 121 | Nikolle Doolin | 8.01 |
| 237 | rachelellen | 8.02 |
| 367 | Kathleen Dang | 6.74 |
| 1221 | Dianne | 8.07 |
| 1284 | Daniel Anaya | 8.16 |
| 1580 | TinyPines | 8.07 |
| 1995 | (unknown) | ~8 |
| 2094 | amycsj | 8.09 |
| 2961 | Leni | 8.07 |
| 3570 | sarac | 8.05 |
| 3575 | supergirl | 8.06 |
| 3729 | Heather Hogan | 8.03 |
| 4446 | Jen Maxwell | 8.00 |
| 4507 | Rachel Nelson-Smith | 8.05 |
| 4970 | airandwaters | 8.15 |
| 4992 | Joyce Martin | 8.21 |
| 5142 | Mary Ballard-Johansson | 8.07 |
| 5570 | Ulf Bjorklund | 8.28 |
| 5683 | Rachael Lapidis | 8.01 |
| 6829 | LadyBug | 8.24 |

### Male Speakers (20)

| ID | LibriVox Name | Minutes |
|----|---------------|---------|
| 61 | Paul-Gabriel Wiener | 8.08 |
| 260 | Brad Bush | 8.05 |
| 672 | Taylor Burton-Edward | 8.27 |
| 908 | Sam Stinson | 8.05 |
| 1089 | Peter Bobbe | 8.05 |
| 1188 | Duncan Murrell | 8.20 |
| 1320 | number6 | 8.02 |
| 2300 | Mitchell L Leopard | 8.19 |
| 2830 | Tim Perkins | 8.04 |
| 4077 | Nathan Markham | 8.14 |
| 5105 | elongman | 8.12 |
| 5639 | (unknown) | ~8 |
| 6930 | Nolan Fout | 8.00 |
| 7021 | (unknown) | ~8 |
| 7127 | (unknown) | ~8 |
| 7176 | (unknown) | ~8 |
| 7729 | (unknown) | ~8 |
| 8230 | (unknown) | ~8 |
| 8455 | (unknown) | ~8 |
| 8463 | (unknown) | ~8 |

Note: Names marked (unknown) were not in the SPEAKERS.TXT mirror we fetched.

## Voice Names

These are our own names -- LibriSpeech only provides LibriVox usernames (like "supergirl",
"LadyBug", "number6"), not character-style voice names.

Names are assigned from pools in `voice_registry.json` and locked permanently.
The first 21 voices (original set) are:

**Female voices:** aria, clara, elena, grace, hazel, iris, luna, maya, ruby, sage, sofia

**Male voices:** atlas, caleb, felix, hugo, jasper, kai, leo, marcus, owen, theo

When all 40 test-clean speakers are registered, the remaining 19 get the next
names from the pools (amber, brooke, cora, ... for female; archer, blake, cole, ... for male).

## How Voices Are Selected

The script picks the best sample per speaker based on:
- Duration: prefers 5-10 seconds (ideal for voice cloning reference)
- Completeness: bonus for sentences ending with a period
- Length: penalty for very long text (>300 chars)

## Gender Verification

Speaker genders are fetched at runtime from the official LibriSpeech `SPEAKERS.TXT`
(via GitHub mirror). This ensures female names always map to female speakers and
male names always map to male speakers. No hardcoded gender assumptions.

Source: https://raw.githubusercontent.com/oscarknagg/voicemap/master/data/LibriSpeech/SPEAKERS.TXT

## Other LibriSpeech Splits

Additional splits can be added as new corpora in the registry:

- **train-clean-100**: ~250 speakers, 100 hours
- **train-clean-360**: ~920 speakers, 360 hours
- **train-other-500**: ~1160 speakers, 500 hours (noisier)
- **test-other**: 33 speakers (noisier conditions)
- **dev-clean**: 40 speakers (validation set)
- **dev-other**: 33 speakers

Using larger splits would give hundreds or thousands of distinct voices, but
test-clean provides the highest quality recordings.

## File Layout

```
voice_registry.json           # Idempotent voice-to-speaker assignments (committed)
cloned-voices/
├── aria.wav                  # Female voice sample (~7s, 16kHz)
├── atlas.wav                 # Male voice sample
├── ...                       # (40 voices when fully expanded)
└── voices_metadata.json      # Speaker IDs, genders, transcripts, durations
```

The `cloned-voices/` directory is mounted into the Docker container at `/app/cloned-voices/`.
Voice config is in `voice_to_speaker.default.yaml`.