# Karaoke Pipeline — Vocal Isolation via Unsandbox

## Overview

MPS separates instrumentals and vocals from audio/video uploads using spectral
mid-side Wiener masking (`voxsplit.c`, zero ML dependencies). Processing runs
inside unsandbox zerotrust containers. The pipeline is disk-backed with ~64KB
memory per worker at every stage.

## Architecture Diagram

```dot
// Render: dot -Tsvg docs/karaoke-pipeline.dot -o docs/karaoke-pipeline.dot.svg
digraph karaoke_pipeline {
    rankdir=LR;
    node [shape=box, style="rounded,filled", fontname="monospace", fontsize=10];
    edge [fontname="monospace", fontsize=9];

    subgraph cluster_mps {
        label="MPS (uWSGI)";
        style=dashed;
        color="#5871ad";

        trigger [label="Trigger\n(upload / on-demand / backfill)", fillcolor="#e8eaf6"];
        s3_down [label="S3 Download\n→ /tmp/karaoke_*/media.bin\n64KB chunks", fillcolor="#e8eaf6"];
        upload_media [label="POST /upload\n(streaming file body)\n→ upload_id", fillcolor="#e8eaf6"];
        upload_vox [label="POST /upload\nvoxsplit.c\n→ upload_id", fillcolor="#e8eaf6"];
        execute [label="POST /execute\n{upload_ids, script}\ntiny JSON, no file bytes", fillcolor="#e8eaf6"];
        stream_resp [label="Stream response\n→ /tmp/karaoke_*/response.json\n64KB chunks", fillcolor="#e8eaf6"];
        decode [label="Decode artifacts\n(base64 → tmpfile)\nupload to S3", fillcolor="#e8eaf6"];
        db_update [label="Update product\nextensions + file_bytes\nset S3 ACLs", fillcolor="#e8eaf6"];
    }

    subgraph cluster_api {
        label="api.unsandbox.com";
        style=dashed;
        color="#ad5871";

        recv_upload [label="Receive upload\nAES-256-CTR encrypt\n→ /tmp/uploads/{id}.enc\nkey in ETS only", fillcolor="#fce4ec"];
        dispatch [label="Dispatch execute\nErlang RPC\n~200B metadata only", fillcolor="#fce4ec"];
    }

    subgraph cluster_pool {
        label="Pool Node";
        style=dashed;
        color="#58ad71";

        pull [label="GET /internal/upload/{id}\nX-Upload-Key auth\nstreaming decrypt", fillcolor="#e8f5e9"];
        inject [label="lxc exec ... cat >\n/root/input/{filename}\n64KB streaming pipe", fillcolor="#e8f5e9"];
    }

    subgraph cluster_container {
        label="Zerotrust Container";
        style=dashed;
        color="#ad8f58";

        compile [label="gcc -O2 voxsplit.c -lm", fillcolor="#fff8e1"];
        extract [label="ffmpeg -i media\n-vn -acodec pcm_s16le\n-ar 44100 -ac 2\naudio.wav", fillcolor="#fff8e1"];
        split [label="voxsplit audio.wav\n→ split-instrumental.wav\n→ split-vocal.wav", fillcolor="#fff8e1"];
        remux [label="ffmpeg remux\n(video: copy video +\nisolated audio)\n(audio: copy wav)", fillcolor="#fff8e1"];
        artifacts [label="/tmp/artifacts/\ninstrumentals.{ext}\nvocals.{ext}", fillcolor="#fff8e1"];
    }

    s3 [label="S3 / CDN\n(DO Spaces or BYOB)", shape=cylinder, fillcolor="#f3e5f5"];
    browser [label="Browser\n(watch.js)", shape=ellipse, fillcolor="#e0f2f1"];

    trigger -> s3_down;
    s3_down -> upload_media [label="file on disk"];
    s3_down -> upload_vox;
    upload_media -> recv_upload [label="streaming\noctet-stream"];
    upload_vox -> recv_upload;
    recv_upload -> execute [label="upload_id", style=dashed, dir=back];
    execute -> dispatch [label="JSON\n{upload_ids}"];
    dispatch -> pull [label="RPC\nmetadata only"];
    pull -> recv_upload [label="HTTPS GET\nstreaming decrypt", style=dotted];
    pull -> inject;
    inject -> compile;
    compile -> extract;
    extract -> split;
    split -> remux;
    remux -> artifacts;
    artifacts -> stream_resp [label="response JSON\nbase64 artifacts", style=dotted];
    stream_resp -> decode;
    decode -> s3 [label="PUT instrumentals\nPUT vocals"];
    db_update -> s3 [label="ACL update", style=dashed];
    decode -> db_update;
    s3 -> browser [label="presigned URL\n15 min TTL"];
}
```

## Pipeline Stages

### Stage 1: Trigger

Three entry points, all converge on `process_karaoke()`:

| Entry Point | File | When |
|-------------|------|------|
| Upload | `views/product.py:382` | User uploads audio/video to product |
| On-demand | `views/watch.py:karaoke_process` | User clicks 🎤 button in watch mode |
| Backfill | `lib/karaoke.py:backfill_karaoke_async` | Shop owner triggers from settings |

On-demand and backfill fork a detached grandchild process (double-fork +
`os.setsid()`) that survives uWSGI worker recycling. Per-product lockfile
(`/tmp/karaoke_{product_id}.lock`) prevents duplicate processing.

### Stage 2: Download from S3

```python
# karaoke.py:217-226
resp = s3_client.get_object(Bucket=bucket, Key=s3_key)
with open(media_path, "wb") as f:
    while True:
        chunk = body.read(65536)  # 64KB
        if not chunk: break
        f.write(chunk)
```

### Stage 3: Upload to Unsandbox API

```python
# karaoke.py:45-77
# HMAC signs empty body — API skips body parsing for /upload
headers = {"X-Filename": filename, "Content-Type": "application/octet-stream"}
with open(file_path, "rb") as body:
    response = requests.post(url, data=body, headers=headers)
# Returns: {"upload_id": "uuid"}
```

Two uploads: the media file + `voxsplit.c` source. Both stream from disk,
constant memory.

### Stage 4: API Receives + Encrypts

`upload_store.ex` writes incoming bytes to `/tmp/uploads/{id}.enc` with
AES-256-CTR encryption. Unique key+IV per upload. Keys live only in ETS
(BEAM memory, never written to disk). Files auto-expire after 15 minutes.

### Stage 5: Execute (Metadata Only)

```python
# karaoke.py:80-125
payload = {
    "language": "bash",
    "code": script,
    "network_mode": "zerotrust",
    "input_files": [
        {"upload_id": media_id, "filename": "media"},
        {"upload_id": voxsplit_id, "filename": "voxsplit.c"},
    ],
}
```

Tiny JSON body. Zero file content crosses Erlang distribution.

### Stage 6: Pool Pulls Files

Pool node calls `GET /internal/upload/{id}` back to the API. API decrypts
on-the-fly, streams plaintext. Pool pipes into container:

```
lxc exec {container} -- sh -c 'cat > /root/input/media'
```

`/tmp/input` symlinks to `/root/input/` (canonical location).

### Stage 7: Container Processing

```bash
# Compile voxsplit (spectral mid-side Wiener masking, libc + libm only)
gcc -O2 -o /tmp/voxsplit /tmp/input/voxsplit.c -lm

# Extract audio from media
ffmpeg -y -i /tmp/input/media -vn -acodec pcm_s16le -ar 44100 -ac 2 /tmp/audio.wav

# Separate instrumentals and vocals
/tmp/voxsplit /tmp/audio.wav -o /tmp/split

# For video: remux original video with each isolated audio
ffmpeg -y -i /tmp/input/media -i /tmp/split-instrumental.wav \
    -c:v copy -map 0:v -map 1:a -shortest /tmp/artifacts/instrumentals.mp4
ffmpeg -y -i /tmp/input/media -i /tmp/split-vocal.wav \
    -c:v copy -map 0:v -map 1:a -shortest /tmp/artifacts/vocals.mp4
```

### Stage 8: Response + S3 Upload

Response JSON streams back to MPS temp file. Artifacts are base64-encoded in
the response. MPS decodes each one to a temp file, uploads to S3, deletes
temp file. Updates product metadata and S3 ACLs.

## On-Demand Flow (Watch Mode)

```dot
// Render: dot -Tsvg docs/karaoke-ondemand.dot -o docs/karaoke-ondemand.dot.svg
digraph karaoke_ondemand {
    rankdir=TB;
    node [shape=box, style="rounded,filled", fontname="monospace", fontsize=10];
    edge [fontname="monospace", fontsize=9];

    click [label="User clicks 🎤\n(watch.js)", fillcolor="#e0f2f1"];
    check [label="Has tracks?", shape=diamond, fillcolor="#fff8e1"];
    cycle [label="cycleKaraoke()\nOriginal → Instrumentals → Vocals", fillcolor="#e8f5e9"];
    post [label="POST /karaoke/{id}\n(fetch, non-blocking)", fillcolor="#e8eaf6"];
    fork [label="Server forks\ndetached grandchild", fillcolor="#e8eaf6"];
    process [label="process_karaoke()\n(~30-120s)", fillcolor="#fce4ec"];
    hourglass [label="Button shows ⌛\nkaraokeProcessing=true", fillcolor="#e0f2f1"];
    refresh [label="10s URL refresh\nfetchWatchData()", fillcolor="#e0f2f1"];
    detect [label="instrumentals_url\nappears in JSON", shape=diamond, fillcolor="#fff8e1"];
    autoswitch [label="Auto-switch to\ninstrumentals 🎤", fillcolor="#e8f5e9"];

    click -> check;
    check -> cycle [label="yes"];
    check -> post [label="no (eligible)"];
    post -> fork [label="200 {status: processing}"];
    post -> hourglass;
    fork -> process;
    hourglass -> refresh [label="every 10s"];
    refresh -> detect;
    detect -> refresh [label="not yet"];
    detect -> autoswitch [label="tracks ready"];
    process -> detect [label="DB updated", style=dashed];
}
```

## Security Properties

- **Encryption at rest**: AES-256-CTR, unique key+IV per upload, key in ETS only
- **Single-use**: Upload deleted immediately after pool pulls it
- **TTL**: 15-minute expiry, swept every 60 seconds
- **Key loss on restart**: BEAM restart = all encryption keys gone = files undecryptable
- **Auth**: `X-Upload-Key` header + 128-bit random UUID per upload
- **Zerotrust network**: Container has no outbound network access

## Limits

| Limit | Value | Source |
|-------|-------|--------|
| Max file size | 3.698 GB (3,698,742,051 bytes) | `upload_store.ex @max_upload_bytes` |
| Memory per worker | ~64 KB | Disk-backed streaming at every stage |
| Concurrency | Per-key limit from `validate_keys()` | Unsandbox account tier |
| Retries | 3 attempts, exponential backoff (5s, 10s, 20s) | `karaoke.py` backfill |
| URL TTL | 15 minutes | Presigned URL expiry |
| Upload TTL | 15 minutes | API auto-cleanup |
| Container TTL | 300 seconds | `/execute` payload `ttl` field |

## S3 Key Structure

```
{shop_id}/products/{product_id}/
    product              # original media file
    preview              # preview (for sellable products)
    thumbnail1-4         # thumbnails
    instrumentals        # karaoke: isolated instrumentals
    vocals               # karaoke: isolated vocals
```

## Product Metadata

```python
product.extensions = {
    "product": "mp4",           # original
    "instrumentals": "mp4",     # same ext for video, "wav" for audio
    "vocals": "mp4",
}
product.file_bytes = {
    "product": 15000000,
    "instrumentals": 12000000,
    "vocals": 8000000,
}
```

## Related Files

| File | Role |
|------|------|
| `lib/karaoke.py` | Core pipeline: download, upload, execute, process response |
| `lib/voxsplit.c` | Spectral mid-side vocal isolation (C, libc+libm only) |
| `lib/un.py` | Unsandbox API client, HMAC signing, key validation |
| `views/watch.py` | On-demand `POST /karaoke/{id}` endpoint + watch JSON |
| `views/product.py` | Upload-triggered karaoke processing |
| `views/content.py` | Content page karaoke URL generation |
| `static/js/watch.js` | Client: button states, on-demand trigger, auto-switch |
| `templates/snippets/related_content.j2` | Karaoke button HTML |
| `templates/content.j2` | `data-karaoke-eligible` attribute |
| `scripts/backfill_karaoke.py` | Standalone backfill script |
