From 8649e6aaae4021023b621116f1fffd4b0aa67d59 Mon Sep 17 00:00:00 2001 From: "russell@unturf.com" Date: Wed, 11 Mar 2026 17:49:57 -0400 Subject: [PATCH] docs: karaoke pipeline architecture with dot diagrams Add docs/karaoke-pipeline.md covering the full streaming pipeline from MPS through unsandbox API to zerotrust container and back. Includes two Graphviz dot diagrams (rendered to SVG): - karaoke-pipeline.dot: full system flow across MPS, API, pool, container - karaoke-ondemand.dot: watch mode on-demand user flow Update architecture.md feature toggle matrix and related docs table. Update CLAUDE.md karaoke section with streaming path and on-demand info. --- CLAUDE.md | 15 +- docs/architecture.md | 2 + docs/karaoke-ondemand.dot | 30 ++++ docs/karaoke-ondemand.dot.svg | 155 ++++++++++++++++ docs/karaoke-pipeline.dot | 77 ++++++++ docs/karaoke-pipeline.dot.svg | 327 ++++++++++++++++++++++++++++++++++ docs/karaoke-pipeline.md | 286 +++++++++++++++++++++++++++++ 7 files changed, 887 insertions(+), 5 deletions(-) create mode 100644 docs/karaoke-ondemand.dot create mode 100644 docs/karaoke-ondemand.dot.svg create mode 100644 docs/karaoke-pipeline.dot create mode 100644 docs/karaoke-pipeline.dot.svg create mode 100644 docs/karaoke-pipeline.md diff --git a/CLAUDE.md b/CLAUDE.md index e57d55e..118fc82 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -29,15 +29,20 @@ Files are NEVER streamed through uwsgi. The server only generates presigned URLs ### Karaoke Pipeline (lib/karaoke.py) -Disk-backed vocal isolation pipeline. Downloads media from S3, builds a JSON -request on disk with streaming base64 encoding, POSTs to unsandbox API, streams -response to disk, decodes artifacts one at a time and uploads to S3. +Disk-backed vocal isolation pipeline using spectral mid-side Wiener masking +(`voxsplit.c`, zero ML deps). Streams media to unsandbox via `POST /upload` +(64KB chunks, constant memory), executes in zerotrust container, streams +response back, uploads instrumentals + vocals to S3. + +Full architecture doc: `docs/karaoke-pipeline.md` (with dot diagrams). - **Concurrency**: `ThreadPoolExecutor` sized to account's unsandbox concurrency limit -- **Memory**: ~64KB per worker on the request side (disk-backed, not in-memory) +- **Memory**: ~64KB per worker at every stage (disk-backed, not in-memory) - **Upstream limit**: 3.698GB / 3,698,742,051 bytes per file (unsandbox `@max_upload_bytes`) - **Retries**: 3 attempts with exponential backoff (5s, 10s) -- **Callers**: `views/product.py`, `views/shop.py` (backfill), `scripts/backfill_karaoke.py` +- **Callers**: `views/product.py` (upload), `views/watch.py` (on-demand), `views/shop.py` (backfill), `scripts/backfill_karaoke.py` +- **On-demand**: `POST /karaoke/{product_id}` — forks detached child, watch.js 10s refresh detects completion, auto-switches to instrumentals +- **Streaming path**: MPS → `POST /upload` → API encrypts to disk → pool pulls via `GET /internal/upload/{id}` → pipes into container `/root/input/` — zero bytes cross Erlang distribution ## Project Setup diff --git a/docs/architecture.md b/docs/architecture.md index 1368f7a..28b0640 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -206,6 +206,7 @@ mps_page_session (raw rows) | Trial | `shop.trial_started_timestamp` | Auto on creation | 21 days | | Discovery ring | `shop.discovery_ring` | Automatic | Auto-computed | | Subscriptions | `shop.subscription_*` | `ribbon-settings` | Off | +| Karaoke (vocal isolation) | `shop.unsandbox_*_key` | `unsandbox-settings` | Off | ## Ticket Index @@ -235,5 +236,6 @@ mps_page_session (raw rows) |-----|---------| | [Design System](design-system.md) | Design tokens, CSS architecture, component library | | [JavaScript](JAVASCRIPT.md) | Client-side JS architecture | +| [Karaoke Pipeline](karaoke-pipeline.md) | Vocal isolation: pipeline, on-demand, streaming architecture | | [Sandbox Mode](sandbox-mode.md) | Creative filter system | | [Testing Performance](testing-performance.md) | Test suite optimization | diff --git a/docs/karaoke-ondemand.dot b/docs/karaoke-ondemand.dot new file mode 100644 index 0000000..2773a5b --- /dev/null +++ b/docs/karaoke-ondemand.dot @@ -0,0 +1,30 @@ +// On-Demand Karaoke — Watch Mode User Flow +// Render: dot -Tsvg docs/karaoke-ondemand.dot -o docs/karaoke-ondemand.dot.svg +digraph karaoke_ondemand { + rankdir=TB; + node [shape=box, style="rounded,filled", fontname="monospace", fontsize=10]; + edge [fontname="monospace", fontsize=9]; + + click [label="User clicks 🎤\n(watch.js)", fillcolor="#e0f2f1"]; + check [label="Has tracks?", shape=diamond, fillcolor="#fff8e1"]; + cycle [label="cycleKaraoke()\nOriginal → Instrumentals → Vocals", fillcolor="#e8f5e9"]; + post [label="POST /karaoke/{id}\n(fetch, non-blocking)", fillcolor="#e8eaf6"]; + fork [label="Server forks\ndetached grandchild", fillcolor="#e8eaf6"]; + process [label="process_karaoke()\n(~30-120s)", fillcolor="#fce4ec"]; + hourglass [label="Button shows ⌛\nkaraokeProcessing=true", fillcolor="#e0f2f1"]; + refresh [label="10s URL refresh\nfetchWatchData()", fillcolor="#e0f2f1"]; + detect [label="instrumentals_url\nappears in JSON?", shape=diamond, fillcolor="#fff8e1"]; + autoswitch [label="Auto-switch to\ninstrumentals 🎤", fillcolor="#e8f5e9"]; + + click -> check; + check -> cycle [label="yes"]; + check -> post [label="no (eligible)"]; + post -> fork [label="200 {status: processing}"]; + post -> hourglass; + fork -> process; + hourglass -> refresh [label="every 10s"]; + refresh -> detect; + detect -> refresh [label="not yet"]; + detect -> autoswitch [label="tracks ready"]; + process -> detect [label="DB updated", style=dashed]; +} diff --git a/docs/karaoke-ondemand.dot.svg b/docs/karaoke-ondemand.dot.svg new file mode 100644 index 0000000..53e6023 --- /dev/null +++ b/docs/karaoke-ondemand.dot.svg @@ -0,0 +1,155 @@ + + + + + + +karaoke_ondemand + + + +click + +User clicks 🎤 +(watch.js) + + + +check + +Has tracks? + + + +click->check + + + + + +cycle + +cycleKaraoke() +Original → Instrumentals → Vocals + + + +check->cycle + + +yes + + + +post + +POST /karaoke/{id} +(fetch, non-blocking) + + + +check->post + + +no (eligible) + + + +fork + +Server forks +detached grandchild + + + +post->fork + + +200 {status: processing} + + + +hourglass + +Button shows ⌛ +karaokeProcessing=true + + + +post->hourglass + + + + + +process + +process_karaoke() +(~30-120s) + + + +fork->process + + + + + +detect + +instrumentals_url +appears in JSON? + + + +process->detect + + +DB updated + + + +refresh + +10s URL refresh +fetchWatchData() + + + +hourglass->refresh + + +every 10s + + + +refresh->detect + + + + + +detect->refresh + + +not yet + + + +autoswitch + +Auto-switch to +instrumentals 🎤 + + + +detect->autoswitch + + +tracks ready + + + diff --git a/docs/karaoke-pipeline.dot b/docs/karaoke-pipeline.dot new file mode 100644 index 0000000..16b5b7d --- /dev/null +++ b/docs/karaoke-pipeline.dot @@ -0,0 +1,77 @@ +// Karaoke Pipeline — MPS → Unsandbox → Container → S3 +// Render: dot -Tsvg docs/karaoke-pipeline.dot -o docs/karaoke-pipeline.dot.svg +digraph karaoke_pipeline { + rankdir=LR; + node [shape=box, style="rounded,filled", fontname="monospace", fontsize=10]; + edge [fontname="monospace", fontsize=9]; + + subgraph cluster_mps { + label="MPS (uWSGI)"; + style=dashed; + color="#5871ad"; + + trigger [label="Trigger\n(upload / on-demand / backfill)", fillcolor="#e8eaf6"]; + s3_down [label="S3 Download\n→ /tmp/karaoke_*/media.bin\n64KB chunks", fillcolor="#e8eaf6"]; + upload_media [label="POST /upload\n(streaming file body)\n→ upload_id", fillcolor="#e8eaf6"]; + upload_vox [label="POST /upload\nvoxsplit.c\n→ upload_id", fillcolor="#e8eaf6"]; + execute [label="POST /execute\n{upload_ids, script}\ntiny JSON, no file bytes", fillcolor="#e8eaf6"]; + stream_resp [label="Stream response\n→ /tmp/karaoke_*/response.json\n64KB chunks", fillcolor="#e8eaf6"]; + decode [label="Decode artifacts\n(base64 → tmpfile)\nupload to S3", fillcolor="#e8eaf6"]; + db_update [label="Update product\nextensions + file_bytes\nset S3 ACLs", fillcolor="#e8eaf6"]; + } + + subgraph cluster_api { + label="api.unsandbox.com"; + style=dashed; + color="#ad5871"; + + recv_upload [label="Receive upload\nAES-256-CTR encrypt\n→ /tmp/uploads/{id}.enc\nkey in ETS only", fillcolor="#fce4ec"]; + dispatch [label="Dispatch execute\nErlang RPC\n~200B metadata only", fillcolor="#fce4ec"]; + } + + subgraph cluster_pool { + label="Pool Node"; + style=dashed; + color="#58ad71"; + + pull [label="GET /internal/upload/{id}\nX-Upload-Key auth\nstreaming decrypt", fillcolor="#e8f5e9"]; + inject [label="lxc exec ... cat >\n/root/input/{filename}\n64KB streaming pipe", fillcolor="#e8f5e9"]; + } + + subgraph cluster_container { + label="Zerotrust Container"; + style=dashed; + color="#ad8f58"; + + compile [label="gcc -O2 voxsplit.c -lm", fillcolor="#fff8e1"]; + extract [label="ffmpeg -i media\n-vn -acodec pcm_s16le\n-ar 44100 -ac 2\naudio.wav", fillcolor="#fff8e1"]; + split [label="voxsplit audio.wav\n→ split-instrumental.wav\n→ split-vocal.wav", fillcolor="#fff8e1"]; + remux [label="ffmpeg remux\n(video: copy video +\nisolated audio)\n(audio: copy wav)", fillcolor="#fff8e1"]; + artifacts [label="/tmp/artifacts/\ninstrumentals.{ext}\nvocals.{ext}", fillcolor="#fff8e1"]; + } + + s3 [label="S3 / CDN\n(DO Spaces or BYOB)", shape=cylinder, fillcolor="#f3e5f5"]; + browser [label="Browser\n(watch.js)", shape=ellipse, fillcolor="#e0f2f1"]; + + trigger -> s3_down; + s3_down -> upload_media [label="file on disk"]; + s3_down -> upload_vox; + upload_media -> recv_upload [label="streaming\noctet-stream"]; + upload_vox -> recv_upload; + recv_upload -> execute [label="upload_id", style=dashed, dir=back]; + execute -> dispatch [label="JSON\n{upload_ids}"]; + dispatch -> pull [label="RPC\nmetadata only"]; + pull -> recv_upload [label="HTTPS GET\nstreaming decrypt", style=dotted]; + pull -> inject; + inject -> compile; + compile -> extract; + extract -> split; + split -> remux; + remux -> artifacts; + artifacts -> stream_resp [label="response JSON\nbase64 artifacts", style=dotted]; + stream_resp -> decode; + decode -> s3 [label="PUT instrumentals\nPUT vocals"]; + db_update -> s3 [label="ACL update", style=dashed]; + decode -> db_update; + s3 -> browser [label="presigned URL\n15 min TTL"]; +} diff --git a/docs/karaoke-pipeline.dot.svg b/docs/karaoke-pipeline.dot.svg new file mode 100644 index 0000000..8681fd9 --- /dev/null +++ b/docs/karaoke-pipeline.dot.svg @@ -0,0 +1,327 @@ + + + + + + +karaoke_pipeline + + +cluster_mps + +MPS (uWSGI) + + +cluster_api + +api.unsandbox.com + + +cluster_pool + +Pool Node + + +cluster_container + +Zerotrust Container + + + +trigger + +Trigger +(upload / on-demand / backfill) + + + +s3_down + +S3 Download +→ /tmp/karaoke_*/media.bin +64KB chunks + + + +trigger->s3_down + + + + + +upload_media + +POST /upload +(streaming file body) +→ upload_id + + + +s3_down->upload_media + + +file on disk + + + +upload_vox + +POST /upload +voxsplit.c +→ upload_id + + + +s3_down->upload_vox + + + + + +recv_upload + +Receive upload +AES-256-CTR encrypt +→ /tmp/uploads/{id}.enc +key in ETS only + + + +upload_media->recv_upload + + +streaming +octet-stream + + + +upload_vox->recv_upload + + + + + +execute + +POST /execute +{upload_ids, script} +tiny JSON, no file bytes + + + +dispatch + +Dispatch execute +Erlang RPC +~200B metadata only + + + +execute->dispatch + + +JSON +{upload_ids} + + + +stream_resp + +Stream response +→ /tmp/karaoke_*/response.json +64KB chunks + + + +decode + +Decode artifacts +(base64 → tmpfile) +upload to S3 + + + +stream_resp->decode + + + + + +db_update + +Update product +extensions + file_bytes +set S3 ACLs + + + +decode->db_update + + + + + +s3 + + +S3 / CDN +(DO Spaces or BYOB) + + + +decode->s3 + + +PUT instrumentals +PUT vocals + + + +db_update->s3 + + +ACL update + + + +recv_upload->execute + + +upload_id + + + +pull + +GET /internal/upload/{id} +X-Upload-Key auth +streaming decrypt + + + +dispatch->pull + + +RPC +metadata only + + + +pull->recv_upload + + +HTTPS GET +streaming decrypt + + + +inject + +lxc exec ... cat > +/root/input/{filename} +64KB streaming pipe + + + +pull->inject + + + + + +compile + +gcc -O2 voxsplit.c -lm + + + +inject->compile + + + + + +extract + +ffmpeg -i media +-vn -acodec pcm_s16le +-ar 44100 -ac 2 +audio.wav + + + +compile->extract + + + + + +split + +voxsplit audio.wav +→ split-instrumental.wav +→ split-vocal.wav + + + +extract->split + + + + + +remux + +ffmpeg remux +(video: copy video + +isolated audio) +(audio: copy wav) + + + +split->remux + + + + + +artifacts + +/tmp/artifacts/ +instrumentals.{ext} +vocals.{ext} + + + +remux->artifacts + + + + + +artifacts->stream_resp + + +response JSON +base64 artifacts + + + +browser + +Browser +(watch.js) + + + +s3->browser + + +presigned URL +15 min TTL + + + diff --git a/docs/karaoke-pipeline.md b/docs/karaoke-pipeline.md new file mode 100644 index 0000000..5370856 --- /dev/null +++ b/docs/karaoke-pipeline.md @@ -0,0 +1,286 @@ +# Karaoke Pipeline — Vocal Isolation via Unsandbox + +## Overview + +MPS separates instrumentals and vocals from audio/video uploads using spectral +mid-side Wiener masking (`voxsplit.c`, zero ML dependencies). Processing runs +inside unsandbox zerotrust containers. The pipeline is disk-backed with ~64KB +memory per worker at every stage. + +## Architecture Diagram + +```dot +// Render: dot -Tsvg docs/karaoke-pipeline.dot -o docs/karaoke-pipeline.dot.svg +digraph karaoke_pipeline { + rankdir=LR; + node [shape=box, style="rounded,filled", fontname="monospace", fontsize=10]; + edge [fontname="monospace", fontsize=9]; + + subgraph cluster_mps { + label="MPS (uWSGI)"; + style=dashed; + color="#5871ad"; + + trigger [label="Trigger\n(upload / on-demand / backfill)", fillcolor="#e8eaf6"]; + s3_down [label="S3 Download\n→ /tmp/karaoke_*/media.bin\n64KB chunks", fillcolor="#e8eaf6"]; + upload_media [label="POST /upload\n(streaming file body)\n→ upload_id", fillcolor="#e8eaf6"]; + upload_vox [label="POST /upload\nvoxsplit.c\n→ upload_id", fillcolor="#e8eaf6"]; + execute [label="POST /execute\n{upload_ids, script}\ntiny JSON, no file bytes", fillcolor="#e8eaf6"]; + stream_resp [label="Stream response\n→ /tmp/karaoke_*/response.json\n64KB chunks", fillcolor="#e8eaf6"]; + decode [label="Decode artifacts\n(base64 → tmpfile)\nupload to S3", fillcolor="#e8eaf6"]; + db_update [label="Update product\nextensions + file_bytes\nset S3 ACLs", fillcolor="#e8eaf6"]; + } + + subgraph cluster_api { + label="api.unsandbox.com"; + style=dashed; + color="#ad5871"; + + recv_upload [label="Receive upload\nAES-256-CTR encrypt\n→ /tmp/uploads/{id}.enc\nkey in ETS only", fillcolor="#fce4ec"]; + dispatch [label="Dispatch execute\nErlang RPC\n~200B metadata only", fillcolor="#fce4ec"]; + } + + subgraph cluster_pool { + label="Pool Node"; + style=dashed; + color="#58ad71"; + + pull [label="GET /internal/upload/{id}\nX-Upload-Key auth\nstreaming decrypt", fillcolor="#e8f5e9"]; + inject [label="lxc exec ... cat >\n/root/input/{filename}\n64KB streaming pipe", fillcolor="#e8f5e9"]; + } + + subgraph cluster_container { + label="Zerotrust Container"; + style=dashed; + color="#ad8f58"; + + compile [label="gcc -O2 voxsplit.c -lm", fillcolor="#fff8e1"]; + extract [label="ffmpeg -i media\n-vn -acodec pcm_s16le\n-ar 44100 -ac 2\naudio.wav", fillcolor="#fff8e1"]; + split [label="voxsplit audio.wav\n→ split-instrumental.wav\n→ split-vocal.wav", fillcolor="#fff8e1"]; + remux [label="ffmpeg remux\n(video: copy video +\nisolated audio)\n(audio: copy wav)", fillcolor="#fff8e1"]; + artifacts [label="/tmp/artifacts/\ninstrumentals.{ext}\nvocals.{ext}", fillcolor="#fff8e1"]; + } + + s3 [label="S3 / CDN\n(DO Spaces or BYOB)", shape=cylinder, fillcolor="#f3e5f5"]; + browser [label="Browser\n(watch.js)", shape=ellipse, fillcolor="#e0f2f1"]; + + trigger -> s3_down; + s3_down -> upload_media [label="file on disk"]; + s3_down -> upload_vox; + upload_media -> recv_upload [label="streaming\noctet-stream"]; + upload_vox -> recv_upload; + recv_upload -> execute [label="upload_id", style=dashed, dir=back]; + execute -> dispatch [label="JSON\n{upload_ids}"]; + dispatch -> pull [label="RPC\nmetadata only"]; + pull -> recv_upload [label="HTTPS GET\nstreaming decrypt", style=dotted]; + pull -> inject; + inject -> compile; + compile -> extract; + extract -> split; + split -> remux; + remux -> artifacts; + artifacts -> stream_resp [label="response JSON\nbase64 artifacts", style=dotted]; + stream_resp -> decode; + decode -> s3 [label="PUT instrumentals\nPUT vocals"]; + db_update -> s3 [label="ACL update", style=dashed]; + decode -> db_update; + s3 -> browser [label="presigned URL\n15 min TTL"]; +} +``` + +## Pipeline Stages + +### Stage 1: Trigger + +Three entry points, all converge on `process_karaoke()`: + +| Entry Point | File | When | +|-------------|------|------| +| Upload | `views/product.py:382` | User uploads audio/video to product | +| On-demand | `views/watch.py:karaoke_process` | User clicks 🎤 button in watch mode | +| Backfill | `lib/karaoke.py:backfill_karaoke_async` | Shop owner triggers from settings | + +On-demand and backfill fork a detached grandchild process (double-fork + +`os.setsid()`) that survives uWSGI worker recycling. Per-product lockfile +(`/tmp/karaoke_{product_id}.lock`) prevents duplicate processing. + +### Stage 2: Download from S3 + +```python +# karaoke.py:217-226 +resp = s3_client.get_object(Bucket=bucket, Key=s3_key) +with open(media_path, "wb") as f: + while True: + chunk = body.read(65536) # 64KB + if not chunk: break + f.write(chunk) +``` + +### Stage 3: Upload to Unsandbox API + +```python +# karaoke.py:45-77 +# HMAC signs empty body — API skips body parsing for /upload +headers = {"X-Filename": filename, "Content-Type": "application/octet-stream"} +with open(file_path, "rb") as body: + response = requests.post(url, data=body, headers=headers) +# Returns: {"upload_id": "uuid"} +``` + +Two uploads: the media file + `voxsplit.c` source. Both stream from disk, +constant memory. + +### Stage 4: API Receives + Encrypts + +`upload_store.ex` writes incoming bytes to `/tmp/uploads/{id}.enc` with +AES-256-CTR encryption. Unique key+IV per upload. Keys live only in ETS +(BEAM memory, never written to disk). Files auto-expire after 15 minutes. + +### Stage 5: Execute (Metadata Only) + +```python +# karaoke.py:80-125 +payload = { + "language": "bash", + "code": script, + "network_mode": "zerotrust", + "input_files": [ + {"upload_id": media_id, "filename": "media"}, + {"upload_id": voxsplit_id, "filename": "voxsplit.c"}, + ], +} +``` + +Tiny JSON body. Zero file content crosses Erlang distribution. + +### Stage 6: Pool Pulls Files + +Pool node calls `GET /internal/upload/{id}` back to the API. API decrypts +on-the-fly, streams plaintext. Pool pipes into container: + +``` +lxc exec {container} -- sh -c 'cat > /root/input/media' +``` + +`/tmp/input` symlinks to `/root/input/` (canonical location). + +### Stage 7: Container Processing + +```bash +# Compile voxsplit (spectral mid-side Wiener masking, libc + libm only) +gcc -O2 -o /tmp/voxsplit /tmp/input/voxsplit.c -lm + +# Extract audio from media +ffmpeg -y -i /tmp/input/media -vn -acodec pcm_s16le -ar 44100 -ac 2 /tmp/audio.wav + +# Separate instrumentals and vocals +/tmp/voxsplit /tmp/audio.wav -o /tmp/split + +# For video: remux original video with each isolated audio +ffmpeg -y -i /tmp/input/media -i /tmp/split-instrumental.wav \ + -c:v copy -map 0:v -map 1:a -shortest /tmp/artifacts/instrumentals.mp4 +ffmpeg -y -i /tmp/input/media -i /tmp/split-vocal.wav \ + -c:v copy -map 0:v -map 1:a -shortest /tmp/artifacts/vocals.mp4 +``` + +### Stage 8: Response + S3 Upload + +Response JSON streams back to MPS temp file. Artifacts are base64-encoded in +the response. MPS decodes each one to a temp file, uploads to S3, deletes +temp file. Updates product metadata and S3 ACLs. + +## On-Demand Flow (Watch Mode) + +```dot +// Render: dot -Tsvg docs/karaoke-ondemand.dot -o docs/karaoke-ondemand.dot.svg +digraph karaoke_ondemand { + rankdir=TB; + node [shape=box, style="rounded,filled", fontname="monospace", fontsize=10]; + edge [fontname="monospace", fontsize=9]; + + click [label="User clicks 🎤\n(watch.js)", fillcolor="#e0f2f1"]; + check [label="Has tracks?", shape=diamond, fillcolor="#fff8e1"]; + cycle [label="cycleKaraoke()\nOriginal → Instrumentals → Vocals", fillcolor="#e8f5e9"]; + post [label="POST /karaoke/{id}\n(fetch, non-blocking)", fillcolor="#e8eaf6"]; + fork [label="Server forks\ndetached grandchild", fillcolor="#e8eaf6"]; + process [label="process_karaoke()\n(~30-120s)", fillcolor="#fce4ec"]; + hourglass [label="Button shows ⌛\nkaraokeProcessing=true", fillcolor="#e0f2f1"]; + refresh [label="10s URL refresh\nfetchWatchData()", fillcolor="#e0f2f1"]; + detect [label="instrumentals_url\nappears in JSON", shape=diamond, fillcolor="#fff8e1"]; + autoswitch [label="Auto-switch to\ninstrumentals 🎤", fillcolor="#e8f5e9"]; + + click -> check; + check -> cycle [label="yes"]; + check -> post [label="no (eligible)"]; + post -> fork [label="200 {status: processing}"]; + post -> hourglass; + fork -> process; + hourglass -> refresh [label="every 10s"]; + refresh -> detect; + detect -> refresh [label="not yet"]; + detect -> autoswitch [label="tracks ready"]; + process -> detect [label="DB updated", style=dashed]; +} +``` + +## Security Properties + +- **Encryption at rest**: AES-256-CTR, unique key+IV per upload, key in ETS only +- **Single-use**: Upload deleted immediately after pool pulls it +- **TTL**: 15-minute expiry, swept every 60 seconds +- **Key loss on restart**: BEAM restart = all encryption keys gone = files undecryptable +- **Auth**: `X-Upload-Key` header + 128-bit random UUID per upload +- **Zerotrust network**: Container has no outbound network access + +## Limits + +| Limit | Value | Source | +|-------|-------|--------| +| Max file size | 3.698 GB (3,698,742,051 bytes) | `upload_store.ex @max_upload_bytes` | +| Memory per worker | ~64 KB | Disk-backed streaming at every stage | +| Concurrency | Per-key limit from `validate_keys()` | Unsandbox account tier | +| Retries | 3 attempts, exponential backoff (5s, 10s, 20s) | `karaoke.py` backfill | +| URL TTL | 15 minutes | Presigned URL expiry | +| Upload TTL | 15 minutes | API auto-cleanup | +| Container TTL | 300 seconds | `/execute` payload `ttl` field | + +## S3 Key Structure + +``` +{shop_id}/products/{product_id}/ + product # original media file + preview # preview (for sellable products) + thumbnail1-4 # thumbnails + instrumentals # karaoke: isolated instrumentals + vocals # karaoke: isolated vocals +``` + +## Product Metadata + +```python +product.extensions = { + "product": "mp4", # original + "instrumentals": "mp4", # same ext for video, "wav" for audio + "vocals": "mp4", +} +product.file_bytes = { + "product": 15000000, + "instrumentals": 12000000, + "vocals": 8000000, +} +``` + +## Related Files + +| File | Role | +|------|------| +| `lib/karaoke.py` | Core pipeline: download, upload, execute, process response | +| `lib/voxsplit.c` | Spectral mid-side vocal isolation (C, libc+libm only) | +| `lib/un.py` | Unsandbox API client, HMAC signing, key validation | +| `views/watch.py` | On-demand `POST /karaoke/{id}` endpoint + watch JSON | +| `views/product.py` | Upload-triggered karaoke processing | +| `views/content.py` | Content page karaoke URL generation | +| `static/js/watch.js` | Client: button states, on-demand trigger, auto-switch | +| `templates/snippets/related_content.j2` | Karaoke button HTML | +| `templates/content.j2` | `data-karaoke-eligible` attribute | +| `scripts/backfill_karaoke.py` | Standalone backfill script |