diff --git a/CLAUDE.md b/CLAUDE.md index b652632..253cbcb 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -246,167 +246,75 @@ structure, button wiring, CSS, the entry/orientation flow before joining a space). For everything that lives inside an existing PC, prefer `leave` + `enter`. -### Audio jitter buffering — userland AudioWorklet, not browser hints +### Audio pipeline — load-bearing invariants -**The browser-native jitter-buffer controls cannot be trusted for music.** -Verified 2026-06-04 with side-by-side telemetry on a Firefox Android phone, -same PeerConnection, three receivers, identical 4s target: +Five rules. Break any and audio dies in a non-obvious way. Each links +to its deep dive. -- `RTCRtpReceiver.playoutDelayHint` is spec'd as a hint — "the user agent MAY - use this." Browsers do whatever they want. -- `RTCRtpReceiver.jitterBufferTarget` is spec'd as a hard target. Honored for - voice-rate Opus and for video. **Ignored for high-bitrate stereo Opus - (256 kbps music) on Firefox Android.** The native music-stream code path - inside libwebrtc isn't wired to the new API there. +1. **Userland AudioWorklet is the real jitter cushion.** Browser + `RTCRtpReceiver.playoutDelayHint` / `jitterBufferTarget` are + honored inconsistently (ignored for high-bitrate stereo Opus on + Firefox Android). `JitterBufferProcessor` in + `web/zebra-spaces.html` queues 128-sample blocks between + `MediaStreamAudioSourceNode` and `GainNode`. Sticky-started: + tolerate ≥267 ms of empty blocks before re-arming or you re-buffer + on every 2.67 ms micro-stall. + → `memory: feedback-audio-jitter-buffer-strategy` -Result: a phone listener with the spec'd 4s target had a 0.21s buffer on the -music stream. Any host-side stall (X11 window wiggle, GC pause, encoder -spike) was instantly audible. +2. **Role-aware target seconds.** Listener 4s + (`RECV_PLAYOUT_DELAY_SEC`), speaker / cohost / host 0.7s + (`SPEAKER_PLAYOUT_DELAY_SEC`). Retarget by posting + `{cmd:'retarget', targetSeconds}` to the worklet port — don't + rebuild. `retargetAllReceivers(role)`. UI gates on the worklet's + `{cmd:'started'}` message so the user isn't shown "connected" + while still buffering. + → `memory: feedback-4s-playout-validated-cellular-music` -**The reliable cushion is a userland `AudioWorklet`.** See -`JitterBufferProcessor` (inline Blob URL) in `web/zebra-spaces.html`. The -worklet sits between `MediaStreamAudioSourceNode` and `GainNode`, queues -incoming 128-sample blocks, holds emission until `targetSamples` are -buffered, then emits with constant delay. The buffer cushions the music -regardless of what the native receiver does. +3. **Chromium decoder anchor is non-negotiable.** Every remote + `MediaStreamTrack` consumed by a `MediaStreamAudioSourceNode` + ALSO needs a hidden muted `