Fox 2026-06-06 on listener-to-speaker promotion: "the music slows
down I think that is the 6% algo which is janky. maybe it's better
to speed up in that case? if we skip ahead from whatever listener
is at to speaker speed, we need to make sure the video skips ahead
the same amount or rate to keep the lips synced".
Two coupled changes in the audio path:
1. JitterBufferProcessor retarget handler — on a SHRINKING retarget
(e.g. 4s → 0.5s) drop the queue down to targetSamples directly
instead of maxSamples (1.5×target). The 0.25s overhang the old
code left behind triggered a 6%-cap stretchFactor adjustment that
played at 1.064× for ~4 seconds — the residual phase fox heard
as "janky". Now: instant skip to the new target, then normal
playback. The grow-direction path is unchanged.
Worklet also reports the dropped sample count back to JS via
{cmd:'dropped', samples}.
2. installJitterBuffer.jbuf.port.onmessage — on 'dropped' from the
worklet, walk the publisher's lipSync.videoReceivers and set
jitterBufferTarget = 0 + playoutDelayHint = 0 on every one. The
browser drops video frames aggressively to converge on the new
target. Without this, audio jumps 3.25s forward instantly and
video drains gradually = broken lip-sync for the duration of the
native video jbuf's drain. Clearing ls.lastApplied lets the next
refreshLipSyncForUuid actually re-apply the role-appropriate
target instead of stopping at the threshold check.
Tests:
test/jitter-buffer-worklet.test.js (NEW, 8 assertions) — extracts
the inline JITTER_BUFFER_WORKLET_CODE template literal and runs
the processor in a Node sandbox with stubbed sampleRate +
registerProcessor + AudioWorkletProcessor. Pins:
- 'started' fires once on first fill
- 'buffered' reports depth
- retarget SHRINK drops to targetSamples (not maxSamples) and
posts 'dropped' with sample count
- retarget SHRINK does NOT leave 1.5×target overhang (the
residual that caused the janky 6% phase)
- retarget GROW does NOT drop
- retarget to same target is no-op
- bogus targetSeconds (NaN/0/negative) ignored
- lock_rate pins stretchFactor
test/listener-audio-attach.test.js (extended, +8 assertions):
- worklet 'dropped' handler zeros every paired video receiver
target (defensive against missing lipSync entry; correct when
seeded)
- speaker mesh+SFU collision tests (single-chain invariant):
mesh state=connected → SFU skip
mesh state=failed → SFU takes over
SFU first then mesh → in-place swap, no rebuild
mesh first then SFU(connected) → SFU skip, one chain
rapid mesh re-ontracks → still one gain feeds destination
SFU cache stays primed even when mesh wins the race
Makefile: new test-jitter-worklet target, added to test-all chain.
Russell Ballestrini
9215eaa3f8