engineering/www.unturf.com
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

zebra-report: deploy shrink-retarget audio + lip-sync coupling

Browse source 
Pulls in zebra-report 9215eaa: JitterBufferProcessor now drops the
queue down to targetSamples (not 1.5×target) on a shrinking retarget,
eliminating the residual 6%-cap speed-up phase fox heard as "janky
slow" during listener-to-speaker promotion. The processor reports
the dropped sample count back to JS so installJitterBuffer's port
handler zeros every paired video receiver's jitterBufferTarget +
playoutDelayHint — video frames drop in lockstep with the audio
skip so lip-sync survives the jump.

Fox 2026-06-06: "if we skip ahead from whatever listener is at to
speaker speed, we need to make sure the video skips ahead the same
amount or rate to keep the lips synced".

Behavior change is isolated to the role-change moment (4s→0.5s on
promote). Steady-state audio and listener-only sessions unchanged.
This commit is contained in:
russell@unturf.com 2026-06-06 15:21:29 -04:00
parent 3a0a30807e
commit e45db971aa
No known key found for this signature in database
1 changed files with 63 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

66
zebra-report/zebra-spaces.html
View file

@ -1878,14 +1878,37 @@ class JitterBufferProcessor extends AudioWorkletProcessor {
if (e.data.cmd === 'retarget'){
const t = +e.data.targetSeconds;
if (!isFinite(t) || t <= 0) return;
const prevTargetSamples = this.targetSamples;
this.targetSeconds = t;
this.maxSeconds = t * 1.5;
this.targetSamples = Math.round(this.targetSeconds * sampleRate);
this.maxSamples = Math.round(this.maxSeconds * sampleRate);
while (this.buffered > this.maxSamples && this.queue.length > 0){
/* On a SHRINKING retarget (listener-to-speaker promotion: 4s
* → 0.5s), drop down to targetSamples directly instead of
* maxSamples. The maxSamples threshold leaves a 1.5× target
* overhang that triggers an audible 6%-cap speed-up phase
* lasting ~4 seconds — what fox 2026-06-06 called "janky"
* during promotion. One hard skip-forward, then normal
* playback — no residual adjustment.
*
* Report the skip back to JS so paired video receivers can
* advance the same amount: an audio jump without a matching
* video jump = broken lip-sync for the duration of the
* native video jbuf's gradual drain. fox 2026-06-06: "if we
* skip ahead from whatever listener is at to speaker speed,
* we need to make sure the video skips ahead the same amount
* or rate to keep the lips synced". */
const shrinking = this.targetSamples < prevTargetSamples;
const dropTo = shrinking ? this.targetSamples : this.maxSamples;
let skipped = 0;
while (this.buffered > dropTo && this.queue.length > 0){
const drop = this.queue.shift();
this.buffered -= drop[0].length;
this.dropped += drop[0].length;
skipped += drop[0].length;
}
if (skipped > 0){
try { this.port.postMessage({ cmd: 'dropped', samples: skipped }); } catch(_){}
}
} else if (e.data.cmd === 'lock_rate'){
/* listeners get this — explicitly forbid time-stretching so
@ -2542,6 +2565,43 @@ function installJitterBuffer(uuid, node){
* enough to catch a wiggle within ~2s of it starting,
* vs the 5s tick path which can be 5-10s late. */
ddNoteWorkletBuffered(uuid, e.data.seconds);
} else if (e.data.cmd === 'dropped'){
/* The worklet just skipped audio forward (shrinking retarget
* — listener→speaker promotion drops 4s→0.5s buffer). Drag
* the paired video receivers forward by the same amount so
* lip-sync survives the jump. fox 2026-06-06: "if we skip
* ahead from whatever listener is at to speaker speed, we
* need to make sure the video skips ahead the same amount
* or rate to keep the lips synced".
*
* No native "skip ahead" API exists for RTCRtpReceiver, but
* jitterBufferTarget is a hard target the browser converges
* to. Setting it to 0 forces aggressive frame-drop until the
* native video jbuf drains; the next refreshLipSyncForUuid
* 'buffered' tick (~2s) restores the proper role-appropriate
* target. Net: audio jumps instantly + video jumps almost
* instantly (browser frame-drop is fast) = lip-sync stays
* within ~tens of ms. */
try {
const mm = members.get(uuid);
const pubHex = mm && mm.pubkey ? hex(unb64(mm.pubkey)) : null;
if (pubHex){
const ls = lipSync.get(pubHex);
if (ls && ls.videoReceivers){
for (const [, rx] of ls.videoReceivers){
try { rx.jitterBufferTarget = 0; } catch(_){}
try { rx.playoutDelayHint = 0; } catch(_){}
}
/* clear lastApplied so the next refreshLipSyncForUuid
* actually re-applies the role's target — without this
* the threshold check could see "no significant change"
* and leave video stuck at 0. */
ls.lastApplied = 0;
}
}
const skippedSec = e.data.samples / (audioCtx ? audioCtx.sampleRate : 48000);
logLine('', 'jitter-buffer skipped '+skippedSec.toFixed(2)+'s uuid='+uuid.slice(0,4)+' — video re-targeting');
} catch(_){}
}
};
/* All roles now allow time-stretching — listeners benefit too
@ -7904,8 +7964,8 @@ logLine('', 'ready — pick a handle, type a rendezvous code, enter the space');
<footer style="margin:2.2rem auto 0;font-size:0.65rem;color:#999;line-height:1.7;word-break:break-all;font-family:monospace">
<span id="pi-seal" style="color:#777;cursor:default;user-select:none" title="">page integrity</span> &nbsp;·&nbsp; built <span class="stamp-date">2026-06-06</span><br>
md5 <span class="stamp-md5">973996244966a40f7370e23bec5dcbb5</span><br>
sha256 <span class="stamp-sha">66f3d4eebd028f7ea16ea58df624ed0d2223505ff5c99715730379757e613630</span><br>
md5 <span class="stamp-md5">bd769bd46730955722b172d51f93e673</span><br>
sha256 <span class="stamp-sha">a8f6964a1f1e0c87131b1df57a635659fdbbb215a29c7b0032997eb8f6bc5cfc</span><br>
<span style="color:#bbb">hashes are of this page with these two fields zeroed — to verify, blank them and re-hash</span><br>
<span style="color:#bbb">one self-contained file — <strong>save a copy</strong> and verify against these hashes; point at your own servers with ?signal= and ?turncred=, or <a href="host-your-own.html" style="color:#999">host your own community</a></span>
</footer>