Russell Ballestrini
9215eaa3f8
Fox 2026-06-06 on listener-to-speaker promotion: "the music slows
down I think that is the 6% algo which is janky. maybe it's better
to speed up in that case? if we skip ahead from whatever listener
is at to speaker speed, we need to make sure the video skips ahead
the same amount or rate to keep the lips synced".
Two coupled changes in the audio path:
1. JitterBufferProcessor retarget handler — on a SHRINKING retarget
(e.g. 4s → 0.5s) drop the queue down to targetSamples directly
instead of maxSamples (1.5×target). The 0.25s overhang the old
code left behind triggered a 6%-cap stretchFactor adjustment that
played at 1.064× for ~4 seconds — the residual phase fox heard
as "janky". Now: instant skip to the new target, then normal
playback. The grow-direction path is unchanged.
Worklet also reports the dropped sample count back to JS via
{cmd:'dropped', samples}.
2. installJitterBuffer.jbuf.port.onmessage — on 'dropped' from the
worklet, walk the publisher's lipSync.videoReceivers and set
jitterBufferTarget = 0 + playoutDelayHint = 0 on every one. The
browser drops video frames aggressively to converge on the new
target. Without this, audio jumps 3.25s forward instantly and
video drains gradually = broken lip-sync for the duration of the
native video jbuf's drain. Clearing ls.lastApplied lets the next
refreshLipSyncForUuid actually re-apply the role-appropriate
target instead of stopping at the threshold check.
Tests:
test/jitter-buffer-worklet.test.js (NEW, 8 assertions) — extracts
the inline JITTER_BUFFER_WORKLET_CODE template literal and runs
the processor in a Node sandbox with stubbed sampleRate +
registerProcessor + AudioWorkletProcessor. Pins:
- 'started' fires once on first fill
- 'buffered' reports depth
- retarget SHRINK drops to targetSamples (not maxSamples) and
posts 'dropped' with sample count
- retarget SHRINK does NOT leave 1.5×target overhang (the
residual that caused the janky 6% phase)
- retarget GROW does NOT drop
- retarget to same target is no-op
- bogus targetSeconds (NaN/0/negative) ignored
- lock_rate pins stretchFactor
test/listener-audio-attach.test.js (extended, +8 assertions):
- worklet 'dropped' handler zeros every paired video receiver
target (defensive against missing lipSync entry; correct when
seeded)
- speaker mesh+SFU collision tests (single-chain invariant):
mesh state=connected → SFU skip
mesh state=failed → SFU takes over
SFU first then mesh → in-place swap, no rebuild
mesh first then SFU(connected) → SFU skip, one chain
rapid mesh re-ontracks → still one gain feeds destination
SFU cache stays primed even when mesh wins the race
Makefile: new test-jitter-worklet target, added to test-all chain.
171 lines
6.9 KiB
Makefile
171 lines
6.9 KiB
Makefile
CC = gcc
|
||
CFLAGS = -Wall -Wextra -O2 -Iinclude $(shell pkg-config --cflags libpulse)
|
||
LDFLAGS = $(shell pkg-config --libs libpulse) -lrt -lpthread
|
||
|
||
PULSE = src/pulse.c
|
||
|
||
.PHONY: all clean serve blog test test-all test-web test-zebra-spaces stamp zebrad
|
||
|
||
# stamp each web page with today's date + its own md5/sha256 (run before deploy)
|
||
stamp:
|
||
@node web/stamp.js web/chat.html web/zebra-audio.html web/zebra-spaces.html web/how-it-works.html web/host-your-own.html
|
||
|
||
all: tx rx chat bt carrier zebrad
|
||
|
||
test: test/unit
|
||
@./test/unit
|
||
|
||
# web modem/protocol tests — pure Node, no browser or second device needed
|
||
test-web:
|
||
@node test/web-protocol.test.js
|
||
|
||
# zebra-spaces state-machine tests — pure unit tests for the FSM framework
|
||
# + each machine spec (publish / subscribe / call / remote-tile). Extracts
|
||
# the live code from web/zebra-spaces.html so the tests track the page.
|
||
test-fsm:
|
||
@node test/zebra-fsm.test.js
|
||
|
||
# Receive-side stream lifecycle (watchVideoTrackForRemoval) — fake-clock
|
||
# state-machine tests covering 'when is it safe to remove a tile'. Catches
|
||
# the regressions where transient mutes (NACK gaps, network blips, mobile
|
||
# handoffs, hard-refresh renegotiation churn) would otherwise kill live
|
||
# tiles. Extracts the function from the page and the shipped mute window
|
||
# so the assertions track what's live.
|
||
test-video-removal:
|
||
@node test/video-track-removal.test.js
|
||
|
||
# Mesh state-sync invariant tests — pins the contract fox stated as
|
||
# "whatever one device shares all should see, and when unshared none
|
||
# should see." Runs the shipped handleRemoteSfuTrack + renderVideoTile +
|
||
# removeVideoTile + watchVideoTrackForRemoval against multi-peer
|
||
# scenarios with synthetic ontrack/mute/unmute/ended events. Catches
|
||
# regressions where one peer's publish/unpublish leaves another peer
|
||
# out of sync.
|
||
test-mesh:
|
||
@node test/multi-peer-mesh.test.js
|
||
|
||
# Mod-action serializer — pins the kick race fix fox hit 2026-06-04
|
||
# ("kicked two phones, only one was kicked"). Extracts runModSerial +
|
||
# awaitStateUpdate + resolvePendingStateUpdate from the live page so
|
||
# the assertions track the shipped code, then drives synthetic mod
|
||
# actions through them with a stubbed roomEpoch counter to prove the
|
||
# second action signs against the FRESH epoch.
|
||
test-mod-actions:
|
||
@node test/mod-action-serializer.test.js
|
||
|
||
# SelfListenerFSM — pins the state-machine contract for the speaker/
|
||
# cohost/host "switch myself to the buffered HTTP listener stream"
|
||
# toggle. Extracts createFSM + selfListenerSpec from the live page so
|
||
# the spec can't drift from shipped transitions (off↔on with
|
||
# TOGGLE/ENABLE/DISABLE/UNMUTE/DEMOTED/CLEAR edges).
|
||
test-self-listener:
|
||
@node test/self-listener-fsm.test.js
|
||
|
||
# Listener audio attach pipeline — pins src→(jbuf)→gain→destination
|
||
# graph reachability through attachAudioStreamViaWorklet,
|
||
# setWorkletStream, installJitterBuffer, attachSfuTrack,
|
||
# flushSfuStreams. Extracts each function from the live page and
|
||
# replays the cascade scenarios (fresh attach, same-stream no-op,
|
||
# different-stream in-place swap, dedup by pubkey, publisher rejoin,
|
||
# zero-live-track reject). Catches silent-listener regressions where
|
||
# a chain builds but never reaches audioCtx.destination.
|
||
test-listener-audio:
|
||
@node test/listener-audio-attach.test.js
|
||
|
||
# sendByeIfRealClose role-aware bfcache split — pins that listeners
|
||
# send bye even on pagehide(persisted=true) so closed-Firefox phones
|
||
# don't sit as 128s roster zombies, while speakers keep the bfcache
|
||
# skip so their PCs survive a tab-switch nap.
|
||
test-sendbye:
|
||
@node test/sendbye-fsm.test.js
|
||
|
||
# JitterBufferProcessor worklet — extracts the inline template literal
|
||
# from the page + sandboxes it in Node, then drives process() + port
|
||
# messages directly. Pins the shrink-retarget contract: on a
|
||
# listener-to-speaker promotion the queue must drop to targetSamples
|
||
# (not 1.5×target) and the processor must report the dropped sample
|
||
# count back to JS so paired video receivers can advance in lockstep.
|
||
test-jitter-worklet:
|
||
@node test/jitter-buffer-worklet.test.js
|
||
|
||
# zebra-spaces JS↔Go protocol parity + vault + ed25519 + (optionally) a live
|
||
# server flow. The live-server tier auto-runs when proxy.unturf.com sits
|
||
# alongside this checkout AND has a Go toolchain — we build the relay binary
|
||
# transparently and point the test at it. Otherwise that tier is skipped and
|
||
# only the pure-protocol/crypto tiers run.
|
||
test-zebra-spaces:
|
||
@bin=""; \
|
||
if [ -d ../proxy.unturf.com/cmd/zebra-spaces-signal ]; then \
|
||
go=$$(command -v go || echo /home/fox/.local/go/bin/go); \
|
||
if [ -x "$$go" ]; then \
|
||
echo "Building zebra-spaces-signal for live-server test..."; \
|
||
(cd ../proxy.unturf.com && "$$go" build -o /tmp/zspc-signal-test ./cmd/zebra-spaces-signal/) && bin=/tmp/zspc-signal-test; \
|
||
fi; \
|
||
fi; \
|
||
ZEBRA_SPACES_BINARY=$$bin node test/zebra-spaces.test.js; \
|
||
rc=$$?; rm -f /tmp/zspc-signal-test; exit $$rc
|
||
|
||
test-all: test/unit test/integration test/functional test-web test-fsm test-video-removal test-mesh test-mod-actions test-self-listener test-listener-audio test-sendbye test-jitter-worklet test-zebra-spaces
|
||
@echo "--- unit ---"
|
||
@./test/unit
|
||
@echo "--- integration ---"
|
||
@./test/integration
|
||
@echo "--- functional ---"
|
||
@./test/functional
|
||
@echo "--- web protocol ---"
|
||
@node test/web-protocol.test.js
|
||
@echo "--- zebra-fsm ---"
|
||
@node test/zebra-fsm.test.js
|
||
@echo "--- video-track-removal ---"
|
||
@node test/video-track-removal.test.js
|
||
@echo "--- multi-peer-mesh ---"
|
||
@node test/multi-peer-mesh.test.js
|
||
@echo "--- mod-action serializer ---"
|
||
@node test/mod-action-serializer.test.js
|
||
@echo "--- self-listener FSM ---"
|
||
@node test/self-listener-fsm.test.js
|
||
@echo "--- listener audio attach ---"
|
||
@node test/listener-audio-attach.test.js
|
||
@echo "--- sendBye FSM ---"
|
||
@node test/sendbye-fsm.test.js
|
||
@echo "--- jitter-buffer worklet ---"
|
||
@node test/jitter-buffer-worklet.test.js
|
||
@echo "--- zebra-spaces ---"
|
||
@$(MAKE) -s test-zebra-spaces
|
||
|
||
test/unit: test/unit.c include/zebra.h include/modem.h test/test.h
|
||
$(CC) $(CFLAGS) -o $@ test/unit.c
|
||
|
||
test/integration: test/integration.c src/pulse.c include/zebra.h include/modem.h test/test.h
|
||
$(CC) $(CFLAGS) -o $@ test/integration.c src/pulse.c $(LDFLAGS)
|
||
|
||
test/functional: test/functional.c src/pulse.c include/zebra.h include/modem.h test/test.h
|
||
$(CC) $(CFLAGS) -o $@ test/functional.c src/pulse.c $(LDFLAGS)
|
||
|
||
tx: src/tx.c $(PULSE)
|
||
$(CC) $(CFLAGS) -o $@ $^ $(LDFLAGS)
|
||
|
||
rx: src/rx.c $(PULSE)
|
||
$(CC) $(CFLAGS) -o $@ $^ $(LDFLAGS)
|
||
|
||
chat: src/chat.c $(PULSE)
|
||
$(CC) $(CFLAGS) -o $@ $^ $(LDFLAGS)
|
||
|
||
bt: src/bt.c $(PULSE)
|
||
$(CC) $(CFLAGS) -o $@ $^ $(LDFLAGS)
|
||
|
||
carrier: src/carrier.c
|
||
$(CC) $(CFLAGS) -o $@ $^ $(LDFLAGS)
|
||
|
||
zebrad: src/zebrad.c include/zebra.h
|
||
$(CC) $(CFLAGS) -o $@ src/zebrad.c $(LDFLAGS) -lm
|
||
|
||
blog:
|
||
python3 blog/build.py
|
||
|
||
serve: blog
|
||
cd web && python3 -m http.server 8765
|
||
|
||
clean:
|
||
rm -f tx rx chat bt carrier zebrad test/unit test/integration test/functional
|
||
rm -rf web/blog/001-volume-modem web/blog/002-sse-chatroom web/blog/index.html
|