Russell Ballestrini
9215eaa3f8
Fox 2026-06-06 on listener-to-speaker promotion: "the music slows
down I think that is the 6% algo which is janky. maybe it's better
to speed up in that case? if we skip ahead from whatever listener
is at to speaker speed, we need to make sure the video skips ahead
the same amount or rate to keep the lips synced".
Two coupled changes in the audio path:
1. JitterBufferProcessor retarget handler — on a SHRINKING retarget
(e.g. 4s → 0.5s) drop the queue down to targetSamples directly
instead of maxSamples (1.5×target). The 0.25s overhang the old
code left behind triggered a 6%-cap stretchFactor adjustment that
played at 1.064× for ~4 seconds — the residual phase fox heard
as "janky". Now: instant skip to the new target, then normal
playback. The grow-direction path is unchanged.
Worklet also reports the dropped sample count back to JS via
{cmd:'dropped', samples}.
2. installJitterBuffer.jbuf.port.onmessage — on 'dropped' from the
worklet, walk the publisher's lipSync.videoReceivers and set
jitterBufferTarget = 0 + playoutDelayHint = 0 on every one. The
browser drops video frames aggressively to converge on the new
target. Without this, audio jumps 3.25s forward instantly and
video drains gradually = broken lip-sync for the duration of the
native video jbuf's drain. Clearing ls.lastApplied lets the next
refreshLipSyncForUuid actually re-apply the role-appropriate
target instead of stopping at the threshold check.
Tests:
test/jitter-buffer-worklet.test.js (NEW, 8 assertions) — extracts
the inline JITTER_BUFFER_WORKLET_CODE template literal and runs
the processor in a Node sandbox with stubbed sampleRate +
registerProcessor + AudioWorkletProcessor. Pins:
- 'started' fires once on first fill
- 'buffered' reports depth
- retarget SHRINK drops to targetSamples (not maxSamples) and
posts 'dropped' with sample count
- retarget SHRINK does NOT leave 1.5×target overhang (the
residual that caused the janky 6% phase)
- retarget GROW does NOT drop
- retarget to same target is no-op
- bogus targetSeconds (NaN/0/negative) ignored
- lock_rate pins stretchFactor
test/listener-audio-attach.test.js (extended, +8 assertions):
- worklet 'dropped' handler zeros every paired video receiver
target (defensive against missing lipSync entry; correct when
seeded)
- speaker mesh+SFU collision tests (single-chain invariant):
mesh state=connected → SFU skip
mesh state=failed → SFU takes over
SFU first then mesh → in-place swap, no rebuild
mesh first then SFU(connected) → SFU skip, one chain
rapid mesh re-ontracks → still one gain feeds destination
SFU cache stays primed even when mesh wins the race
Makefile: new test-jitter-worklet target, added to test-all chain.
|
||
|---|---|---|
| blog | ||
| include | ||
| src | ||
| test | ||
| web | ||
| .gitignore | ||
| CLAUDE.md | ||
| Makefile | ||
| README.md | ||
zebra-report
Covert peer-to-peer chat over PulseAudio sink-input volume. Userland only. No kernel module. No network packets carry the chat content. Public domain.
Build & run
Single entry point: make. Every workflow goes through a target — never
invoke gcc or python3 by hand.
make # builds all five binaries (default target = `all`)
make test # builds and runs unit tests (pure logic, no PA needed)
make test-all # unit + integration + functional (PA + sink-inputs required)
make blog # rebuilds the static blog under web/blog/
make serve # builds blog, serves web/ on http://127.0.0.1:8765
make clean # removes binaries and generated blog output
make is the contract. If a workflow isn't a target, add the target before
adding the workflow.
Dependencies
gcc, make, pkg-config # build chain
libpulse-dev # PulseAudio client lib (pkg-config: libpulse)
python3 # blog builder + dev server
Ubuntu / Debian:
sudo apt install build-essential pkg-config libpulse-dev python3
make checks libpulse via pkg-config --cflags libpulse and
pkg-config --libs libpulse; if pkg-config can't find it, the build fails
early with a clear message.
Targets
make all — six binaries
| Binary | Source | Role |
|---|---|---|
tx |
src/tx.c |
reads stdin, transmits via volume modulation |
rx |
src/rx.c |
reads target sink-input volumes, decodes to stdout |
chat |
src/chat.c |
bidirectional tx+rx, line-based chat UI |
bt |
src/bt.c |
"battle toads" dual-channel stereo UART (2× throughput) |
carrier |
src/carrier.c |
publishes a silent PA sink so volume reads have something to read |
zebrad |
src/zebrad.c |
PA→WS introspector for web/chat.html |
All link against libpulse, librt, and libpthread. Headers come from
include/zebra.h (protocol constants) and include/modem.h (inline encode/
decode + benchmark).
make test — unit tests
test/unit pure logic, no PA, no audio
Covers: signal encoding (bit_to_vol / vol_to_bit), timing arithmetic
(ts_add_ns with overflow), baud math (baud_from_avg_ns), handshake frame
build/parse for both OFFER and READY (magic, checksum, corruption).
Currently: 84 cases pass. Required to be green before any commit.
make test-all — integration + functional
Integration and functional tests need a running PulseAudio daemon and
real sink-inputs to operate on. They are not run by make test by default.
# integration tests need ONE sink-input
ZEBRA_TEST_SINK=<sink_input_index> ./test/integration
# functional tests need TWO sink-inputs (data + ctrl channels)
ZEBRA_DATA_SINK=<idx> ZEBRA_CTRL_SINK=<idx> ./test/functional
Find sink-input indices with pactl list sink-inputs short — the first
column is the index. The test/integration benchmark sub-test reports
measured baud for the current host.
To avoid disrupting real audio applications, spawn dedicated silent sink-inputs as test targets:
paplay --raw --format=s16le --rate=44100 --channels=1 \
--stream-name=zebra-test /dev/zero &
# then use `pactl list short sink-inputs` to find this stream's index
Currently: 17/17 integration pass; 8/10 functional pass. The two functional failures are timing-bound on non-realtime kernels at the auto-negotiated baud and are not code defects (the same data path passes at fixed 50 baud).
make blog & make serve
make blog runs python3 blog/build.py, which reads Markdown sources from
blog/posts/ and writes static HTML into web/blog/. make serve rebuilds
the blog and serves web/ on port 8765 for local preview.
The chat UI (web/chat.html) is plain static HTML and works directly under
make serve — open http://127.0.0.1:8765/chat.html.
make zebrad — PulseAudio → WebSocket introspector
web/chat.html modulates audio output via Web Audio GainNode (mic stays off).
A browser tab cannot read another tab's PA state, so to close the receive
loop each peer runs zebrad. Decoded frames are forwarded over a local
WebSocket; chat.html auto-connects to ws://127.0.0.1:7777.
make zebrad
./zebrad --verbose # default: @DEFAULT_MONITOR@, port 7777
./zebrad --source <name> --port 7777
Single C file, no third-party deps beyond libpulse. Embedded WebSocket
server: SHA-1 + base64 inline, server→client binary frames only (RFC 6455
opcode 0x82); any inbound data closes the socket (browser auto-reconnects).
Bound to 127.0.0.1 only — never accessible from the LAN.
Adaptive baud: starts at ZEBRA_BAUD_HANDSHAKE (50), watches for a READY
frame, locks in the negotiated rate from its payload. Peak tracker has
2-second half-life decay so threshold adapts to room volume.
make clean
Removes:
tx rx chat bt carrier
test/unit test/integration test/functional
web/blog/index.html
web/blog/001-volume-modem/
web/blog/002-sse-chatroom/
Source files, headers, fonts, recovered binaries on disk, and committed
artifacts under web/ (other than the generated blog) are not touched.
Layout
include/ protocol constants + inline encode/decode/benchmark
src/ five binaries: tx, rx, chat, bt (battle toads), carrier
test/ unit (pure), integration (PA needed), functional (full chain)
blog/ markdown sources + python build
web/ static site: index, kernel, chat, fonts, generated blog
Makefile every workflow lives here
CLAUDE.md agent operating rules for this repo
License
Public domain. Patches gratefully accepted via merge request at
git.unturf.com/engineering/unturf/zebra-report.