Russell Ballestrini
c3ff58ca39
Fox 2026-06-04: "the cohost on fedora chrome has the wiggle issue when host
is messing with tabs is there any way to sync the other stream and recover
the glitches assuming it is still buffering where the low-latency version
is glitched, this means we need to delay more than what we are or some
other trick mixed in halp..." (plus: "the video would need to be slid
depending on the mode to keep it in sync. complicated but possible.")
Before: mesh audio bypassed the worklet — went straight to an <audio>
element with the native receiver's ~50ms buffer. The SFU worklet path
had 0.5s of cushion that absorbed 200ms host stalls; mesh did not, so
the same wiggle that was inaudible on SFU was clicky on mesh.
After: ONE worklet per remote uuid, source swappable in-place via the
new setWorkletStream(uuid, newStream). Both the SFU stream (cached in
sfuStreamsByPubHex) and the mesh stream contain the same publisher's
content at slightly different network delays — so disconnecting the
old source and connecting a new one to the same worklet is seamless
(the queue's 0.5s of already-buffered samples covers the transition
while the new source fills it).
Wiring:
- mesh ontrack: setWorkletStream(uuid, meshStream). Fallback to a
fresh attachAudioStreamViaWorklet if no worklet existed (rare —
only when AudioContext failed at SFU attach).
- mesh connectionState='failed': setWorkletStream(uuid, sfuCachedStream)
+ registerLipSyncAudio(pubHex, uuid, sfuReceiver). attachCachedSfuStreamFor
remains as the no-worklet fallback.
- HTTP /stream toggle still uses rampWorkletGain (separate <audio>
element path).
Lip-sync receiver rebinding ("video slid depending on the mode"):
- New sfuAudioReceivers map caches the SFU receiver per publisher.
- handleRemoteSfuTrack mic path: registerLipSyncAudio with SFU receiver
AND store it in sfuAudioReceivers cache.
- mesh ontrack: registerLipSyncAudio with the MESH receiver — the
next worklet 'buffered' message will retarget video to mesh's jbuf
+ worklet (≈ 0.55s) instead of SFU's (≈ 1s).
- mesh fail: registerLipSyncAudio back to the cached SFU receiver
→ video re-targets again.
Removes the previous gain-ramp hack for SFU↔mesh transitions —
single worklet means no parallel paths, no need to crossfade. The
gain-ramp is still used for HTTP toggle (where there genuinely are
two paths: worklet + HTTP <audio>).
CPU cost: same as before — the SFU sub PC still decodes audio for
every speaker (we just route the decoded stream to the worklet or
not). Net change is "the mesh <audio> element is gone" — small
saving.
|
||
|---|---|---|
| blog | ||
| include | ||
| src | ||
| test | ||
| web | ||
| .gitignore | ||
| CLAUDE.md | ||
| Makefile | ||
| README.md | ||
zebra-report
Covert peer-to-peer chat over PulseAudio sink-input volume. Userland only. No kernel module. No network packets carry the chat content. Public domain.
Build & run
Single entry point: make. Every workflow goes through a target — never
invoke gcc or python3 by hand.
make # builds all five binaries (default target = `all`)
make test # builds and runs unit tests (pure logic, no PA needed)
make test-all # unit + integration + functional (PA + sink-inputs required)
make blog # rebuilds the static blog under web/blog/
make serve # builds blog, serves web/ on http://127.0.0.1:8765
make clean # removes binaries and generated blog output
make is the contract. If a workflow isn't a target, add the target before
adding the workflow.
Dependencies
gcc, make, pkg-config # build chain
libpulse-dev # PulseAudio client lib (pkg-config: libpulse)
python3 # blog builder + dev server
Ubuntu / Debian:
sudo apt install build-essential pkg-config libpulse-dev python3
make checks libpulse via pkg-config --cflags libpulse and
pkg-config --libs libpulse; if pkg-config can't find it, the build fails
early with a clear message.
Targets
make all — six binaries
| Binary | Source | Role |
|---|---|---|
tx |
src/tx.c |
reads stdin, transmits via volume modulation |
rx |
src/rx.c |
reads target sink-input volumes, decodes to stdout |
chat |
src/chat.c |
bidirectional tx+rx, line-based chat UI |
bt |
src/bt.c |
"battle toads" dual-channel stereo UART (2× throughput) |
carrier |
src/carrier.c |
publishes a silent PA sink so volume reads have something to read |
zebrad |
src/zebrad.c |
PA→WS introspector for web/chat.html |
All link against libpulse, librt, and libpthread. Headers come from
include/zebra.h (protocol constants) and include/modem.h (inline encode/
decode + benchmark).
make test — unit tests
test/unit pure logic, no PA, no audio
Covers: signal encoding (bit_to_vol / vol_to_bit), timing arithmetic
(ts_add_ns with overflow), baud math (baud_from_avg_ns), handshake frame
build/parse for both OFFER and READY (magic, checksum, corruption).
Currently: 84 cases pass. Required to be green before any commit.
make test-all — integration + functional
Integration and functional tests need a running PulseAudio daemon and
real sink-inputs to operate on. They are not run by make test by default.
# integration tests need ONE sink-input
ZEBRA_TEST_SINK=<sink_input_index> ./test/integration
# functional tests need TWO sink-inputs (data + ctrl channels)
ZEBRA_DATA_SINK=<idx> ZEBRA_CTRL_SINK=<idx> ./test/functional
Find sink-input indices with pactl list sink-inputs short — the first
column is the index. The test/integration benchmark sub-test reports
measured baud for the current host.
To avoid disrupting real audio applications, spawn dedicated silent sink-inputs as test targets:
paplay --raw --format=s16le --rate=44100 --channels=1 \
--stream-name=zebra-test /dev/zero &
# then use `pactl list short sink-inputs` to find this stream's index
Currently: 17/17 integration pass; 8/10 functional pass. The two functional failures are timing-bound on non-realtime kernels at the auto-negotiated baud and are not code defects (the same data path passes at fixed 50 baud).
make blog & make serve
make blog runs python3 blog/build.py, which reads Markdown sources from
blog/posts/ and writes static HTML into web/blog/. make serve rebuilds
the blog and serves web/ on port 8765 for local preview.
The chat UI (web/chat.html) is plain static HTML and works directly under
make serve — open http://127.0.0.1:8765/chat.html.
make zebrad — PulseAudio → WebSocket introspector
web/chat.html modulates audio output via Web Audio GainNode (mic stays off).
A browser tab cannot read another tab's PA state, so to close the receive
loop each peer runs zebrad. Decoded frames are forwarded over a local
WebSocket; chat.html auto-connects to ws://127.0.0.1:7777.
make zebrad
./zebrad --verbose # default: @DEFAULT_MONITOR@, port 7777
./zebrad --source <name> --port 7777
Single C file, no third-party deps beyond libpulse. Embedded WebSocket
server: SHA-1 + base64 inline, server→client binary frames only (RFC 6455
opcode 0x82); any inbound data closes the socket (browser auto-reconnects).
Bound to 127.0.0.1 only — never accessible from the LAN.
Adaptive baud: starts at ZEBRA_BAUD_HANDSHAKE (50), watches for a READY
frame, locks in the negotiated rate from its payload. Peak tracker has
2-second half-life decay so threshold adapts to room volume.
make clean
Removes:
tx rx chat bt carrier
test/unit test/integration test/functional
web/blog/index.html
web/blog/001-volume-modem/
web/blog/002-sse-chatroom/
Source files, headers, fonts, recovered binaries on disk, and committed
artifacts under web/ (other than the generated blog) are not touched.
Layout
include/ protocol constants + inline encode/decode/benchmark
src/ five binaries: tx, rx, chat, bt (battle toads), carrier
test/ unit (pure), integration (PA needed), functional (full chain)
blog/ markdown sources + python build
web/ static site: index, kernel, chat, fonts, generated blog
Makefile every workflow lives here
CLAUDE.md agent operating rules for this repo
License
Public domain. Patches gratefully accepted via merge request at
git.unturf.com/engineering/unturf/zebra-report.