Russell Ballestrini
9b91c726e5
Fox 2026-06-05: "the web page for zebra spaces seems noticeably slower
after enabling transcribe… even the tones for entering and leaving are
showing up way way later even on the host side."
Root cause: ONNX Runtime via transformers.js was running on the main
thread, blocking JS for 1–3 seconds per chunk. Join/leave chimes,
button clicks, scroll, EVERY UI gesture queues behind it.
Two changes:
1. Move Whisper to a dedicated module Web Worker. The worker imports
transformers.js + loads the whisper-tiny.en pipeline ONCE; each
chunk is transferred (zero-copy) via postMessage, processed in
isolation from the UI thread, and the resulting text is posted
back. Main thread is free during inference now — UI stays
responsive. Worker is created on first toggle ON; same ~40MB
model download still happens, just off-thread.
2. Global "inflight gate" on the main thread side. Only one
transcribe request can be in flight at a time. If a new chunk
arrives while busy, DROP it (don't queue). Counter is logged
every ~30s so we can see worker saturation. With N speakers all
talking at once, dropping is correct — stale chunks from 10s ago
aren't worth transcribing.
Combined with the RMS silence gate in the capture worklet, the result
is: silent chunks never even reach the main thread, busy chunks are
processed sequentially by the worker, and the UI never blocks.
Phase 4 (deferred): switch to WebGPU backend for ONNX where supported
— roughly 2-5× faster than WASM on capable devices. transformers.js
v3 supports this with `{ device: 'webgpu' }` in pipeline opts.
|
||
|---|---|---|
| blog | ||
| include | ||
| src | ||
| test | ||
| web | ||
| .gitignore | ||
| CLAUDE.md | ||
| Makefile | ||
| README.md | ||
zebra-report
Covert peer-to-peer chat over PulseAudio sink-input volume. Userland only. No kernel module. No network packets carry the chat content. Public domain.
Build & run
Single entry point: make. Every workflow goes through a target — never
invoke gcc or python3 by hand.
make # builds all five binaries (default target = `all`)
make test # builds and runs unit tests (pure logic, no PA needed)
make test-all # unit + integration + functional (PA + sink-inputs required)
make blog # rebuilds the static blog under web/blog/
make serve # builds blog, serves web/ on http://127.0.0.1:8765
make clean # removes binaries and generated blog output
make is the contract. If a workflow isn't a target, add the target before
adding the workflow.
Dependencies
gcc, make, pkg-config # build chain
libpulse-dev # PulseAudio client lib (pkg-config: libpulse)
python3 # blog builder + dev server
Ubuntu / Debian:
sudo apt install build-essential pkg-config libpulse-dev python3
make checks libpulse via pkg-config --cflags libpulse and
pkg-config --libs libpulse; if pkg-config can't find it, the build fails
early with a clear message.
Targets
make all — six binaries
| Binary | Source | Role |
|---|---|---|
tx |
src/tx.c |
reads stdin, transmits via volume modulation |
rx |
src/rx.c |
reads target sink-input volumes, decodes to stdout |
chat |
src/chat.c |
bidirectional tx+rx, line-based chat UI |
bt |
src/bt.c |
"battle toads" dual-channel stereo UART (2× throughput) |
carrier |
src/carrier.c |
publishes a silent PA sink so volume reads have something to read |
zebrad |
src/zebrad.c |
PA→WS introspector for web/chat.html |
All link against libpulse, librt, and libpthread. Headers come from
include/zebra.h (protocol constants) and include/modem.h (inline encode/
decode + benchmark).
make test — unit tests
test/unit pure logic, no PA, no audio
Covers: signal encoding (bit_to_vol / vol_to_bit), timing arithmetic
(ts_add_ns with overflow), baud math (baud_from_avg_ns), handshake frame
build/parse for both OFFER and READY (magic, checksum, corruption).
Currently: 84 cases pass. Required to be green before any commit.
make test-all — integration + functional
Integration and functional tests need a running PulseAudio daemon and
real sink-inputs to operate on. They are not run by make test by default.
# integration tests need ONE sink-input
ZEBRA_TEST_SINK=<sink_input_index> ./test/integration
# functional tests need TWO sink-inputs (data + ctrl channels)
ZEBRA_DATA_SINK=<idx> ZEBRA_CTRL_SINK=<idx> ./test/functional
Find sink-input indices with pactl list sink-inputs short — the first
column is the index. The test/integration benchmark sub-test reports
measured baud for the current host.
To avoid disrupting real audio applications, spawn dedicated silent sink-inputs as test targets:
paplay --raw --format=s16le --rate=44100 --channels=1 \
--stream-name=zebra-test /dev/zero &
# then use `pactl list short sink-inputs` to find this stream's index
Currently: 17/17 integration pass; 8/10 functional pass. The two functional failures are timing-bound on non-realtime kernels at the auto-negotiated baud and are not code defects (the same data path passes at fixed 50 baud).
make blog & make serve
make blog runs python3 blog/build.py, which reads Markdown sources from
blog/posts/ and writes static HTML into web/blog/. make serve rebuilds
the blog and serves web/ on port 8765 for local preview.
The chat UI (web/chat.html) is plain static HTML and works directly under
make serve — open http://127.0.0.1:8765/chat.html.
make zebrad — PulseAudio → WebSocket introspector
web/chat.html modulates audio output via Web Audio GainNode (mic stays off).
A browser tab cannot read another tab's PA state, so to close the receive
loop each peer runs zebrad. Decoded frames are forwarded over a local
WebSocket; chat.html auto-connects to ws://127.0.0.1:7777.
make zebrad
./zebrad --verbose # default: @DEFAULT_MONITOR@, port 7777
./zebrad --source <name> --port 7777
Single C file, no third-party deps beyond libpulse. Embedded WebSocket
server: SHA-1 + base64 inline, server→client binary frames only (RFC 6455
opcode 0x82); any inbound data closes the socket (browser auto-reconnects).
Bound to 127.0.0.1 only — never accessible from the LAN.
Adaptive baud: starts at ZEBRA_BAUD_HANDSHAKE (50), watches for a READY
frame, locks in the negotiated rate from its payload. Peak tracker has
2-second half-life decay so threshold adapts to room volume.
make clean
Removes:
tx rx chat bt carrier
test/unit test/integration test/functional
web/blog/index.html
web/blog/001-volume-modem/
web/blog/002-sse-chatroom/
Source files, headers, fonts, recovered binaries on disk, and committed
artifacts under web/ (other than the generated blog) are not touched.
Layout
include/ protocol constants + inline encode/decode/benchmark
src/ five binaries: tx, rx, chat, bt (battle toads), carrier
test/ unit (pure), integration (PA needed), functional (full chain)
blog/ markdown sources + python build
web/ static site: index, kernel, chat, fonts, generated blog
Makefile every workflow lives here
CLAUDE.md agent operating rules for this repo
License
Public domain. Patches gratefully accepted via merge request at
git.unturf.com/engineering/unturf/zebra-report.