Find a file
Russell Ballestrini e23808d410
zebra-spaces: self-transcribe + Whisper telemetry stats
Two related additions:

1. Self-transcription. The host's own voice wasn't being transcribed
   because capture nodes only attached to listenerAudioNodes (remote
   speakers' streams from SFU/mesh worklets). micStream — the local
   capture for publishing — was never tapped. Added startSelfCapture
   / stopSelfCapture that wraps micStream in a parallel
   whisper-capture worklet. Lines are tagged with myUUID so the local
   user's handle shows in the transcript log.

   Wired into:
   - toggleTranscribe ON/OFF → starts/stops self capture alongside
     remote captures
   - getMic → starts self capture if transcribe is already on (covers
     "user enabled transcribe before granting mic permission")
   - applyMicMode / reacquireMic → stops + restarts self capture
     against the new micStream so we don't keep transcribing a
     stopped MediaStreamTrack

2. Whisper telemetry stats. Each 5s tick now appends:
   xcr=on  sent=N drp=N emt=N avgMs=N wrkr=1 caps=N selfCap=1
     - xcr: on/off
     - sent: chunks shipped to the worker this tick
     - drp:  chunks dropped by the inflight gate (worker saturated)
     - emt:  text lines emitted after silence/hallucination/dup filters
     - avgMs: average inference latency per chunk (device-perf proxy)
     - wrkr: 1 once the worker has loaded the model
     - caps: # of active remote-capture worklets
     - selfCap: 1 if local mic is being captured
   Reset each tick so we see RATE, not cumulative.

   Lets us compare devices across mesh from server-side signal log:
   Snapdragon-8 should show avgMs ~150-300; mid-range phones 1500-
   3000; desktop x86 50-150. drp > 0 means worker can't keep up with
   the audio chunks arriving (multiple speakers talking at once).

Factored the chunk-handling pipeline into handleWhisperChunk(uuid,
chunk) so both remote and self captures share the same filter +
inflight-gate + telemetry path.
2026-06-05 13:21:49 -04:00
blog phase 1: unfirehose reconstruction from session JSONL ingest 2026-05-27 13:51:14 -04:00
include phase 2: gap analysis — cat>> append + binary asset recovery 2026-05-27 13:54:19 -04:00
src add zebrad: pulseaudio→websocket introspector for chat.html 2026-05-27 15:57:22 -04:00
test test/zebra-fsm: MuteFSM transition tests — 14 new cases (102 total) 2026-06-04 14:14:52 -04:00
web zebra-spaces: self-transcribe + Whisper telemetry stats 2026-06-05 13:21:49 -04:00
.gitignore add .gitignore for build artifacts 2026-05-27 14:03:06 -04:00
CLAUDE.md CLAUDE.md: audio jitter buffering section — userland AudioWorklet, not browser hints 2026-06-04 16:59:37 -04:00
Makefile zebra-spaces: formalize self-listener as FSM — pure spec + observer-driven side effects + 12 unit tests 2026-06-04 13:10:29 -04:00
README.md add zebrad: pulseaudio→websocket introspector for chat.html 2026-05-27 15:57:22 -04:00

zebra-report

Covert peer-to-peer chat over PulseAudio sink-input volume. Userland only. No kernel module. No network packets carry the chat content. Public domain.

Build & run

Single entry point: make. Every workflow goes through a target — never invoke gcc or python3 by hand.

make            # builds all five binaries (default target = `all`)
make test       # builds and runs unit tests (pure logic, no PA needed)
make test-all   # unit + integration + functional (PA + sink-inputs required)
make blog       # rebuilds the static blog under web/blog/
make serve      # builds blog, serves web/ on http://127.0.0.1:8765
make clean      # removes binaries and generated blog output

make is the contract. If a workflow isn't a target, add the target before adding the workflow.

Dependencies

gcc, make, pkg-config        # build chain
libpulse-dev                 # PulseAudio client lib (pkg-config: libpulse)
python3                      # blog builder + dev server

Ubuntu / Debian:

sudo apt install build-essential pkg-config libpulse-dev python3

make checks libpulse via pkg-config --cflags libpulse and pkg-config --libs libpulse; if pkg-config can't find it, the build fails early with a clear message.

Targets

make all — six binaries

Binary Source Role
tx src/tx.c reads stdin, transmits via volume modulation
rx src/rx.c reads target sink-input volumes, decodes to stdout
chat src/chat.c bidirectional tx+rx, line-based chat UI
bt src/bt.c "battle toads" dual-channel stereo UART (2× throughput)
carrier src/carrier.c publishes a silent PA sink so volume reads have something to read
zebrad src/zebrad.c PA→WS introspector for web/chat.html

All link against libpulse, librt, and libpthread. Headers come from include/zebra.h (protocol constants) and include/modem.h (inline encode/ decode + benchmark).

make test — unit tests

test/unit                    pure logic, no PA, no audio

Covers: signal encoding (bit_to_vol / vol_to_bit), timing arithmetic (ts_add_ns with overflow), baud math (baud_from_avg_ns), handshake frame build/parse for both OFFER and READY (magic, checksum, corruption).

Currently: 84 cases pass. Required to be green before any commit.

make test-all — integration + functional

Integration and functional tests need a running PulseAudio daemon and real sink-inputs to operate on. They are not run by make test by default.

# integration tests need ONE sink-input
ZEBRA_TEST_SINK=<sink_input_index> ./test/integration

# functional tests need TWO sink-inputs (data + ctrl channels)
ZEBRA_DATA_SINK=<idx> ZEBRA_CTRL_SINK=<idx> ./test/functional

Find sink-input indices with pactl list sink-inputs short — the first column is the index. The test/integration benchmark sub-test reports measured baud for the current host.

To avoid disrupting real audio applications, spawn dedicated silent sink-inputs as test targets:

paplay --raw --format=s16le --rate=44100 --channels=1 \
       --stream-name=zebra-test /dev/zero &
# then use `pactl list short sink-inputs` to find this stream's index

Currently: 17/17 integration pass; 8/10 functional pass. The two functional failures are timing-bound on non-realtime kernels at the auto-negotiated baud and are not code defects (the same data path passes at fixed 50 baud).

make blog & make serve

make blog runs python3 blog/build.py, which reads Markdown sources from blog/posts/ and writes static HTML into web/blog/. make serve rebuilds the blog and serves web/ on port 8765 for local preview.

The chat UI (web/chat.html) is plain static HTML and works directly under make serve — open http://127.0.0.1:8765/chat.html.

make zebrad — PulseAudio → WebSocket introspector

web/chat.html modulates audio output via Web Audio GainNode (mic stays off). A browser tab cannot read another tab's PA state, so to close the receive loop each peer runs zebrad. Decoded frames are forwarded over a local WebSocket; chat.html auto-connects to ws://127.0.0.1:7777.

make zebrad
./zebrad --verbose                  # default: @DEFAULT_MONITOR@, port 7777
./zebrad --source <name> --port 7777

Single C file, no third-party deps beyond libpulse. Embedded WebSocket server: SHA-1 + base64 inline, server→client binary frames only (RFC 6455 opcode 0x82); any inbound data closes the socket (browser auto-reconnects). Bound to 127.0.0.1 only — never accessible from the LAN.

Adaptive baud: starts at ZEBRA_BAUD_HANDSHAKE (50), watches for a READY frame, locks in the negotiated rate from its payload. Peak tracker has 2-second half-life decay so threshold adapts to room volume.

make clean

Removes:

tx rx chat bt carrier
test/unit test/integration test/functional
web/blog/index.html
web/blog/001-volume-modem/
web/blog/002-sse-chatroom/

Source files, headers, fonts, recovered binaries on disk, and committed artifacts under web/ (other than the generated blog) are not touched.

Layout

include/        protocol constants + inline encode/decode/benchmark
src/            five binaries: tx, rx, chat, bt (battle toads), carrier
test/           unit (pure), integration (PA needed), functional (full chain)
blog/           markdown sources + python build
web/            static site: index, kernel, chat, fonts, generated blog
Makefile        every workflow lives here
CLAUDE.md       agent operating rules for this repo

License

Public domain. Patches gratefully accepted via merge request at git.unturf.com/engineering/unturf/zebra-report.