docs: compress CLAUDE.md + ticket 0002 diagnosis + cross-refs

CLAUDE.md "Audio jitter buffering" section was the longest in the file
(~70 lines of restated rules). Collapse to a 5-rule list with pointers
to memory + tickets for the deep dives. The old structure restated
content that already lived in feedback_audio_jitter_buffer_strategy
and the per-defect tickets.

Telemetry + tickets sections compressed similarly. Same content,
fewer words, plus explicit pointers so a future session knows where
to look for the canonical version of each rule.

Ticket 0001 grows a Cross-references section pointing back to
CLAUDE.md + memory + the pinning test.

Ticket 0002 (firefox camera) updated with the signal-log diagnosis:
every attempt shows "The request is not allowed by the user agent or
the platform in the current context." = NotAllowedError fired
instantly, no permission prompt. Conclusion: firefox has a remembered
"Block" for the origin. picked.cam= is empty so it isn't the saved-
deviceId case. Lays out two prongs: user-side permission reset
(immediate) and code-side NotAllowedError handling (next pass) so the
page surfaces a user-friendly notice instead of just logging the error
text. Cross-refs to CLAUDE.md telemetry section + the catch site.

docs/tickets/README.md gets a "See also: CLAUDE.md" pointer up top so
the loop closes from the ticket index back to the shared context.

Memory side wired bidirectionally: chromium-decoder-anchor and
audio-jitter-buffer-strategy now point at CLAUDE.md + ticket 0001.
This commit is contained in:
Russell Ballestrini 2026-06-07 20:19:33 -04:00
parent 40de2aa9c0
commit c361a0304f
No known key found for this signature in database
4 changed files with 156 additions and 191 deletions

194
CLAUDE.md
View file

@ -246,167 +246,75 @@ structure, button wiring, CSS, the entry/orientation flow before joining a
space). For everything that lives inside an existing PC, prefer space). For everything that lives inside an existing PC, prefer
`leave` + `enter`. `leave` + `enter`.
### Audio jitter buffering — userland AudioWorklet, not browser hints ### Audio pipeline — load-bearing invariants
**The browser-native jitter-buffer controls cannot be trusted for music.** Five rules. Break any and audio dies in a non-obvious way. Each links
Verified 2026-06-04 with side-by-side telemetry on a Firefox Android phone, to its deep dive.
same PeerConnection, three receivers, identical 4s target:
- `RTCRtpReceiver.playoutDelayHint` is spec'd as a hint — "the user agent MAY 1. **Userland AudioWorklet is the real jitter cushion.** Browser
use this." Browsers do whatever they want. `RTCRtpReceiver.playoutDelayHint` / `jitterBufferTarget` are
- `RTCRtpReceiver.jitterBufferTarget` is spec'd as a hard target. Honored for honored inconsistently (ignored for high-bitrate stereo Opus on
voice-rate Opus and for video. **Ignored for high-bitrate stereo Opus Firefox Android). `JitterBufferProcessor` in
(256 kbps music) on Firefox Android.** The native music-stream code path `web/zebra-spaces.html` queues 128-sample blocks between
inside libwebrtc isn't wired to the new API there. `MediaStreamAudioSourceNode` and `GainNode`. Sticky-started:
tolerate ≥267 ms of empty blocks before re-arming or you re-buffer
on every 2.67 ms micro-stall.
→ `memory: feedback-audio-jitter-buffer-strategy`
Result: a phone listener with the spec'd 4s target had a 0.21s buffer on the 2. **Role-aware target seconds.** Listener 4s
music stream. Any host-side stall (X11 window wiggle, GC pause, encoder (`RECV_PLAYOUT_DELAY_SEC`), speaker / cohost / host 0.7s
spike) was instantly audible. (`SPEAKER_PLAYOUT_DELAY_SEC`). Retarget by posting
`{cmd:'retarget', targetSeconds}` to the worklet port — don't
rebuild. `retargetAllReceivers(role)`. UI gates on the worklet's
`{cmd:'started'}` message so the user isn't shown "connected"
while still buffering.
→ `memory: feedback-4s-playout-validated-cellular-music`
**The reliable cushion is a userland `AudioWorklet`.** See 3. **Chromium decoder anchor is non-negotiable.** Every remote
`JitterBufferProcessor` (inline Blob URL) in `web/zebra-spaces.html`. The `MediaStreamTrack` consumed by a `MediaStreamAudioSourceNode`
worklet sits between `MediaStreamAudioSourceNode` and `GainNode`, queues ALSO needs a hidden muted `<audio>` element with
incoming 128-sample blocks, holds emission until `targetSamples` are `srcObject = stream`, or chromium produces silence (firefox
buffered, then emits with constant delay. The buffer cushions the music unaffected). Lives on the per-uuid node as `node.anchor`;
regardless of what the native receiver does. `setWorkletStream` swaps it alongside the source.
→ `docs/tickets/0001`, `memory: feedback-chromium-decoder-anchor`
Rules that took blood to find: 4. **SFU is the only receive-audio path.** Mesh PCs (`peers`) carry
outbound mic only; inbound mesh audio is ignored. The previous
mesh→worklet swap stalled chromium decoders at the source-swap
moment. If you reintroduce mesh receive, gate the swap on the
mesh receiver's `jitterBufferEmittedCount > 0` BEFORE
disconnecting the SFU source.
→ `docs/tickets/0001` (root cause section)
1. **Sticky-started is non-negotiable.** Once the buffer fills and `started` 5. **HTTP DJ pull (`/stream` on the SFU)** is the AudioWorklet-less
becomes true, do NOT set `started=false` on a single empty-queue tick. fallback. Worklet is primary; HTTP-pull is the safety net.
That tears down playback and forces a full re-buffer (~4s of silence) on
every 2.67ms upstream micro-stall — sounds like constant chopping.
Tolerate ~267ms of consecutive empty blocks (`emptyStreak >=
rearmThresholdBlocks=100`) before re-arming; emit silence in the interim.
2. **Role-aware buffer depth.** Listener gets 4s (lean-back, latency doesn't A/V QoS: audio sender `priority='high'`, video `'low'` — keeps a
matter, ride out wiggle-stalls). Speaker / cohost / host get **0.7s** screen-share keyframe burst from queuing audio packets behind it.
(`SPEAKER_PLAYOUT_DELAY_SEC` in `web/zebra-spaces.html` — bumped from Set hints + jitterBufferTarget anyway at the receiver; they're free
0.5s when we removed the mesh→worklet swap; the worklet cushion now and layer cleanly with the worklet downstream.
absorbs the 200ms host-wiggle unaided on the SFU receive path). → `memory: feedback-av-qos-and-sync`
`playoutDelayForRole(role)` returns the active target.
3. **Worklet retarget on role change**, don't rebuild. Post
`{cmd:'retarget', targetSeconds}` to the worklet's port — recomputes
targetSamples/maxSamples and shrinks the queue if smaller. Audio path
stays continuous; only the buffer depth adjusts. `retargetAllReceivers(role)`
walks every live receiver and applies the new target.
4. **UI gating on buffer-ready.** Worklet posts `{cmd:'started'}` on first
fill. Listener status text sits in "connecting — buffering 4s audio…"
until the first started message lands, then flips to "connected as
listener." Without this, users see "connected" but hear nothing for ~4s
and assume the app is broken.
5. **Bound the queue at 1.5× target** to absorb clock drift without growing
unbounded. Drop oldest on overflow.
6. **Set hints AND target anyway** at `ev.receiver.playoutDelayHint = ...`
and `ev.receiver.jitterBufferTarget = ... * 1000` — they're free, they
work where the browser implements them (video, voice mic), and they
layer cleanly with the userland worklet downstream.
7. **Audio gets `priority='high'` at the sender.** Video gets `'low'`. Stops
a screen-share keyframe burst (e.g. X11 wiggle dirty regions) from
queuing audio packets behind it. `setSenderBitrate` / `setSenderMaxBitrate`
already apply this.
**HTTP DJ pull (`/stream` on the SFU)** exists as a fallback for listeners
who don't have AudioWorklet support — natural deep buffer on the
`<audio>` preload side. Was deadlocking on `bcastMu` until 2026-06-04
(commit fixed `bcastInitCapture.Write` self-recursion). Worklet is the
primary path; HTTP-pull is the safety net.
### Chromium decoder anchor — required for remote audio tracks
**Every remote `MediaStreamTrack` consumed by a
`MediaStreamAudioSourceNode` must ALSO be attached to a hidden muted
`<audio>` element in the same page.** Without that anchor, chromium
does NOT run its WebRTC audio decoder for the track, and the source
node produces silence — even though packets arrive at full rate,
`audioCtx.state === 'running'`, and the worklet's `started` event
fires (queue filled with zeros).
Firefox does NOT have this restriction. Defects of this shape present
as **chrome-only silence with every JS-side invariant green**.
The anchor lives on the `listenerAudioNodes` entry as `node.anchor`
(see `attachAudioStreamViaWorklet`). It is muted, hidden, and
autoplays. `setWorkletStream` swaps `anchor.srcObject` alongside the
source node. `detachListenerStream` tears the anchor down.
Diagnostic signal: pavucontrol Playback shows the chrome stream
present, level meter pinned at 0 while RTP telemetry shows
`aud.recv pkt=N → N+50/sec bytes=growing level=0.000 jbuf=?` and
worklet `started` event has fired. → anchor is the load-bearing
fix. Do NOT remove it during a refactor.
Full root-cause writeup: `docs/tickets/0001-fedora-chrome-cannot-hear-speakers.md`.
Memory: `feedback-chromium-decoder-anchor`.
### Audio receive path — SFU only
**SFU is the only receive-audio path for every role**, including
speakers. Mesh PCs (`peers` map) carry our OUTBOUND mic only. Their
inbound audio is ignored (`pc.ontrack` in `connectToPeer` is a
breadcrumb log only).
This is a 2026-06-07 change. The previous mesh→worklet swap caused
chromium decoders to silently stall at the source-swap moment. The
0.5s → 0.7s `SPEAKER_PLAYOUT_DELAY_SEC` bump compensates for the lost
mesh latency benefit by giving the worklet enough cushion to absorb
the 200ms host-wiggle on the SFU path.
If you reintroduce a mesh receive path, the right shape is
verify-before-swap (poll mesh receiver stats for
`jitterBufferEmittedCount > 0` BEFORE disconnecting the SFU source).
### Telemetry — signal-server CLIENT_LOG ### Telemetry — signal-server CLIENT_LOG
Every `logLine` call in `web/zebra-spaces.html` ships to the Every `logLine` ships to `zebra-spaces-signal` as a `client-log` WS
zebra-spaces-signal server as a `client-log` WebSocket message. The message. Logs land at `/var/log/zebra-spaces-signal.log` on
server logs to `/var/log/zebra-spaces-signal.log` on `proxy.uncloseai.com`.
proxy.uncloseai.com with the publisher's pubkey + handle attached.
Diagnostic SSH (read-only):
```bash ```bash
ssh -i ~/.ssh/digitalocean -p 22222 root@proxy.uncloseai.com \ ssh -i ~/.ssh/digitalocean -p 22222 root@proxy.uncloseai.com \
'tail -2000 /var/log/zebra-spaces-signal.log | grep "actor_handle=\"HANDLE\""' 'tail -2000 /var/log/zebra-spaces-signal.log | grep "actor_handle=\"HANDLE\""'
``` ```
Per-tick (~5s) telemetry line shape: Per-tick (~5 s) shape: `· role=… sub=… pub=… mesh=N … ctx.sink=… aud.recv … vid.recv … mesh.recv u=XXXX … mic.send.aud … cam.send.vid …`
``` One-shot fingerprint on entry: `session: ua=… ctx={sr/baseLat/sinkSupp/sink} devs={out/in/cam} picked={mic/cam/spk}`
· role=X sub=X/X pub=X/X mesh=N sListen=N streamMode=N muted=X xcr=X ctx.sink=…
aud.recv pkt=N lost=N bytes=N jitter=N level=N jbuf=Xs lp=Xs (per SFU sub PC audio receiver)
vid.recv pkt=N lost=N frames=N jbuf=Xs lp=Xs (per SFU sub PC video receiver)
mesh.recv u=XXXX pkt=N … level=N jbuf=Xs (per mesh PC audio receiver)
mic.send.aud / cam.send.vid / scr.send.vid / etc. (outbound stats per publish PC)
```
One-shot session fingerprint on entry (after `joined as X — uuid …`): Live `/version` per service: `cors-proxy.uncloseai.com/zebra-spaces-signal/version` (and `/zebra-spaces-sfu/version`, `/zebra-signal/version`, `/version`). Each returns `{git_commit, build_time, service}` of the upstream Go binary.
```
session: ua=chrome/desktop ctx={sr=48000 baseLat=0.0107 sinkSupp=1 sink=…}
devs={out=N in=N cam=N} picked={mic=… cam=… spk=…}
```
The signal-server endpoint backs `cors-proxy.uncloseai.com`. Per-service
`/version` is exposed at `/zebra-spaces-signal/version` /
`/zebra-spaces-sfu/version` / `/zebra-signal/version` / `/version` (each
returns the upstream Go binary's commit + build_time).
### Defect tracking — docs/tickets/ ### Defect tracking — docs/tickets/
Anything that takes more than 15 minutes or needs cross-browser / Anything > 15 min or needing cross-browser repro → numbered ticket.
cross-session repro lives as a numbered markdown file under Format + index in `docs/tickets/README.md`. Loop:
`docs/tickets/`. Format spec + index in `docs/tickets/README.md`. New telemetry → failing test → fix → `make stamp` → push both repos →
tickets get the next number. The resolution loop is: mark `fixed` with commit link.
1. Reproduce or grab signal-server telemetry.
2. Write a failing test in `test/` that pins the contract.
3. Fix until test passes.
4. `make stamp`, push both repos.
5. Update ticket Status to `fixed`, link commit, note what to
re-test in production.

View file

@ -40,6 +40,15 @@ Pinned by tests in `test/listener-audio-attach.test.js`:
- anchor follows the stream on setWorkletStream - anchor follows the stream on setWorkletStream
- anchor torn down on detach - anchor torn down on detach
## Cross-references
- `../../CLAUDE.md` "Audio pipeline — load-bearing invariants" rules 3
+ 4 (anchor + always-SFU) are sourced from this ticket.
- `feedback-chromium-decoder-anchor` memory carries the
apply-everywhere version of rule 3 so a future session can't
silently undo it.
- `test/listener-audio-attach.test.js` pins the anchor contract.
## Symptom ## Symptom
On fedora chrome, no audio from any speaker in the room is audible. Affects On fedora chrome, no audio from any speaker in the room is audible. Affects

View file

@ -1,71 +1,115 @@
# 0002: fedora firefox cannot share camera # 0002: fedora firefox cannot share camera
**Status:** open **Status:** open (diagnosis pinned; awaiting user-side action + retest with new telemetry)
**Priority:** High (feature unavailable — speakers expect to share camera) **Priority:** High (feature unavailable)
**Surface:** fedora firefox — speaker or host attempting to publish camera **Surface:** fedora firefox — speaker / host attempting to publish camera
**First seen:** 2026-06-06 **First seen:** 2026-06-06
**Reporter:** fox **Reporter:** fox (blanka identity)
## Symptom ## Symptom
On fedora firefox, the "share camera" button does not result in a camera Click "share camera" on fedora firefox → instant `pub.camera: off →
publish. Same firefox can share screen and publish mic — only camera acquiring → off [FAILED]` cycle. Chrome on the same machine shares
publish fails. Same machine + chrome publishes camera fine. camera fine. Mic publish works (so general WebRTC + permission grants
are not all broken).
## Telemetry / Repro ## Telemetry
Not yet captured. Every attempt in `/var/log/zebra-spaces-signal.log` produces the same
verbatim error:
Repro candidates to try: ```
1. Open fedora firefox → enter a room as speaker click: btn-camera-share
2. Click "share camera" pub.camera: off → acquiring [START]
3. Note whether the browser permission prompt appears camera open cancelled: The request is not allowed by the user agent or the platform in the current context.
4. If granted, note whether the camera tile renders locally pub.camera: acquiring → off [FAILED]
5. Grab the in-page signal log + browser console output ```
Things to specifically look for: Captured pre-deploy of the Jun 6 telemetry improvement (no `e.name`,
- `getUserMedia` errors (`NotFoundError`, `NotReadableError`, no `constraints:` suffix). Next attempt on the live page will surface
`OverconstrainedError`, `NotAllowedError`) both, but the error text is already definitive.
- Whether `navigator.mediaDevices.enumerateDevices()` lists the camera
pre-permission (label is empty, deviceId present)
- Whether the camera deviceId saved in `localStorage` (`CAM_DEV_KEY`)
matches a device firefox still recognizes (deviceIds rotate across
sessions on chrome but should be stable on firefox)
## Hypothesis Session fingerprint (latest):
```
session: ua=firefox/desktop ctx={sr=48000 baseLat=0.0000 sinkSupp=0 sink=n/a}
devs={out=1 in=2 cam=1} picked={mic=dmV7HOOY cam= spk=+tR1ptfV}
```
Candidates, none yet confirmed: - `picked.cam=` is **empty** → no saved cameraDeviceId. Not the
OverconstrainedError + stale-id case my Jun 6 auto-recovery
handles.
- `devs.cam=1` → firefox enumerates one camera. Hardware is there.
- Error fires within the SAME SECOND as the click → **no permission
prompt is being shown**.
1. **Saved cameraDeviceId pointing at a stale / removed device.** Firefox ## Diagnosis
would `OverconstrainedError` and the page's catch logs `err`. Fix
would be: on `OverconstrainedError`, retry with `deviceId` unset and
re-prompt the picker — same shape as the mic-label-fallback at
line ~5041.
2. **Firefox v4l2 backend conflicting with chrome's hold on the camera.** Firefox's error text "The request is not allowed by the user agent or
If chrome was using the camera, firefox can't open it (single-consumer the platform in the current context" maps to **`NotAllowedError`**.
on most fedora kernels). User-visible workaround: close chrome's On a fresh request, Firefox shows a permission prompt. The
camera-using tab. instant-fail signature means Firefox has a **remembered "Block"** for
the origin (`www.unturf.com`) on camera — most likely the user (or a
previous version of the page) dismissed the prompt with "Block" +
"Remember this decision," and Firefox auto-denies forever after.
3. **Firefox MediaTrackConstraints quirk** — possibly the page's Mic permission is independent and was granted separately, which is
`videoConstraints` (line ~4493) demanding a resolution / framerate why audio publish works.
the camera doesn't support → `OverconstrainedError`. Fix: relax the
constraints to `{ ideal }` for non-essential fields.
4. **Permission state stuck.** Firefox's permission prompt is gated per This is NOT a code defect — the page's `getUserMedia` call is correct
origin + device; a previous "block" decision can persist. and chrome accepts the identical call. But the page can still help
`about:permissions` would show this. Not a code issue if so. the user recover.
5. **PipeWire / v4l2loopback fakery on the system** — if the camera ## Path forward
device is actually a virtual capture (OBS Virtual Cam etc), firefox
may reject it where chrome accepts. Two prongs:
### Prong 1 — user-side recovery (immediate)
1. Visit `www.unturf.com/zebra-report/` (or wherever the page is
loaded from).
2. Click the **lock icon** in the URL bar → **Connection secure** →
**More information** → **Permissions** tab.
3. Find **Use the camera** → uncheck "Use Default" → select **Allow**
(or **Always Ask**).
4. Reload, click share camera.
Alternative: `about:preferences#privacy` → scroll to **Permissions**
→ **Camera → Settings…** → find the origin → change from Block /
remove the entry.
### Prong 2 — code-side help (next code pass)
When `e.name === 'NotAllowedError'`, the page should surface a
**non-technical user notice** with the recovery steps above, AND
suggest clicking the camera-icon in Firefox's URL bar (Firefox
shows that icon after a blocked permission, click → "Always Allow"
without leaving the page).
Don't auto-retry — Firefox won't change its decision until the user
acts. Show the notice, log the breadcrumb, leave the FSM in `off`.
## Fix ## Fix
(none yet — needs telemetry) Pending. After fox's user-side recovery + retest with the Jun 6
telemetry live (showing `e.name`), I'll add prong 2 to
`sfuPublishCamera`'s catch.
## Cross-references
- CLAUDE.md "Telemetry" section — how to grep the signal-server log
for these events.
- The catch site in `web/zebra-spaces.html` is `sfuPublishCamera`
(search `camera open cancelled`).
- Auto-recovery for `OverconstrainedError + saved deviceId` already
landed Jun 6; this prong covers the orthogonal `NotAllowedError`
case.
## Status notes ## Status notes
- **2026-06-07 (open):** ticket created. Next step is to capture the - **2026-06-07:** diagnosis pinned via signal-server log grep. Five
firefox console / signal log when the share-camera button is clicked identical instant-fail attempts across 4 sessions over 2 days. No
and see which of the hypotheses fits. prompt shown → Firefox saved-block on origin. Awaiting fox
permission reset + retest.
- **2026-06-06:** Jun 6 telemetry improvement (`e.name` + constraints
suffix) deployed. Auto-recovery for the saved-deviceId case shipped
but doesn't apply here (picked.cam is empty).

View file

@ -4,6 +4,10 @@ Defect tracker for zebra-report. Anything that takes more than 15 minutes
or needs to be reproduced across browsers, devices, or sessions lives here or needs to be reproduced across browsers, devices, or sessions lives here
as a numbered markdown file. as a numbered markdown file.
See also: `../../CLAUDE.md` "Audio pipeline — load-bearing invariants"
and "Telemetry — signal-server CLIENT_LOG" for the shared context
every ticket here references.
## File format ## File format
Each ticket is `docs/tickets/NNNN-slug.md` where NNNN is monotonically Each ticket is `docs/tickets/NNNN-slug.md` where NNNN is monotonically