Russell Ballestrini
e23808d410
Two related additions:
1. Self-transcription. The host's own voice wasn't being transcribed
because capture nodes only attached to listenerAudioNodes (remote
speakers' streams from SFU/mesh worklets). micStream — the local
capture for publishing — was never tapped. Added startSelfCapture
/ stopSelfCapture that wraps micStream in a parallel
whisper-capture worklet. Lines are tagged with myUUID so the local
user's handle shows in the transcript log.
Wired into:
- toggleTranscribe ON/OFF → starts/stops self capture alongside
remote captures
- getMic → starts self capture if transcribe is already on (covers
"user enabled transcribe before granting mic permission")
- applyMicMode / reacquireMic → stops + restarts self capture
against the new micStream so we don't keep transcribing a
stopped MediaStreamTrack
2. Whisper telemetry stats. Each 5s tick now appends:
xcr=on sent=N drp=N emt=N avgMs=N wrkr=1 caps=N selfCap=1
- xcr: on/off
- sent: chunks shipped to the worker this tick
- drp: chunks dropped by the inflight gate (worker saturated)
- emt: text lines emitted after silence/hallucination/dup filters
- avgMs: average inference latency per chunk (device-perf proxy)
- wrkr: 1 once the worker has loaded the model
- caps: # of active remote-capture worklets
- selfCap: 1 if local mic is being captured
Reset each tick so we see RATE, not cumulative.
Lets us compare devices across mesh from server-side signal log:
Snapdragon-8 should show avgMs ~150-300; mid-range phones 1500-
3000; desktop x86 50-150. drp > 0 means worker can't keep up with
the audio chunks arriving (multiple speakers talking at once).
Factored the chunk-handling pipeline into handleWhisperChunk(uuid,
chunk) so both remote and self captures share the same filter +
inflight-gate + telemetry path.
|
||
|---|---|---|
| .. | ||
| blog | ||
| fonts | ||
| chat.html | ||
| host-your-own.html | ||
| how-it-works.html | ||
| index.html | ||
| kernel.html | ||
| stamp.js | ||
| zebra-audio.html | ||
| zebra-spaces.html | ||