engineering/zebra-report
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

zebra-spaces: filter Whisper silence-hallucinations (RMS gate + stop-word skip)

Browse source 
Fox 2026-06-05: "fxhp and fxhp-chrome both are triggering clients to
transcode `you` on a new line over and over could you sort that out?
… this happens even when mic is closed."

whisper-tiny.en is notorious for hallucinating common stop-words on
silent / low-energy audio: "you", "thank you", "thanks for watching",
a lone period. Mic muted or peer dead-silent → the model still
produces a chunk of audio (silence frames from Opus / WebRTC) and
spits one of these phrases out. Two filters:

1. RMS-based silence gate IN the capture worklet (cheaper — no
   pipeline invocation at all). Compute RMS of the 5s/16kHz chunk
   before posting; skip if below 0.005 (~-46 dBFS, well below any
   real speech). The vast majority of "mic-closed" hallucinations
   stop here.

2. JS-side filter against a known-hallucination set. Catches the
   rest (quiet-room ambient that passes the RMS gate). Normalizes
   lowercase + strips trailing punctuation so "You." / "you " /
   "YOU!" all collapse to "you" and match the set. Easy to extend
   as new hallucinations are observed.

3. Per-uuid consecutive-duplicate suppression. Even non-hallucination
   text sometimes re-emits the same short phrase across consecutive
   chunks ("uh huh" "uh huh") — drop the dupe.

CPU win: silent chunks no longer go through ONNX inference (~50-200ms
per chunk on tiny.en).
This commit is contained in:
Russell Ballestrini 2026-06-05 13:06:54 -04:00
parent 227588f147
commit e2f279d57a
No known key found for this signature in database
1 changed files with 49 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

55
web/zebra-spaces.html
View file

@ -2048,8 +2048,23 @@ class WhisperCaptureProcessor extends AudioWorkletProcessor {
}
this.decimateCounter = (this.decimateCounter + ch.length) % this.decimationFactor;
if (this.bufPos >= this.chunkTargetSamples){
const chunk = this.buf.slice(0, this.bufPos);
this.port.postMessage({ chunk }, [chunk.buffer]);
/* RMS-based silence gate. whisper-tiny.en hallucinates common
* stop-words ("you", "thank you", "thanks for watching", ".")
* when fed silence — even with the mic closed. Skip chunks
* whose energy is below the speech threshold entirely. Saves
* CPU AND suppresses the hallucinated lines.
* Fox 2026-06-05: "this happens even when mic is closed." */
let sumSq = 0;
for (let i = 0; i < this.bufPos; i++){
const s = this.buf[i];
sumSq += s * s;
}
const rms = Math.sqrt(sumSq / this.bufPos);
const SILENCE_THRESHOLD = 0.005; /* ~ -46 dBFS, well below speech */
if (rms >= SILENCE_THRESHOLD){
const chunk = this.buf.slice(0, this.bufPos);
this.port.postMessage({ chunk }, [chunk.buffer]);
}
this.bufPos = 0;
}
return true;
@ -2104,6 +2119,26 @@ async function ensureWhisperPipeline(){
}
let transcribeEnabled = false;
/* whisper-tiny.en hallucinations on silence / low-energy chunks. The
* RMS gate in the capture worklet catches dead silence; this set
* catches what gets past it — quiet-room ambient with the same
* spurious outputs ("you", "thanks for watching", "thank you.", a
* lone period). Lowercased + trimmed of trailing punctuation for
* matching. List grows as new hallucinations are observed. */
const WHISPER_HALLUCINATIONS = new Set([
'you', 'thank you', 'thanks', 'thanks for watching',
'thanks for watching!', 'thank you for watching',
'thank you for watching.', 'thanks for watching.',
'bye', 'okay', 'mm', 'mhm', 'uh huh', 'ah', 'oh',
'um', 'uh', 'hmm', 'yeah', 'yes', 'no', 'ok', 'so',
'and', 'the', 'a', 'i', 'it', 'we',
'.', '..', '...', '!', '?'
]);
/* per-uuid dedupe of the most recent emitted line. whisper sometimes
* re-emits the same short phrase across consecutive chunks even when
* the speaker has paused — looks like spam in the log. */
const _lastTranscriptByUuid = new Map();
function appendTranscriptLine(uuid, text){
const log = document.getElementById('transcript-log');
if (!log) return;
@ -2142,8 +2177,16 @@ async function startCaptureForUuid(uuid){
try {
const result = await pipeline(e.data.chunk, { sampling_rate: 16000 });
const txt = (result && result.text ? result.text : '').trim();
/* skip empty / placeholder transcriptions ("." " " "[BLANK_AUDIO]" etc.) */
if (txt.length >= 2 && !/^\[/.test(txt)) appendTranscriptLine(uuid, txt);
if (txt.length < 2) return;
if (/^\[/.test(txt)) return; /* "[BLANK_AUDIO]" etc. */
/* normalize for hallucination match: lowercase, strip trailing
* punctuation. "You." / "you" / " YOU!" all collapse to "you". */
const norm = txt.toLowerCase().replace(/[.!?,;:\s]+$/,'').trim();
if (WHISPER_HALLUCINATIONS.has(norm)) return;
/* consecutive-duplicate suppression per speaker */
if (_lastTranscriptByUuid.get(uuid) === txt) return;
_lastTranscriptByUuid.set(uuid, txt);
appendTranscriptLine(uuid, txt);
} catch (err){
logLine('err', 'whisper transcribe '+uuid.slice(0,4)+': '+err.message);
}
@ -7117,8 +7160,8 @@ logLine('', 'ready — pick a handle, type a rendezvous code, enter the space');
<footer style="margin:2.2rem auto 0;font-size:0.65rem;color:#999;line-height:1.7;word-break:break-all;font-family:monospace">
<span id="pi-seal" style="color:#777;cursor:default;user-select:none" title="">page integrity</span> &nbsp;·&nbsp; built <span class="stamp-date">2026-06-05</span><br>
md5 <span class="stamp-md5">66fd5efdb6e64d748dbca14181864f64</span><br>
sha256 <span class="stamp-sha">2216311e5dacc548b5c5364b10acce241b5baaa7c2068ca169e749478d5e5fe6</span><br>
md5 <span class="stamp-md5">2773c799f927f392736fa437c9af62e2</span><br>
sha256 <span class="stamp-sha">c4fae24ca1a2a75c5893c4b4498cac724bdaddeb9ba351f758f8aafe3c96df77</span><br>
<span style="color:#bbb">hashes are of this page with these two fields zeroed — to verify, blank them and re-hash</span><br>
<span style="color:#bbb">one self-contained file — <strong>save a copy</strong> and verify against these hashes; point at your own servers with ?signal= and ?turncred=, or <a href="host-your-own.html" style="color:#999">host your own community</a></span>
</footer>