Tech blog
Silence makes medical AI invent clinical conversations
Contents
- What silence actually returns
- Length tells you nothing
- This is not one vendor’s problem
- Asking the model to stay silent does not work
- So we measure the audio instead
- Never let “not measured” read as “fine”
- Put the guard where it cannot be skipped
- Measuring long recordings without decoding them
- Measuring telephone-recorder WAV files
- A WAV file is a container, not a format
- Silence is not zero
- Do not trust the header
- Can the model take μ-law as is?
- Different reasons, different words on screen
- What we would tell another team
This article is also available in Japanese.
When you build clinical documentation on top of a speech model, the frightening failure is not the model being wrong. It is the model being wrong in a way nobody can see.
Give it a recording that contains no sound at all. Intuitively it should say it heard nothing. It does not. It returns a fluent, well-formed clinical conversation that a reader cannot distinguish from a real one.
What silence actually returns
We measured it. We took audio containing no signal whatsoever and ran it through our production documentation pipeline. Repeating identical input returns cached results, so we varied only a name hint to make each run independent.
| Model | Runs | Runs that invented a conversation | Characters generated |
|---|---|---|---|
| Current production model | 8 | 8 of 8 | 410 / 421 / 422 / 434 / 447 / 496 / 603 / 775 |
| Previous production model | 8 | 6 of 6 that completed | 63 / 170 / 171 / 240 / 263 / 414 |
Every run that completed invented a clinical conversation. Here is one, translated from the Japanese, with a dummy patient name. The drug name is real and appeared verbatim in the output.
Mr. Yamada, hello. How have you been feeling lately? — Hello. Lately I keep waking up several times in the night and can’t sleep soundly. — So you’re having middle-of-the-night awakening. What about daytime sleepiness or fatigue? — … You’re currently taking Dayvigo. Do you feel it isn’t working as well?
Note what is present: a presenting complaint, the clinician’s follow-up question, a real prescription drug, and a check on its effect. It reads as an ordinary insomnia follow-up. Nothing in the text suggests it came from silence.
No stage in that pipeline raises an error. Transcription succeeds. Every field of the template is populated. The record saves. The only signal a human has is the memory of never having had that conversation.
Length tells you nothing
The obvious first defence is to distrust short output. It does not work.
Look at the table again. From the same duration of the same silence, one run produced 63 characters and another produced 775. A twelvefold spread. The amount written is essentially unrelated to the input; it reflects how much the model felt like writing on that pass.
So no threshold exists, on character count or on characters per second. Any threshold you pick will flag a terse consultation with a quiet patient and wave through a verbose fabrication.
We tried other discriminators as well.
| Candidate signal | What it looks at | Result |
|---|---|---|
| Output length or density | How much the model wrote | ❌ Same silence yields 63 or 775 characters. Unrelated to input |
| Bytes per second of the file | Encoded data rate | ❌ We hold silent recordings at 3,627 B/s and real speech at 1,072 B/s. The ordering inverts |
| Peak amplitude alone | Loudest sample in the file | ❌ A single click can push a silent recording to nearly full scale |
| Share of time below a level | Portion under −35 dBFS | ❌ Real but soft speech also measures as 100% silent |
| Switching models | Use a different provider | ❌ Both the current and previous models fabricated on every completed run |
| Failing thin output wholesale | Error out when output is short | ❌ On real recordings this caught a meaningful number of legitimate short notes |
This is not one vendor’s problem
The phenomenon is documented repeatedly across speech AI. Every figure below is taken from the primary source.
| Source | What was measured | Result |
|---|---|---|
| Barański et al. 2025, AGH Kraków | 301,317 non-speech clips through Whisper | 40.3% produced text |
| Same paper | Real speech with silence appended | 17.1% hallucinated. Padding alone causes it |
| Koenecke et al. 2024, ACM FAccT | 13,140 speech segments from AphasiaBank (speakers with aphasia and a control group) | 187 segments reliably hallucinated; 38% of hallucinations were harmful: violence, made-up names and health conditions, fake website links |
| Calm-Whisper 2025, Interspeech | 8,732 environmental sound clips | Whisper emitted text 99.97% of the time. It returned empty three times |
| When Silence Matters 2025, NTU | Five seconds of unrelated silence appended for audio language models | Accuracy fell consistently. Silence is not a neutral input |
Clinical harm has followed. In May 2026 the Auditor General of Ontario procurement-tested 20 approved AI scribe vendors on simulated consultations and found inaccuracies in all twenty. Nine invented treatment plans that were never discussed. Twelve recorded medications different from what the physician prescribed.
In August 2026 ABC reported that an Australian patient’s post-operative letter from her specialist to her GP had acquired a drug history she had never had; she had noticed it in March. The part that matters most is not the fabrication. It is that the clinician could not determine where it came from. Without the original audio there is nothing left to check against.
So we do not delete the audio. When a record looks wrong, being able to listen again is the last line of defence against this class of error.
Asking the model to stay silent does not work
The next instinct is to instruct the model: if you hear nothing, return nothing. This has been tested and disproven, from three independent directions.
- With Whisper’s initial prompt, telling it not to invent words causes the instruction itself to appear in the transcript. The prompt enters the decoder as context, so you have simply given it one more thing to repeat.
- Measured on audio language models (NTU 2025), mitigation prompts improved results inconsistently and made some conditions worse. The paper concludes that a single explicit instruction is insufficient to counteract cross-modal interference systematically.
- Work on audio hallucination attacks found that chain-of-thought prompting helped against explicit attacks but not implicit ones, and in one setting raised the attack success rate from 68.74% to 82.90%. What helped substantially was training-time alignment (DPO).
Do not delegate hallucination safety to a prompt. Prompts get edited. Models get swapped. A defence that lives in the instruction text is not a defence.
So we measure the audio instead
We stopped trying to recognise fabrication and started making sure the model never sees silence. Front-end voice activity detection is also the strongest single mitigation in the literature: Whisper’s hallucination rate drops from 21.3% to 0.2% with Silero VAD in front of it.
Our rule is deliberately narrow.
Total, not longest run. Our first implementation required one continuous second above the threshold. Run a genuine but heavily attenuated conversation through that and each word falls short of a second, so real speech is judged silent. Conversation is intermittent by nature; requiring continuity penalises the people who speak quietly.
The detector itself is a short function. It takes the RMS of one millisecond at a time, adds up the time spent above the threshold, and stops reading the moment the total reaches one second.
export const SILENCE_LEVEL_DBFS = -50; // floor for "voiced"
export const SILENCE_MIN_VOICED_SEC = 1.0; // total time above it
const threshold = Math.pow(10, SILENCE_LEVEL_DBFS / 20); // dBFS to amplitude
const win = Math.round(sampleRate * SILENCE_RMS_WINDOW_MS / 1000);
const need = Math.round(sampleRate * SILENCE_MIN_VOICED_SEC);
for (let i = 0; i + win <= pcm.length; i += win) {
let sum = 0;
for (let j = i; j < i + win; j++) { const v = pcm[j]; sum += v * v; }
if (Math.sqrt(sum / win) > threshold) { // this window is voiced
voicedSamples += win;
if (voicedSamples >= need) return true; // one second total, stop here
}
}The same function runs in the browser during recording and on the server after upload — fed from an AudioWorklet in one case and from the decoder in the other. A check in CI pins both the constants and the hash of that core block so the two cannot drift apart.
The two constants, −50 dBFS and one second in total, come from measurement rather than intuition. We tested four questions separately.
| Question | What we measured |
|---|---|
| Is −50 dBFS low enough? | Silent files never cross it (peak −68.7 dBFS at most; isolated clicks cross but never last a second) |
| Does it catch quiet speech? | The real recording with the least voiced time still had 11.6 seconds. There is a gap, not a gradient |
| Is one second the right length? | It is the shortest length that still classifies a single click (peak −0.089 dBFS) as silence |
| Could we be stricter? | Yes, and there is no reason to. The quietest real speech peaks at −15.2 dBFS, leaving 35 dB of headroom |
Against our sample set: 20 of 20 silent recordings caught, 0 of 85 real recordings flagged. The guard is meant to catch recordings with no signal, and it must not go looking for quiet ones.
Later, short solo dictations for nursing records by soft-spoken staff did fall short of this line and were stopped. Our sample set had no recordings that short and that quiet. For those recordings only, we lowered the line and added a cap on transcript characters per second, so fabrication is still stopped.
Never let “not measured” read as “fine”
The verdict has three values, not two: voiced, silent, and unmeasured.
type SilenceVerdict = 'voiced' | 'silent' | 'unmeasured';
type AudioCheck =
| { verdict: 'voiced' } // goes to the model
| { verdict: 'silent'; reason: 'no_voiced_run' } // stopped
| { verdict: 'unmeasured'; reason: 'unsupported_format' // never actually measured
| 'no_audio_track'
| 'decode_failed'
| 'budget_exhausted' };Storing the reason next to the verdict is what makes “how many unsupported containers did we see this month” a query rather than an investigation.
Unsupported container, no audio track, a decode failure, the scan budget running out — collapse these into “silent” and a recording that may hold a conversation is filed as silence; collapse them into “voiced” and the model receives something you never checked. Both are wrong, so we added the third value, hold the file back from the model, and record the reason alongside it. Afterwards it is possible to tell an unsupported format from an exhausted budget.
When two different causes produce the same outcome, nobody can ever trace back to the cause.
Put the guard where it cannot be skipped
One more design point mattered as much as the rule itself.
Five separate code paths fetched audio, one per record type. Writing a measureAudio() helper and calling it from all five means someone eventually forgets one call — and nothing reveals it until that record type produces a fabricated note.
When the same shape exists five times, fixing one leaves four copies as templates for the next change. What multiplies by copying gets fixed by copying too, and one copy is always missed. We have watched layered safeguards all miss the same single path when we measured how far our own checks actually reached.
So we collapsed the five fetches into a single function. Audio cannot be read without passing through it.
When the verdict is silent we skip the model entirely, leave the record empty, and keep the audio. Staff are told the recording did not capture sound, and they can listen for themselves. We would rather have an absent record than an invented one.
Measuring long recordings without decoding them
Fully decoding a forty-minute recording just to answer one question is too slow, so we sample across the timeline. Two things went wrong here, and both are worth passing on.
First, where you sample matters more than how much. Our initial design decoded one continuous second at a fixed stride. Tested against a deliberately hard case — a 62-minute recording containing just 15 seconds of speech — it returned “silent” on three runs out of five. With a single window, if that one second falls in a gap, you miss.
Splitting the same budget into five windows of 0.2 seconds fixed it. Same total audio decoded, five out of five correct.
The whole policy reduces to deciding which frames to decode. The stride is chosen at runtime from the estimated duration and the remaining budget:
const SAMPLE_WINDOW_FRAMES = 50; // 20ms x 50 = one second of audio per cycle
const SAMPLE_SUB_WINDOWS = 5; // placed as five separate 0.2s windows
export function shouldDecodeFrame(seen: number, stride: number): boolean {
if (stride <= 1) return true; // no need to skip, read it all
const sub = Math.round(SAMPLE_WINDOW_FRAMES / SAMPLE_SUB_WINDOWS); // 0.2s = 10 frames
return (seen % (stride * sub)) < sub; // read 0.2s, skip the gap
}False positives did not increase: a 62-minute pure silence and a 1h42m entirely silent recording both stayed silent, and across 89 real recordings not one verdict changed. Worst-case runtime was 1,343ms against a 1,500ms budget.
| Sampling strategy | Audio decoded | 62 min recording, 15 s of speech | Pure silence (negative control) |
|---|---|---|---|
| One 1-second window | same | 2 of 5 runs correct | correctly silent |
| Five 0.2-second windows | same | 5 of 5 runs correct | correctly silent |
Second, the guard we added created a new way to lose data. We capped how much audio the scanner would read. In our runtime, however, a blob’s stream yields the entire blob as a single chunk, so the cap tripped on chunk one and any recording above the cap became unreadable in its entirety. The larger the recording, the more certain the loss.
We caught it before release, rewrote the reader to slice fixed-size ranges, and added tests using files just under and just over the limit. Then we deliberately broke each guard to confirm the tests actually turn red.
Every guard you add should come with one condition under which it fails. If you cannot name that condition, you have not finished designing the guard.
Measuring telephone-recorder WAV files
A hospital that records phone calls on a dedicated device asked us whether the files that device exports could go through the same flow as every other recording. We started by asking how they work today: export a WAV from the device, upload it to MENTRA. That is the whole procedure.
The files turned out to be shaped differently from microphone recordings: G.711 μ-law, stereo, 8 kHz, 8-bit. It is the codec telephone networks use, and a single byte means something entirely different from a byte of Opus or 16-bit PCM.
The guard in this article is “if we cannot measure it, the model does not see it”. That leaves exactly one way to let more recordings through: teach the reader more formats, and make sure it measures each of them correctly.
| Before | Now | |
|---|---|---|
| What the hospital does | Export from the device, upload | The same — nothing changes |
| Telephone WAV (G.711) | Treated as unmeasurable and held back from the model, on the safe side | Decoded properly, measured, and sent to the model when there is speech |
| Silent recordings | Stopped | Stopped — the guard is unchanged |
| Screen when a format cannot be read | Same message as silence: “could not transcribe the recording” | “Could not read the audio file”, with a pointer to support |
A WAV file is a container, not a format
“WAV” names the container. The shape of the audio inside is declared by numbers in the header, and the reader has to start there.
Each format tag changes how a byte is read. What we added were the two telephone codecs and the extensible format.
| Format tag | Content | Sample size | Silence is encoded as |
|---|---|---|---|
| 1 | PCM (integer) | 8, 16, 24, 32 bit | 0x80 for 8-bit, 0 otherwise |
| 3 | IEEE float | 32 bit | 0.0 |
| 6 | G.711 A-law (telephony; Europe and elsewhere) | 8 bit | 0xD5 / 0x55 |
| 7 | G.711 μ-law (telephony; North America and Japan) | 8 bit | 0xFF / 0x7F |
| 0xFFFE | WAVE_FORMAT_EXTENSIBLE | depends | depends on the sub-format |
The extensible format names the real encoding with a 16-byte GUID after the basic fmt fields. Its first two bytes correspond to the format tags above, so we map it onto 1, 3, 6 or 7.
Silence is not zero
This is the easiest place to get it wrong. G.711 compands the signal — fine steps for quiet sounds, coarse steps for loud ones — into an 8-bit code. As a result, the byte that means silence is not zero.
What happens if you read those codes as ordinary 8-bit PCM (unsigned, centred on 0x80)? We computed it with the same formula the implementation uses.
| Byte | What it encodes | Read as 8-bit PCM | Read as G.711 |
|---|---|---|---|
| 0xFF | μ-law silence | −0.1 dBFS (almost full scale) | 0 (silence) |
| 0x7F | μ-law silence, other sign | −42.1 dBFS (above our −50 dBFS threshold) | 0 (silence) |
| 0xD5 | A-law silence | −3.6 dBFS | −72.2 dBFS (silence) |
| 0x55 | A-law silence, other sign | −9.5 dBFS | −72.2 dBFS (silence) |
All four become “voiced” when read as PCM. Sustain that for one second and a silent recording reaches the model. One wrong assumption about byte meaning switches the entire guard off. “Just read it as 8-bit” looks like a fix and is the most dangerous one available.
So G.711 is only ever read through a lookup table that maps all 256 codes to amplitude. The table is built once at start-up.
// G.711 8-bit code -> amplitude in -1..1. Format tag 6 = A-law, 7 = μ-law
const G711_MULAW = new Float32Array(256);
const G711_ALAW = new Float32Array(256);
for (let i = 0; i < 256; i++) {
const u = ~i & 0xff; // μ-law codes are defined bit-inverted
const mu = (((u & 0x0f) << 3) + 0x84) << ((u >> 4) & 0x07);
G711_MULAW[i] = ((u & 0x80) ? -(mu - 0x84) : (mu - 0x84)) / 32768;
const a = i ^ 0x55; // A-law inverts the even bits
const e = (a >> 4) & 0x07;
const m = (a & 0x0f) << 4;
const al = e === 0 ? m + 8 : (m + 0x108) << (e - 1);
G711_ALAW[i] = ((a & 0x80) ? al : -al) / 32768;
}The table is tested against silence generated by a separate G.711 reference encoder written independently of this code: ten seconds of stereo at 0xFF, 0x7F, 0xD5 and 0x55. All four are classified as silent.
Do not trust the header
A header is only what the exporting device claims. Real WAV files carry values that disagree with the arithmetic in the spec. For every field, we first asked what breaks if we believe it, then decided how to read it.
| Field | What breaks if you trust it | What we do |
|---|---|---|
| Block align | Files whose declarations disagree (for example, claiming 24-bit while using 4 bytes per sample) do not match channels × bytes per sample. Step by the arithmetic and every frame after the first is misread | Step by the declared width, and place each channel at width ÷ channels. When that slot is wider than the sample, infer from the content whether the padding sits above or below it (zero bytes, or sign extension); until that is settled, take the quieter of the two readings. Stop if neither fits |
| Sample rate | “One second in total” is counted in samples. Too low and a single click becomes a second of speech; too high and a whole conversation is shorter than a second | Below 1,000 Hz or above 768,000 Hz is “unmeasured” |
| Extensible GUID | Mapping on the first two bytes alone reads vendor-specific formats (often compressed audio) as PCM and takes noise for speech | Map only when all 16 bytes match the standard GUID |
| Data chunk length | Trailing metadata is read as audio. The four letters “LIST” decode as μ-law to roughly −29 dBFS | Stop at the end of the data chunk |
| Measured length | Under one second, a recording can never reach “one second of speech in total” | Do not call it silent; call it “unmeasured” |
Each of these can fail in two directions. Taking silence for speech switches off the fabrication guard; taking speech for silence throws a record away. Neither is acceptable, so an impossible value becomes “unmeasured”: the model does not see the file, and the reason is recorded.
const WAV_MIN_SAMPLE_RATE = 1000; // covers 8 kHz telephony and 16-192 kHz audio
const WAV_MAX_SAMPLE_RATE = 768000; // a high rate that real hardware supports
if (sampleRate < WAV_MIN_SAMPLE_RATE || sampleRate > WAV_MAX_SAMPLE_RATE
|| blockAlign < bytesPerSample * channels) {
return unreadable('bad_header'); // "unmeasured" plus the cause
}
const frameBytes = blockAlign; // frame width is whatever the header declaresWhen a file cannot be read, the cause (one of seven, such as the format tag, the bit-depth combination or the padding position) and the header values go into the log. Those are format facts only, never audio content. The next unfamiliar WAV tells us which field stopped it without anyone opening the recording.
Can the model take μ-law as is?
Measuring the audio correctly left one more question: does the transcription model hear μ-law correctly without conversion? If not, the server would have to re-encode it to PCM first.
We ran a real telephone recording of about 17 minutes through our staging environment both ways.
| Input | Size | Transcript | Speaker turns | Processing time |
|---|---|---|---|---|
| μ-law as exported | 16.0 MB | 6,004 characters | 97 | 43 s |
| Converted to 16-bit PCM | 32.0 MB | 6,004 characters | 97 | 41 s |
The transcript text and the speaker segmentation matched down to the md5 hash. Had the model taken μ-law as noise, the content would have changed. Because it did not, we concluded that the model hears μ-law the same way it hears PCM, and we do not convert. Conversion would double the file and add a step. A-law and extensible-format files, built from synthesized speech, also completed transcription in staging.
Different reasons, different words on screen
When we tell staff that a recording was stopped, a different reason gets a different message. If a format problem is reported as “the microphone may not have picked up any sound”, people go and check the microphone and never reach the real cause.
All five record types — consultation, nursing, interview, welfare visit and minutes — take these messages from a single table. Writing them per record type is how one of the five ends up forgotten. It is the same idea as the single audio entry point above, applied to words on a screen.
What we would tell another team
Two questions decide whether this class of error is survivable in your system. Can someone listen to the original audio afterwards? And when a record ends up empty, does anyone find out? If both answers are yes, every mistake remains traceable.
MENTRA is a medical AI built for psychiatry and psychosomatic medicine in Japan. All five of our record types — consultation notes, nursing records, interview records, welfare visit records and conference minutes — read audio through a single entry point that runs this check every time, and a scheduled job reports the results daily.
The full Japanese version of this article, with the same measurements, is here. If you are working on clinical documentation and want to compare notes on any of this, we would be glad to hear from you — get in touch.
Frequently asked questions
Does the guard also block quiet consultations or soft-spoken patients?
It stops recordings with no signal, or with under one second of voiced time in total. Our real speech samples had at least 11.6 seconds. Short solo nursing dictations by soft-spoken staff were sometimes stopped; for those only, we lowered the line and cap characters per second.
How much latency does measuring the audio add?
Tens of milliseconds in practice. Long recordings are never fully decoded. We sample across the timeline and stop reading the moment we find enough voiced audio, so a normal recording is decided within the first fraction of it.
What happens when the audio cannot be measured at all?
It becomes a third state, not a pass or a fail, and the model does not see the file. Unsupported container, no audio track, decode failure and budget exhaustion are each recorded with their reason, so the cause can be traced later. Collapsing it into either answer hides whether the check ran.
Can recordings exported from a telephone recorder be used?
Yes. G.711 WAV files (μ-law and A-law), the codec telephone systems use, are measured like any other recording before the model sees them. The exported file is uploaded as is, with no conversion. If you are unsure what your device exports, get in touch.
Why not just detect the fabrication afterwards?
Because the fabrication is fluent, complete and internally consistent. It fills every field of the template and passes every format check. Detection has to happen where the evidence still exists, which is the audio, not the text.
