'99 % accurate' means nothing: how to read a transcription accuracy claim, and how to catch the errors that matter

A vendor that advertises 99 % accuracy has told you almost nothing. The number is a word error rate subtracted from 100, and a word error rate means something only when you also know which audio it was measured on, how the text was normalised before the comparison, and who was speaking. Change one of those and the same engine scores 5 % or 17 %. What follows is how the metric works, which errors it hides, and a test you can run in twenty minutes on your own worst recording.

How word error rate is calculated

Take a reference transcript, meaning what was really said, written out by a human. Align the machine output against it word by word and count three kinds of mismatch: substitutions (a word came out as a different word), deletions (a word is missing) and insertions (a word appears that nobody said). Word error rate is their sum divided by the number of words in the reference, written WER = (S + D + I) / N (Wikipedia, Word error rate, accessed September 2026). "99 % accurate" is another way of writing "1 % WER". Two things follow from the arithmetic and neither is obvious from a marketing page. WER can exceed 100 %, because insertions are not capped by the length of the reference. And every error weighs exactly one: losing an "um" costs the same as losing a "not".

That second point is the whole problem. Consider a line from a budget meeting:

Reference: "we are not going to approve the second tranche this quarter" Output: "we are going to approve the second tranche this quarter"

One deletion in eleven words: a 9 % WER, which reads like a good result, and a sentence that now says the opposite of what was said in the room.

Three things a percentage needs before it counts as a claim

A named dataset

Picovoice publishes the clearest illustration of this, about their own engine: they could legitimately advertise a WER of 5.39 %, while the same engine "could be 9 %, 12 % or 17 % based on the dataset" (Picovoice, Things to know about word error rate, accessed September 2026). Nothing was tuned between those figures. Only the audio changed.

Google's speech-to-text documentation tells customers to measure for themselves instead: gather "a representative sample of audio files", keep it random and "as close to the target environment as possible", and record between 30 minutes and 3 hours of it (Google Cloud, Speech-to-Text accuracy, accessed September 2026). The largest vendor in the category is telling you to ignore published benchmarks, including its own.

A normalisation scheme

Before any counting, both transcripts are normalised: punctuation stripped, everything lower-cased, and a script deciding which spellings count as the same word. AssemblyAI notes that standard normalisers handle "don't" and "do not" but not "healthcare" and "health care", so a model with better formatting can score worse (AssemblyAI, Word error rate is broken, accessed September 2026). Numbers and currency amounts are the same trap: "eighteen", "18" and "18.00" are one word or two depending on whose script ran. A meaningful part of any published score is decided before a single word is recognised.

A speaker mix

Speechmatics notes that a clean test set barely exists, because "it is very difficult to get a test set that has no mistakes from human transcribers", and shows a transcript scoring 125 % WER that reads almost correctly next to one at 20 % WER with a meaning-altering error (Speechmatics, The problem with word error rate, accessed September 2026). Apple's research group built an alternative metric for the same reason: in their sample WER averaged 9.2 %, while their human-weighted version, which counts only errors that change meaning, hurt readability or misspell a proper noun, came out at 1.4 % (Apple, Humanizing word error rate, accessed September 2026).

Questions to put to any vendor number:

A vendor who cannot answer four of those five is selling decoration.

The errors that cost you, and the ones that do not

Weighted by damage rather than by count, the list is short.

• Numbers. The most reported failure in practice. From a discussion of AI notetakers in legal work: "the person said 8 to 10, LLM transcribed as 18 ... But the AI isn't smart enough, and then 18 goes into the record" (Hacker News, 11 May 2026). Both readings are grammatical, so nothing downstream flags it.
• Names and domain vocabulary. A surname the model has never seen becomes the nearest common word, and so do drug names, case names, part numbers and codenames. AssemblyAI's example is "lisinopril" arriving as "listening a pill", which WER counts the same as "gonna" becoming "going to".
• Negations. "Not", "never", "no longer", and contracted forms inside fast speech. One deleted word, an inverted meaning.
• Overlapping speech. Two people at once is the hardest input any engine gets. Words are dropped and attributed to the wrong speaker at the same time.
• Long silences and non-speech audio. Its own failure mode, and a stranger one.

When the transcript contains a sentence nobody said

Sometimes the output is not a misheard word but a fluent, plausible sentence nobody spoke, usually in a stretch where nobody was talking, or over music, coughing or a long pause.

The mechanism is a property of how large speech models are built, not a defect in one product. They are trained on very large collections of audio paired with text, and in that training data the text is not always a faithful transcript of the audio. The model learns that some stretches of non-speech audio are followed by text, and at inference it produces text for them. Researchers have reproduced this deliberately by feeding non-speech sounds into a widely used open model, and documented recurring patterns in the output (Investigation of Whisper ASR hallucinations induced by non-speech audio, arXiv 2501.11378, January 2025).

The measured scale is small and unevenly distributed. In an analysis of 13,140 audio samples, roughly 1 % of transcriptions contained a hallucination, and close to 40 % of those were actively harmful: invented references to violence, false names and health details, or fabricated authoritative-sounding statements. They were much more common for speakers with longer non-vocal stretches, so people who pause, hesitate or have a speech impairment get the worst of it (Montreal AI Ethics Institute, Careless Whisper, accessed September 2026).

One per cent sounds harmless until you notice that a fabricated sentence is invisible to every check that reads only the text. It is grammatical, it fits the topic, and it carries no low-confidence marker. The only reliable detector is the audio.

Run the test yourself, in about twenty minutes

This will not produce a publishable benchmark, for which Google's 30 minutes to 3 hours is the right sample. It will produce a decision you can defend.

Keep both columns, because they will disagree, and the disagreement is the finding:

Engine B loses on the metric and wins the purchase, which is common enough to be why this article exists.

If other people are on the recording, tell them you are transcribing and check what your local law and the meeting's terms require. In Germany, section 201 of the Criminal Code protects the spoken word, and consent from everyone present is the safe path. A test recording is still a recording.

Kalima's free plan covers this test without a credit card: two hours of transcription a week, 45-minute sessions, uploads up to 50 MB and 30 minutes per file (pricing).

Catching the errors in a transcript you already have

Whatever the number said, errors remain, so the question is how fast you find them.

Treat the audio as the record, not the text. In Kalima's Studio, click any word in the transcript and playback jumps to the moment it was spoken, with the current word highlighted as the audio runs. Checking a suspicious figure is one click rather than a hunt through a waveform, and playback slows to 0.25x where it matters. See playback and karaoke.

Read the passages where nobody changes speaker. A long block attributed to one speaker with no interjections is where an invented sentence hides, as is any stretch around a break in the conversation.

Check the faint words. Kalima renders lower-confidence words fainter, a map of where to look first. It is no hallucination detector, since a fabricated sentence can arrive at full confidence, but it finds misheard names quickly.

Two settings reduce the error count before you record anything.

Session context and expected-language hints. A short description of the meeting plus the names, acronyms and domain terms you expect tilts recognition towards them. The description field holds up to 2,000 characters, and a context template can be reused or set as a project default. Language hints guide rather than lock: listing English and Spanish does not shut out a German sentence. See improve accuracy with context.

Realtime or Post-Process+. Kalima records in two modes for the same transcription time. Realtime shows text while people speak, which you need for live captions, live translation or viewers following along. Post-Process+ records first and delivers the transcript a few minutes after you stop, with the highest accuracy and the best speaker separation, because the engine sees the whole recording instead of a stream. Note the limit: a running Post-Process+ take can switch to Realtime once, and a Realtime take cannot be reprocessed into Post-Process+ afterwards. See recording modes.

Be clear about what can be fixed afterwards today. You can rename speakers so real names carry into every export, and run Check voices, which re-listens on your own device (a 38 MB voice model downloads once and stays there, no audio leaves the machine, about 40 seconds per hour) and lets you reassign passages the speaker separation got wrong. Word-level correction of the text, which Kalima calls AutoCorrect+, is still in development and marked as coming on the pricing page. Until it ships, the fix for a recurring misheard term is session context before the next recording.

Why Kalima publishes no accuracy percentage

We do not put a number on it, for every reason above. Any figure we published would be measured on audio we chose, normalised by a script we wrote, on a speaker mix we picked. It would be true, and useless to you, because your audio is a four-person call with someone on a laptop mic in a room with a fan.

What we can do is make our transcripts checkable: audio kept alongside the text, every word clickable back to its moment, confidence visible, speaker assignment reviewable on your own device, and a free tier large enough to run the test above against us and against anyone else. Our comparison pages are written on features and limits rather than on a benchmark, for the same reason.

Frequently asked questions

Is 95 % accuracy good for transcription?

It is unanswerable as stated, which is the point. 95 % means a 5 % word error rate on some dataset, under some normalisation scheme, with some set of speakers, none of them disclosed. On clean studio audio, 5 % is mediocre. On a four-person call with crosstalk and accents it may be very good.

Why is my meeting transcript wrong when the vendor claims 99 %?

Almost certainly because your audio is nothing like their test set. Distance from the microphone, room reverb, a Bluetooth headset, two people talking at once and specialist vocabulary each cost more accuracy than any difference between engines. Fixing the input helps more than switching vendors.

Can an AI transcript invent words that were never spoken?

Yes, and it is documented. Large speech models can output a fluent, plausible sentence for a stretch of audio containing no speech, most often over silence, music or noise. One analysis of 13,140 samples found it in roughly 1 % of transcriptions, more often for speakers who pause a lot. It is a property of this class of model, not of one product, and the way to catch it is to listen.

How do I verify a meeting transcript without listening to the whole thing?

Check what carries cost rather than reading linearly: every number, every proper name, every negation in a sentence that commits someone to something, and any long block with no speaker change. If you can click a word to jump to that moment in the audio, each check takes seconds rather than minutes.

Related

• Tips for accurate transcription
• Improve accuracy with context
• Recording modes: Realtime and Post-Process+
• Playback and karaoke: click a word to hear it
• How to improve transcription accuracy
• Compare Kalima with other transcription tools