Why speech recognition is harder in live selling than in meetings

Speech recognition is the one stage of the pipeline where an error can never be recovered. A translation mistake can be patched by a human in the comments; a recognition mistake means everything downstream faithfully translates the wrong sentence.

Recognition in meetings is a solved-enough problem, but that experience transfers badly to live selling, for four specific reasons.

Difference one: background music

Meeting rooms are quiet. Live selling almost always has music playing.

Recognition models are trained predominantly on relatively clean speech, and music is an unusually difficult interferer: it has stable rhythm and pitch, and models readily mistake instrumental content for speech. Microsoft’s evaluation guidance gives you a way to confirm this on your own audio: a preponderance of insertion errors points at a noisy recording environment with possible crosstalk. If your error list is full of words nobody said, investigate the music before you investigate the model.

How far down “well under” is does not have to be a guess. The OBS audio mixer guide already defines the zones, and the technical details page puts numbers on them: the Alignment Level sits at -20 dBFS — chosen, in OBS’s own words, to be near the average audio level for speech — and the Permitted Maximum Level at -9 dBFS. Translated into practice:

• Voice peaks belong between roughly -20 and -9 dBFS, and background music and gift effects belong in the green zone, below -20 dBFS. No extra tooling required; the OBS mixer tells you directly;
• Or alternate explicit music segments with speech segments and mute during speech;
• If music must be continuous, use instrumental tracks. Music with vocals is the worst case.

Difference two: sales pacing

Live selling runs faster than conversational speech, and the pacing is uneven — prices come out in a burst, benefit statements get drawn out. How much faster is not something anyone has published, but the consequence is documented: Microsoft lists “the speaker talks significantly faster or slower than usual” as the standard reason to retune the segmentation silence timeout, because fast delivery chains several sentences into one recognition result.

Speed alone is not fatal. The damage comes from elision at speed. Numbers are the usual victim: rapid delivery blurs the difference between “thirty-nine” and “thirty-nine ninety-nine”, and recognition has very little to go on.

This is hard to fix technically, so concede it in the script: deliberately slow down for numbers. Prices, sizes, discount percentages and stock counts each deserve half a beat more.

Difference three: brand names and model numbers

The most common problem, and the most tractable. In a phrase like “the second-generation 30 ml serum”, the brand name is typically a coined or foreign word the model has never seen.

The fix is a two-layer glossary:

• Recognition layer — map the brand name and its common misrecognitions so the recogniser emits the correct spelling;
• Translation layer — mark the brand as do-not-translate, or pin it to one approved rendering.

Doing only the translation layer is the usual mistake. If recognition already misheard the brand name, the translation glossary has nothing to match against.

An hour spent assembling this before going live is the highest-return preparation available.

Difference four: no second take

In a meeting you can say “let me repeat that”. Live pacing does not tolerate frequent self-correction, and viewers are watching a continuously updating caption stream.

That means the system has to tolerate self-repair. When a host says “this one is black — sorry, dark grey”, the desirable output is “dark grey”, not both colours. Most current systems cannot do this, so in practice hosts have to minimise verbal corrections.

A counterintuitive recommendation

The instinct is to reach for a stronger recognition model. But two vendors’ documentation agrees on capture, and both put it ahead of the model:

• Google’s speech recognition best practices call for placing the microphone as close to the speaker as possible, and in the same list require disabling noise reduction and automatic gain control — on the grounds that the service is designed for noisy audio and pre-processing typically reduces accuracy. The “ambient noise suppression” many people leave enabled in their sound card utility is pointing the wrong way.
• Microsoft’s recognition quality guidance attributes deletion errors to weak audio signal strength, and prescribes collecting audio closer to the source.

The order is therefore not a matter of taste. A directional microphone, correct gain, a consistent mouth-to-mic distance and no pre-processing cost almost nothing — and they are a precondition, because while capture is bad the differences between recognition engines are not measurable at all.

How to measure your own accuracy

Do not go by feel:

1. Record 10 minutes of genuine broadcast audio, not a read-aloud script;
2. Run it through offline recognition and export the transcript;
3. Listen through once, mark every error, and count insertions, deletions and substitutions separately.

Microsoft’s general rubric: a word error rate of 5% to 10% is good quality and ready to use, 20% is acceptable but worth further work, and 30% or above signals poor quality. Live selling is unusually unforgiving about brand names and numbers, so a practical acceptance bar normally sits tighter than the general rubric — how much tighter depends on your category.

The split matters more than the total, and Microsoft maps it for you: deletions mean weak signal, insertions mean noise and crosstalk, substitutions mean missing domain terminology. Two recordings can both score 12% while only one of them has anything to do with the recognition engine. The full acceptance protocol is in the tool evaluation article.