LLM Selection: Improving ASR Transcript Quality via Zero-Shot Prompting

When transcribing telephony audio, Automatic Speech Recognition (ASR) engines often produce noisy output with high word error rates (WER), which impacts the efficacy of downstream analyses that process this transcribed text. We present two experiments demonstrating how an LLM selection method can be used to improve the quality of telephony-based transcripts. This involves generating transcripts for a dataset from multiple ASR engines and prompting an LLM to select the best ASR transcript from the options. Our results indicate that our proposed technique approaches the optimal performance that would be achieved if every transcript was verified against the corresponding ground truth (the Oracle approach). Overall, we find that this method can make notable WER improvements to ASR transcriptions of telephony audio. Further, maximizing performance gain requires utilizing an appropriate targeted improvement strategy.