DiarizationLM: Speaker Diarization Post-Processing with Large Language Models

In this paper, we introduce DiarizationLM, a framework to leverage large language models (LLM) to post-process the outputs from a speaker diarization system.In this framework, the outputs of the automatic speech recognition (ASR) and speaker diarization systems are represented as a compact textual format, which is included in the prompt to an optionally finetuned LLM.The outputs of the LLM can be used as the refined diarization results with the desired enhancement.As a post-processing step, this framework can be easily applied to any off-the-shelf ASR and speaker diarization systems without retraining existing components.Our experiments show that a finetuned PaLM 2-S model can reduce the WDER by rel.55.5% on the Fisher telephone conversation dataset, and rel.44.9% on the Callhome English dataset.