Study and Analysis the effectiveness of Whisper segmentation techniques in speech diarization: A review

Abstract -This review paper aims to study and analyzing the efficacy of the Whispersegmentation model and its techniques in identification of Language and the Diarization. Acommon application of whisper segmentation is in the analysis of audio data, especiallywhen combined with Open AI's Whisper model. An automatic speech recognition (ASR)technology called Whisper is used to convert spoken words into text. In this application,segmentation refers to the process of breaking up an audio recording into moremanageable chunks that the ASR system can process one at a time. With an emphasis oncorrectly processing Mandarin speech, the system primarily addresses the difficulties ofspeaker diarization (SD) and automatic speech recognition (ASR) in dynamic, multispeaker contexts. Accents from different Indian speakers and handling frequent speakerchanges. Particularly in SD, conventional speech recognition systems frequently fail in suchintricate multilingual and multi speaker situations. Therefore, this work combines speakerdiarization algorithms that are designed for real-time applications with advanced speechrecognition. Speaker embedding technology and effective model output management aretwo examples of these optimizations. Data from 46 different Indian speakers on debate inIndia were used to assess the system. With an overall word-diarization degree of 5.967%,the results demonstrated a promising word-diarization degree of 2.688% in two-speechesconsequences and 10.955% in three-speeches consequences.