K-means to Spectral Clustering: Improving Diarization Accuracy with ECAPA-TDNN Embeddings

This paper presents a high-accuracy speaker diarization system that determines speakers and their corresponding time segments within multi-speaker audio. The proposed system employs ECAPA-TDNN-based speaker embeddings combined with spectral clustering to segment and group speech based on speaker identities. Evaluation was conducted using TOEFL English practice audio and VoxCeleb conversational data, with combined audio files simulating real-world multi-speaker scenarios. Spectral clustering was chosen over K-means for its ability to capture similarity relationships between embeddings, improving robustness against overlapping speech and high speaker similarity. Experimental results demonstrate that the approach accurately identifies and separates multiple speakers in diverse and noisy environments. This work highlights the effectiveness of combining ECAPA-TDNN embeddings with spectral clustering to achieve scalable and reliable diarization performance, even under challenging acoustic conditions, contributing to the advancement of speaker diarization techniques for applications such as automated transcription, smart assistants, and audio analytics.