Localize, Filter, Segment: Towards Multi-Microphone Speaker Segmentation

Speaker Diarization addresses the question Who spoke and when?-a crucial task in conversational AI.We propose Localize, Filter, Segment (LoFi-Seg), a novel framework for speaker segmentation in multi-microphone setups.LoFi-Seg consists of three modules: a direction-of-arrival (DOA) estimator, a spatial filter bank (beamforming), and a Voice Activity Detection (VAD) model.The framework processes multichannel audio by steering beamformers-one per speaker-toward predicted directions from the DOA module.The output of each beamformer is then passed through the VAD model to determine speaker activity.Combining explicit DoA estimation and VAD improves the model transparency, thus preserving physical interpretability during multichannel filtering.We validate LoFi-Seg through experiments on simulated multi-speaker, multi-microphone conversations, where speaker positions and acoustic conditions are controlled.The system is evaluated on speaker segmentation performance, with additional assessments of speaker localization performance drift across setups.This approach demonstrates how LoFi-Seg combines robust speaker segmentation with interpretable processing, making it a valuable tool for advancing speaker diarization in complex audio environments.