A Hybrid AI-Powered System for Real-Time Speaker Diarization and Meeting Summarization

Real-time meeting transcription and summarization are pivotal for enhancing productivity in modern collaborative environments. This paper presents a robust, real-time speaker diarization and meeting summarization system. The system comprises a client-side app and a cloud-based virtual machine pipeline, leveraging advanced technologies such as Whisper for speech-to-text, WhisperNER for entity-enriched transcription, and a fine-tuned T5 model for summarization. A virtual audio driver captures system-wide audio inputs, ensuring seamless data acquisition. This hybrid approach achieves accurate speaker attribution, enriched transcription, and contextualized summaries while maintaining low latency. Experimental results demonstrate high accuracy in diarization (DER), transcription (WER), and summarization quality (ROUGE).