Deploying Multilingual ASR in Digital Twin Systems: A Performance and Efficiency Analysis of Whisper and SeamlessM4T

This study presents a systematic comparison of two leading automatic speech recognition (ASR) model families—OpenAI's Whisper and Meta's SeamlessM4T—across three typologically diverse languages: English (Germanic), Russian (Slavic), and Kazakh (Turkic). Motivated by the need to identify optimal ASR solutions for multilingual collaborative environments in digital twin development, we evaluate these models using standardized benchmarks including LibriSpeech and Common Voice datasets. Our analysis employs Word Error Rate (WER), Character Error Rate (CER), and GPU memory utilization as primary metrics. Results demonstrate that Whisper large-v3 achieves the best performance for English and Russian with WERs of 3.29% and 9.83% respectively, while SeamlessM4T-v2-large offers superior performance for Kazakh with a WER of 13.7%. GPU memory requirements range from 144.4MB to 6025.7MB, scaling with model size. These findings provide valuable guidance for selecting appropriate ASR models for multilingual applications in collaborative environments where accurate voice transcription is essential for human-machine interfaces in digital twin systems.