Multilingual Speech Command Recognition for Voice Controlled Robots and Smart Systems

Speech Command Recognition (SCR) has many applications in smart home systems, voice-controlled robots, and voice assistants. The modern SCR systems employ deep learning models trained on speech command datasets. Nowadays, the Google Speech Commands (GSC) dataset serves as a benchmark for training and evaluating SCR models, but it only includes English commands. In this work, we introduce an equivalent of the GSC for the Kazakh, Tatar, and Russian languages. We collected the dataset via a Telegram bot in real-world settings. The dataset includes the same 35 commands as GSC and contains 3,623 utterances for Kazakh, 3,547 for Tatar, and 1,625 for Russian. We trained and evaluated both monolingual and multilingual SCR models. The experimental results show that the multilingual model outperforms the monolingual models with an accuracy of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{9 8. 5 7 \%}, \mathbf{9 9. 3 3 \%}, \mathbf{9 6. 8 6 \%}$</tex>, and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{9 7. 3 3 \%}$</tex> on the test sets of the Kazakh, Tatar, Russian datasets and GSC, respectively. Furthermore, we deployed the model on an NVIDIA Jetson Orin NX 16GB, achieving an average inference time of 0.008 seconds. This performance makes it suitable for real-time applications. We have made the dataset, source code, and pretrained models publicly available<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup><sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>https://github.com/IS2AI/Multilingual-Speech-Command-Recognition to encourage further research in this area.