Decoding silent speech: a machine learning perspective on data, methods, and frameworks

Abstract At the nexus of signal processing and machine learning (ML), silent speech recognition (SSR) has evolved as a game-changing technology that allows for communication without audible voice. This study offers a thorough overview of SSR, tracing its evolution from early waveform analysis to the most recent ML methods. We start by examining current SSR techniques using ML and determining the essential conditions for efficient SSR systems. After that, we look at the datasets and data collection techniques currently employed in SSR research, highlighting the difficulties posed by the variety of articulatory movements and the scarcity of data. Examining state-of-the-art SSR frameworks, the paper covers important topics such signal processing, feature extraction, ML techniques for decoding and optimizing and assessing the performance of SSR models. We emphasize how deep learning (DL) and ML models have evolved to increase SSR resilience and accuracy. The field's proposed procedures are examined, with an emphasis on sophisticated feature extraction and classification methods. Modern SSR techniques are compared in terms of performance, highlighting the advantages and disadvantages of different models. There is also discussion of ethical issues, especially those pertaining to privacy and consent. The integration of multimodal information—visual cues, electromyography signals, and neuroimaging data—to improve SSR systems is covered in this work. We investigate the functions of transfer learning and domain adaptation in handling cross-subject variability. Lastly, the study offers suggestions and future prospects for SSR research, providing practitioners, engineers, and academics with a road map. As SSR continues to push the frontiers of human–machine interaction, our study aims to increase our collective understanding of the technological advances and societal effects of SSR in the ML age.