Lip Movement Detection in Konkani Using Deep Neural Networks

Lip reading, which uses visual lip movements to identify spoken words, is essential for improving communication. To precisely detect and analyze lip movements, this presents a hybrid deep learning model in this study that combines Convolutional Neural Networks (CNN) with Bidirectional Gated Recurrent Units (Bi-GRU). This method is based on real-time video input, wherein a Bi- GRU model is used to capture temporal dependencies after the mouth region has been identified and processed through a CNN to extract spatial characteristics. Using Konkani language datasets for training and evaluation, the model achieves strong word recognition accuracy. The system shows a significant improvement in lip-reading accuracy by incorporating both the spatial and temporal components of lip motions, which makes it suitable for assistive communication technologies and automatic speech recognition.