Research on Tibetan Lip-Reading Recognition Methods Based on Attention Mechanisms

In response to the scarcity of Tibetan lip-reading data resources, which has led to limited research in this area, and the presence of a large number of homophones with different written forms in Tibetan, this paper constructs a word-level Tibetan lip-reading dataset, MTLRW-50, in complex scenarios. Additionally, it proposes a ResNet-Attention-BiLSTM-based approach for Tibetan lip-reading recognition. First, leveraging a transfer learning strategy, the pre-trained ResNet-50 residual network is employed to extract deep features from video frames. Second, an attention mechanism is introduced to capture contextual information and reduce visual ambiguities. Finally, a bidirectional LSTM (BiLSTM) network is utilized to fully capture temporal dependencies and perform the final classification. Experimental results demonstrate that the proposed model achieves a TOP-1 classification accuracy of 95.82% on the MTLRW-50 dataset, providing an effective solution to the Tibetan lip-reading recognition task.