Chinese Lip-Reading Based on Residual Network and Attention Mechanism
To address the common issues in existing lip-reading recognition systems, such as insufficient feature extraction and visual ambiguities, a Chinese sentence-level lip-reading recognition system based on residual networks and attention mechanisms (Spatio-Temporal Cascade Network, STCNet) is proposed. This system can first translate continuous videos containing speakers into pinyin sequences sentence by sentence, and then convert them into continuous Chinese sentences. Firstly, the residual network in the front-end feature extraction module can achieve bidirectional transmission and fusion of multi-level features, enhancing the extraction capability of long-term temporal features. In the back-end pinyin classification module, a Temporal Convolutional Network (TCN) is used to extract long-term dependency information, and an attention mechanism is introduced to weight discriminative features at different time intervals to reduce visual ambiguities
Authors
- ZiLong Liu
- Chuan Qin
Publication Details
- Journal
- International Journal of Innovative Science and Research Technology (IJISRT)
- Published
- 2026-11-19
- DOI
- https://doi.org/10.38124/ijisrt/26nov599
- Primary Topic
- Speech and Audio Processing
- Type
- article
- Field-Weighted Citation Impact
- 0.00