Chinese Lip-Reading Based on Residual Network and Attention Mechanism

To address the common issues in existing lip-reading recognition systems, such as insufficient feature extraction and visual ambiguities, a Chinese sentence-level lip-reading recognition system based on residual networks and attention mechanisms (Spatio-Temporal Cascade Network, STCNet) is proposed. This system can first translate continuous videos containing speakers into pinyin sequences sentence by sentence, and then convert them into continuous Chinese sentences. Firstly, the residual network in the front-end feature extraction module can achieve bidirectional transmission and fusion of multi-level features, enhancing the extraction capability of long-term temporal features. In the back-end pinyin classification module, a Temporal Convolutional Network (TCN) is used to extract long-term dependency information, and an attention mechanism is introduced to weight discriminative features at different time intervals to reduce visual ambiguities

Authors

Publication Details

Journal
International Journal of Innovative Science and Research Technology (IJISRT)
Published
2026-11-19
DOI
https://doi.org/10.38124/ijisrt/26nov599
Primary Topic
Speech and Audio Processing
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Chinese Lip-Reading Based on Residual Network and Attention Mechanism

ZiLong Liu, Chuan Qin
International Journal of Innovative Science and Research Technology (IJISRT)
Speech and Audio Processing
article

Chinese Lip-Reading Based on Residual Network and Attention Mechanism

ZiLong Liu, Chuan Qin
article en

Abstract

To address the common issues in existing lip-reading recognition systems, such as insufficient feature extraction and visual ambiguities, a Chinese sentence-level lip-reading recognition system based on residual networks and attention mechanisms (Spatio-Temporal Cascade Network, STCNet) is proposed. This system can first translate continuous videos containing speakers into pinyin sequences sentence by sentence, and then convert them into continuous Chinese sentences. Firstly, the residual network in the front-end feature extraction module can achieve bidirectional transmission and fusion of multi-level features, enhancing the extraction capability of long-term temporal features. In the back-end pinyin classification module, a Temporal Convolutional Network (TCN) is used to extract long-term dependency information, and an attention mechanism is introduced to weight discriminative features at different time intervals to reduce visual ambiguities

International Journal of Innovative Science and Research Technology (IJISRT)Vol. 11(11)
Speech and Audio Processing
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon