Multiscale dilated convolution TCN lip reading recognition model research

Aiming at the problems of insufficient feature extraction in lip-reading recognition, which affect the recognition accuracy and high model complexity, a multi-scale expansive convolutional time domain network based on time domain convolutional network is proposed. The proposed network expands the receptive field of feature extraction through the multi-scale expansion convolutional part, and the network captures features of different scales at the same time, improving the ability of the network to capture features when facing data sets with complex features. The channel and spatial attention mechanism are added after the multi-scale expansion convolution features are merged, and the importance of each channel is evaluated to adjust accordingly, and the importance of specific regions in the image is paid further attention to the feature information. Finally, the output feature maps of channel attention and spatial attention mechanism are fused by element addition operation, and finally the enhanced feature maps are obtained. The model is tested on LRW data set to verify the feasibility of the proposed model, and the experimental results show that the accuracy of the model reaches 88.1%.