Intelligent recognition of visual speech based on custom CNN and Optimized Quaternion Charlier Moments using PSO

This paper presents a novel model termed as Optimized Quaternion Charlier Moments Convolutional Neural Network (QCMs-PSO-CNN) to develop an intelligent method of recognizing the visual speech accurately that is phrase or word that a person spoke in the video through Lip-reading. The proposed method put forward the Optimized QCMs using our enhanced particle swarm optimization (PSO). The suggested PSO is improvised on linear inertia weights and a sine–cosine learning factor, which effectively strengthens the optimization capability. This method suggested a custom-built shallow CNN which incorporates optimized QCM as a filter in the first layer in a novel way for better recognition ability of the method. The study shows that this method is a good solution for reducing the high video image dimension and gaining time for training. The proposed architecture QCMs-PSO-CNN found to classify the digits, letters, or words effectively. Three standard video datasets such as GRID, LRW, and GLips are chosen to evaluate the performance of the proposed method. The experimental analyses show that the proposed method attains excellent recognizing performance results compared to the other recently investigated approaches which made use of complex models and deeper architecture.