Design and implementation of metaheuristic-driven BiLSTM hybrid model for silent communication

• BiLSTM-Based Visual Speech Synthesis with Hybrid Optimization Techniques. • Visual Speech Generation Using Grey Wolf Optimization and Sparrow Search Algorithm. • Speech Signal Generation from Lip Movements. • Mouth Region Detection and Extraction Using Dlib Framework. • Lip Region Feature Extraction for Visual-to-Audio Mapping. • Performance Evaluation Using Metrics: PESQ, STOI, PSNR, and RMSE. In visual computing field, speech synthesis is a demanding technology today. Despite the advancement in visual speech synthesis (VSS), existing strategies focus on complex architectures involving greater training parameters and auxiliary losses, which leads to performance degradation. In the proposed methodology, the lower face encoder is implemented using Resnet-50 to derive vision information from lip images of the GRID and LRS2 datasets. The proposed methodology, named Hybrid Optimized Bidirectional Long Short-Term Memory (BiLSTM-Hybrid), is designed for producing target acoustic signals from visual features. Six models, namely deep convolutional neural network (CNN), LSTM, BiLSTM, BiLSTM-GWO, BiLSTM-SSA, and BiLSTM-Hybrid, are implemented to decode the speech. Our experiments show that the BiLSTM-Hybrid model can synthesize a speech signal with 3.161 “Perceptual Evaluation of Speech Quality (PESQ)” and 0.8012 “Short Time Objective Intelligibility (STOI)”. The “signal-to-noise ratio (SNR)” obtained by comparing the original and predicted speech signals is 26.7546 dB. The results are compared with previous research works and found that the proposed methodology's performance is improved. The STOI measure is improved by a factor of 0.0772 over VGAN, while the PESQ metric is enhanced by a factor of 1.153 over VGAN.