An innovative multi-head attention mechanism-driven recurrent neural network model with feature representation fusion for enhanced image captioning to assist individuals with visual impairments

Developments in image captioning technologies played a crucial role in improving the quality of life for individuals with visual impairments, advancing better social inclusivity. Image captioning is the task of representing the visual content of the images in natural language, applying a language method and a visual understanding system able to generate significant and syntactically correct sentences. Image captioning is a field of research of vast significance, targeting the creation of natural language representations for visual content in static images. Automatically representing the image content is a significant challenge in artificial intelligence (AI). Therefore, the emergence of deep learning (DL) and the most recent vision-language pre-training methods have significantly advanced the domain, resulting in more advanced techniques and enhanced performance. DL-based methods can process the difficulties and nuances of image captioning. This paper proposes an Innovative Multi-Head Attention Mechanism-Driven Recurrent Neural Network with Feature Representation Fusion for Image Captioning Performance (MARNN-FRFICP) approach to assist individuals with visual impairments. The MARNN-FRFICP approach aims to enhance image captioning by employing an effective method focused on improving accessibility for individuals with visual impairments. Initially, the Gaussian filtering (GF) technique is utilized in the image pre-processing stage to enhance image quality by removing the noise. In addition, the fusion of advanced DL models, namely InceptionResNetV2, convolutional vision transformer (CvT), and DenseNetl69, is employed to enhance the effectiveness of the feature extraction process. Moreover, the hybrid of multi-head attention mechanism-based bi-directional long short-term memory and gated recurrent unit (MH-BLG) technique is used for classification. Finally, the Lyrebird optimization algorithm (LOA) technique is employed for tuning. The efficiency of the MARNN-FRFICP methodology is examined under the Flickr8k, Flickr30k, and MSCOCO datasets. The experimental analysis demonstrates that the MARNN-FRFICP methodology has improved scalability and performance compared to recent techniques in various measures.