An intelligent context-aware facial emotion recognition system for hearing-impaired individuals using large language models and neural network-based feature fusion

Facial emotion recognition (FER) plays a vital role in understanding human behavior and communications, with applications in human-computer interaction, surveillance, healthcare, and multimedia content analysis. Emotion recognition is challenging as it necessitates capturing and analysing subtle variations in facial expressions together with dynamic speech variations. Hearing impairment in youngsters and children has significant effects on social, emotional, and behavioural growth. Communication problems could affect the social and emotional improvement of a hearing-impaired individual. Emotion is a very regular incident of all human beings, whether impaired or normal. Usually, emotions are identified by facial expressions. FER is the most influential, nonverbal, and natural means for individuals to convey their strength and emotions. Automatic FER has received great attention currently. FER is a significant area in the domains of artificial intelligence (AI) and computer vision (CV) because of its important commercial and academic potential. The facial expression detection is challenging for machine learning (ML) methods; meanwhile, people can differ considerably in the manner in which they show their expression. This article develops a Context-Aware Interaction with Fusion Feature Models for Enhanced Facial Emotion Recognition (CAIFFM-EFER) approach in disabled hearing individuals. The main purpose of this article is to develop and calculate a model for accurate and real-time FER by applying advanced techniques. At first, the image processing stage employs the Wiener filter (WF) to enhance image quality by eliminating the noise. For the feature representation process, the CAIFFM-EFER method employs a fusion of VGG-19, MobileNetV1, and InceptionNetV3 models. Followed by, the stacked sparse autoencoder (SSAE) approach is used for facial emotion classification. Finally, context-aware interactions driven by large language models (LLMs) facilitate more refined reasoning and generate adaptive, context-sensitive responses. The experimentation evaluation of the CAIFFM-EFER model portrayed a superior accuracy value of 99.27% over existing methods under the Emotion detection dataset.