Masked Attention-Driven Unified Multitask Representation Learning for Affective Computing

Identifying emotions and sentiments in artificial intelligence remains a difficult task due to the nuanced nature of textual expressions and the numerous relationships between audio-visual signals. Conventional single-modality methods frequently fail to adequately capture the intricate link between spoken language and nonverbal communication, thereby diminishing their efficacy in real-world applications. This study presents a novel fusion model that combines textual, auditory, and visual input through a Bidirectional Gated Recurrent Unit (Bi-GRU) in conjunction with a masked attention mechanism. By selectively emphasizing essential features and disregarding less pertinent information, the model improves both interpretability and classification precision. This study employs a unified learning paradigm through multi-tasking network, contrasting with traditional methods that see sentiment and emotion analysis as separate tasks, to attain a more thorough comprehension of emotional states. Experiments on the MELD dataset reveal that the proposed strategy exceeds unimodal baselines and conventional fusion strategies, highlighting the importance of masked attention in enhancing multimodal integration. The results advance affective computing and have practical ramifications for human-computer interaction, AI-based emotion recognition, and real-time sentiment analysis in applications including virtual assistants, customer engagement, and psychological evaluation.