Robust Facial Expression and Emotion Classification: A Comparative Analysis of Image and Text-Based Sentiment

Facial expression recognition (FER) and text-based emotion identification are critical for understanding human behavior, as they play an important part in social interaction and communication. This research investigates using deep learning and machine learning algorithms to categorize emotions from image and text datasets. Using advances in machine learning, image processing, and affordable computational power, we evaluate five image datasets (Fer2013, CKPLUS, CK+, RAF, and Fer2013 + CK+) and two text datasets (EDER and ENBT) to solve issues provided by varied poses, lighting, and context. The ResNet50 model regularly outperformed other models for image-based emotion recognition, reaching 99% accuracy on CKPLUS and CK+. In contrast, CNN demonstrated the lowest accuracy, although the VIT model produced promising results on selected datasets. RoBERTa surpassed rival models in text-based emotion recognition, scoring 94% on the EDER dataset and 92.5% on the ENBT. These results emphasize ResNet50 and RoBERTa's superior performance, demonstrating their potential for real-world applications in behavior analysis, human-computer interaction, and emotion-driven systems.