A systematic approach to fine-tuning transformers for emotion detection on the empathetic dialogues benchmark

Abstract Detecting emotions in dialogues faces challenges like context dependency and label imbalance. This paper presents a systematic methodology for fine-tuning pre-trained Transformer models on the Empathetic Dialogues benchmark to address these issues. Our methodology investigates and demonstrates significant performance improvements achieved by providing conversational context through utterance grouping, optimizing learning parameters via hyperparameter tuning, and mitigating label bias using class weighting. We perform a comparative study fine-tuning four pre-trained models, RoBERTa, DistilRoBERTa, DialoGPT, and T5, under this systematic framework. Results demonstrate that this combined approach significantly enhances performance. The best-performing configuration achieved 83% accuracy on an unseen test set for the benchmark’s 32 fine-grained emotion labels and 87% accuracy when predicting 6 broader emotion categories. These findings underscore the effectiveness of the proposed systematic approach in enhancing the accuracy and robustness of conversational emotion recognition models.