Deep multimodal K-fold Model for emotion and sentiment analysis in figurative language
Abstract The “Deep multimodal K-fold model” is an approach to machine learning that uses multiple modalities, such as text and audio, to analyze emotion and sentiment in Figurative Language. This model uses a K-fold cross-validation method to test how well it works and ensure it can be used in other situations. Using the CMU-MOSEI (Carnegie Mellon University Multimodal Opinion Sentiment and Emotion Intensity) database, we explore sentiment and emotion, considering various aspects like ironic, sarcastic, and subjective sentences in this study. We have four models in our deep learning analysis approach (audio emotion, text emotion, text sentiment, and audio sentiment). We suggested a single-tasking multimodal framework outperforming others to benefit from the interdependence of two related activities (sentiment and emotion). Our experiment was conducted using CNN and LSTM. Specific experimental findings show that LSTM performs better than CNN for the two emotion models, whereas CNN is slightly better for the two sentiment models. We achieved 99% accuracy in the k-Fold Deep Learning Text Emotion model and 88% in the k-Fold Deep Learning Audio Emotion model with the LSTM technique (85% and 85% with CNN). In text and audio sentiment analysis, we got 94% and 79% with CNN (93% and 78% with LSTM), respectively. All reported figures are accuracies on a single held-out test split rather than averages over the K folds. In conclusion, the Deep multimodal K-fold model is a promising way to analyze emotions and feelings in Figurative Language. Within the limits of the single-split evaluation on a 1,354-utterance subset reported here, it offers a reliable baseline for this task because it combines multiple modalities with K-fold cross-validation.
Authors
- Brojo Kishore Mishra (ORCID: https://orcid.org/0000-0002-7836-052X)
- Sudipta Bhattacharya (ORCID: https://orcid.org/0000-0001-9778-0506)
- Samarjeet Borah (ORCID: https://orcid.org/0000-0001-9304-3525)
Institutions
- University of Engineering & Management (IN)
- Sikkim Manipal University (IN)
- Berhampur University (IN)
Publication Details
- Journal
- International Journal of Speech Technology
- Published
- 2026-11-14
- DOI
- https://doi.org/10.1007/s10772-026-10308-2
- Primary Topic
- Sentiment Analysis and Opinion Mining
- Type
- article
- Field-Weighted Citation Impact
- 0.00