Deep multimodal K-fold Model for emotion and sentiment analysis in figurative language

Abstract The “Deep multimodal K-fold model” is an approach to machine learning that uses multiple modalities, such as text and audio, to analyze emotion and sentiment in Figurative Language. This model uses a K-fold cross-validation method to test how well it works and ensure it can be used in other situations. Using the CMU-MOSEI (Carnegie Mellon University Multimodal Opinion Sentiment and Emotion Intensity) database, we explore sentiment and emotion, considering various aspects like ironic, sarcastic, and subjective sentences in this study. We have four models in our deep learning analysis approach (audio emotion, text emotion, text sentiment, and audio sentiment). We suggested a single-tasking multimodal framework outperforming others to benefit from the interdependence of two related activities (sentiment and emotion). Our experiment was conducted using CNN and LSTM. Specific experimental findings show that LSTM performs better than CNN for the two emotion models, whereas CNN is slightly better for the two sentiment models. We achieved 99% accuracy in the k-Fold Deep Learning Text Emotion model and 88% in the k-Fold Deep Learning Audio Emotion model with the LSTM technique (85% and 85% with CNN). In text and audio sentiment analysis, we got 94% and 79% with CNN (93% and 78% with LSTM), respectively. All reported figures are accuracies on a single held-out test split rather than averages over the K folds. In conclusion, the Deep multimodal K-fold model is a promising way to analyze emotions and feelings in Figurative Language. Within the limits of the single-split evaluation on a 1,354-utterance subset reported here, it offers a reliable baseline for this task because it combines multiple modalities with K-fold cross-validation.

Authors

Institutions

Publication Details

Journal
International Journal of Speech Technology
Published
2026-11-14
DOI
https://doi.org/10.1007/s10772-026-10308-2
Primary Topic
Sentiment Analysis and Opinion Mining
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Deep multimodal K-fold Model for emotion and sentiment analysis in figurative language

Brojo Kishore Mishra, Sudipta Bhattacharya, Samarjeet Borah
1 links
International Journal of Speech Technology
Sentiment Analysis and Opinion Mining
article

Deep multimodal K-fold Model for emotion and sentiment analysis in figurative language

Brojo Kishore Mishra, Sudipta Bhattacharya, Samarjeet Borah
article en

Abstract

Abstract The “Deep multimodal K-fold model” is an approach to machine learning that uses multiple modalities, such as text and audio, to analyze emotion and sentiment in Figurative Language. This model uses a K-fold cross-validation method to test how well it works and ensure it can be used in other situations. Using the CMU-MOSEI (Carnegie Mellon University Multimodal Opinion Sentiment and Emotion Intensity) database, we explore sentiment and emotion, considering various aspects like ironic, sarcastic, and subjective sentences in this study. We have four models in our deep learning analysis approach (audio emotion, text emotion, text sentiment, and audio sentiment). We suggested a single-tasking multimodal framework outperforming others to benefit from the interdependence of two related activities (sentiment and emotion). Our experiment was conducted using CNN and LSTM. Specific experimental findings show that LSTM performs better than CNN for the two emotion models, whereas CNN is slightly better for the two sentiment models. We achieved 99% accuracy in the k-Fold Deep Learning Text Emotion model and 88% in the k-Fold Deep Learning Audio Emotion model with the LSTM technique (85% and 85% with CNN). In text and audio sentiment analysis, we got 94% and 79% with CNN (93% and 78% with LSTM), respectively. All reported figures are accuracies on a single held-out test split rather than averages over the K folds. In conclusion, the Deep multimodal K-fold model is a promising way to analyze emotions and feelings in Figurative Language. Within the limits of the single-split evaluation on a 1,354-utterance subset reported here, it offers a reliable baseline for this task because it combines multiple modalities with K-fold cross-validation.

International Journal of Speech TechnologyVol. 29(4)
University of Engineering & Management (IN), Sikkim Manipal University (IN), Berhampur University (IN)
Sentiment Analysis and Opinion Mining
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon