Cross-modal Sentiment Analysis of Text Image Fusion Based on Hybrid Fusion Strategy

Due to the different modalities of data such as images and text, the difficulty of sentiment analysis increases. Therefore, a cross-modal sentiment analysis method based on text image fusion is established in this study. In the experiment, a bidirectional network is used to extract text and image features. On this basis, a cross-modal sentiment analysis method based on image text fusion is established. At the same time, concepts such as image attributes are introduced in the experiment to detect irony in graphic and textual data. Finally, a hybrid strategy cross-modal sentiment analysis method is established in the experiment. After comparison, the proposed research method has the highest subject working characteristic curve and PR, which are 5% and 3% higher than the comparative methods, respectively. When the validation set size is 400, the recognition time of the proposed research method is 2.1 seconds. When iterating 50, the recognition time of this method is 0.9 seconds. In practical applications, the proposed research method has accurately analyzed six types of graphic and textual content with different emotional tendencies. This method has the best detection results for both single graphic and cross-modal modes.

Authors

Institutions

Publication Details

Journal
Informatica
Published
2024-11-27
DOI
https://doi.org/10.31449/inf.v48i21.6767
Primary Topic
Simulation and Modeling Applications
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Cross-modal Sentiment Analysis of Text Image Fusion Based on Hybrid Fusion Strategy

Yuan Fang, Yi Wang
Informatica
Simulation and Modeling Applications
article

Cross-modal Sentiment Analysis of Text Image Fusion Based on Hybrid Fusion Strategy

Yuan Fang, Yi Wang
article en

Abstract

Due to the different modalities of data such as images and text, the difficulty of sentiment analysis increases. Therefore, a cross-modal sentiment analysis method based on text image fusion is established in this study. In the experiment, a bidirectional network is used to extract text and image features. On this basis, a cross-modal sentiment analysis method based on image text fusion is established. At the same time, concepts such as image attributes are introduced in the experiment to detect irony in graphic and textual data. Finally, a hybrid strategy cross-modal sentiment analysis method is established in the experiment. After comparison, the proposed research method has the highest subject working characteristic curve and PR, which are 5% and 3% higher than the comparative methods, respectively. When the validation set size is 400, the recognition time of the proposed research method is 2.1 seconds. When iterating 50, the recognition time of this method is 0.9 seconds. In practical applications, the proposed research method has accurately analyzed six types of graphic and textual content with different emotional tendencies. This method has the best detection results for both single graphic and cross-modal modes.

InformaticaVol. 48(21)
Shanghai International Studies University (CN), Shanghai Medical Information Center (CN)
Openalex Percentile: Top 75%
Simulation and Modeling Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Cross-modal Sentiment Analysis of Text Image Fusion Based on Hybrid Fusion Strategy — Yuan Fang, Yi Wang · Informatica (2024) | TGRS Research Map | TGRS