MM-OpinionNet: a cross-modal deep learning framework for multimodal news mining and intelligent public opinion analysis

Traditional public opinion analysis methods mostly focus on single modal data, and it is difficult to make full use of the correlation and complementarity between multi-modal data. In this paper, a multi-modal news data mining and intelligent public opinion analysis model (MM-OpinionNet) is proposed. By fusing multi-modal data, a more comprehensive and accurate public opinion analysis model is constructed. In the research, text data is preprocessed by word segmentation and stop words removal, image and video data are processed by standardization and object detection, and audio data is processed by speech-to-text and acoustic feature extraction. In the feature extraction stage, BERT embedding and TF-IDF are used for text, ResNet-50 for image, 3D-CNN for video and MFCC and prosodic features for audio. The model realizes feature fusion through cross-modal attention mechanism and graph neural network, and introduces cross-modal comparative learning loss to solve the problem of data heterogeneity. The experiment was based on a synthetic multimodal news dataset containing 100,000 news samples encompassing text, images, videos, and audio. The results demonstrated that MM-OpinionNet outperformed unimodal baseline methods and traditional multimodal approaches in event detection, sentiment analysis, and public opinion prediction. It achieved an accuracy of 89.7%, reduced the Root Mean Square Error (RMSE) for sentiment analysis by 34%, and attained a Mean Absolute Percentage Error (MAPE) of 8.3% and a Mean Squared Error (MSE) of 4.1×10³ for public opinion prediction. Ablation studies further confirmed the importance of the cross-modal attention mechanism and multitask learning in enhancing model performance.