Attention-Driven ResNet-101 with Bonobo Optimized DNN for Cross-Modal Event Detection

Multimodal data in the form of text, video, audio, and image has become increasingly available in social networks due to the advancement of technology. Event extraction has a serious trigger word ambiguity problem, making event prediction a crucial and challenging sub-task. Moreover, recently, the text-image fusion has made it difficult to adapt the model to higher-resolution image synthesis. Therefore, this research focuses on event detection by integrating both visual and textual information to improve the performance of the mining process. This multimodal integration approach includes four stages: preprocessing, multimodal feature extraction, dual multimodal integration, and event classification. The first stage comprises the preprocessing of both image and text. Then, the multimodal information is extracted using term frequency-inverse document frequency and ResNet 101 in the second stage. Follow that, a dual attention mechanism is applied to extract the text context that is more relevant to the image features. After the multimodal features are fused in the integration phase. Ultimately, the detection process is carried out using a DNN classifier, with Bonobo Optimization applied to select the dropout and momentum parameters. This approach is used to enhance the accuracy values for event prediction. This multimodal integration is tested on Python software to evaluate its performance. The proposed approach is compared with several approaches, such as Artificial Neural Network (ANN), XGBoost (XGB), Support Vector Machine (SVM), and Naive Bayes (NB). The DNN-BO technique achieved 99% accuracy, 98% precision, 99% specificity, and 1% error value, while compared to the prior approaches, the ResNet-DNN attained better values. Furthermore, the present multimodal integration-based event detection approach predicts events effectively on social media.