Attention-Residual Multi-Modal Fusion Framework for Crisis Categorization in Social Media Feeds

Crisis categorization in social media feeds perform an important part in modern disaster management and response techniques. With the increasing employ of social platforms as a primary source of information during crises, effective categorization algorithms are essential for quickly and accurately assessing the severity and impact of events. This study introduces the Attention Residual Multi Modal (ARMM) Fusion Framework, which addresses difficulties in MM data processing for damage assessment. For image processing, the system uses Visual Refinement with Feature Forge, which includes Bilateral Filtering for noise reduction and edge preservation, Bicubic Interpolation for upscaling, and Residual Network with Drop Block for detailed and robust image feature extraction. The framework cleans and pre-processes text using an LSTM-Residual with Embedding Network, converting it into compact vector representations, and then uses residual LSTM connections to capture temporal dependencies and maintain feature integrity for robust text feature extraction. Image and text information are then combined and processed using a MM channel attention method, which improves sensitivity to informative features. The proposed method produces outstanding performance metrics, including precision of 98.00%, recall of 94.12%, F1 score of 95.86%, and accuracy of 96.13%. This method efficiently identifies damage severity (severe, medium, or minor) in tweets that include both images and text, leading risk management strategies (rescue, volunteering and contribution) depending on the assessed damage.