Reliability-Conditioned Heterogeneous Foundation-Model Residual Fusion for Multimodal Disaster Classification

During sudden-onset disasters, social media text and images provide time-critical evidence for emergency awareness, damage assessment, and humanitarian response, but their cues are often incomplete, noisy, or conflicting. Direct equal-status fusion of heterogeneous pretrained representations can introduce semantic misalignment and allow unreliable evidence to influence the classifier. This paper proposes foundation-augmented reliability-conditioned dynamic adaptive fusion (FA–RC–DAF) for multimodal disaster classification. The model first preserves CLIP as a unit-weight text–image alignment base, maintaining a stable cross-modal decision space. It then projects BERTweet and SigLIP features into the CLIP-aligned space as bounded residual corrections, enabling domain-specific linguistic and complementary vision–language cues to refine the base without overwriting it. A confidence–agreement router estimates sample-level residual reliability from normalized-entropy predictive concentration and cross-encoder agreement, selectively admitting each residual before fusion. Explicit cross-modal interaction is performed only after the two streams have been reliability-refined. On CrisisMMD, FA–RC–DAF achieves 92.26% accuracy, 92.25% weighted F1, and 91.26% macro F1 under the retained five-class protocol. The protocol-aware published comparison is reported separately from the five-seed internally matched architectural comparison, which provides the primary evidence for method-level claims. Additional evaluations show differentiated behavior under event- and disaster-type shifts, stronger difficulty under forward temporal drift, and condition-dependent sensitivity to corrupted or unavailable inputs, providing a more fully characterized basis for multimodal disaster decision support.