Multilabel Corrosion Morphology Classification Using Deep Learning with Multimodality Fusion

Abstract Accurate characterization of corrosion morphology is critical for understanding degradation mechanisms and supporting predictive maintenance. Corrosion images often exhibit multiple coexisting morphologies, making single-label classification inadequate. This study presents a multilabel classification framework using ResNet-18, Contrastive Language-Image Pretraining (CLIP), and Swin Transformer (Swin-T), with environmental and experimental metadata incorporated through multimodal fusion. A detailed experimental evaluation examines model architecture and training configurations. Comprehensive experiments demonstrate that classweighted loss functions are critical for enabling metadata fusion and improving macroF1 performance for minority labels. Transformerbased and visionlanguage models exhibit strong adaptability to complex corrosion patterns. Ablation studies further quantify the effects of input modality, loss weighting, training epochs, and batch size. The results establish a benchmark for multilabel corrosion morphology classification and highlight the importance of lossfunction design in multimodal learning.