Co-ViTDefect: A Bayesian Ensemble of Vision Transformers for Multiclass Concrete Defect Identification with Interpretability

In-service concrete structures often suffer from various types of defects. Automatically identifying multiple defect types in operating concrete structures remains challenging for inspectors and maintenance engineers worldwide. Therefore, efficient tools for evaluating surface defects in concrete structures are essential. In particular, vision transformers (ViTs) are relatively new tools for computer vision tasks and often outperform traditional convolutional neural networks (CNNs). Several attempts have utilized ViTs for crack classification in concrete structures, but they mostly rely on single-model identification and implement binary classification. By leveraging ensemble learning, this work proposes a novel ensemble approach, Co-ViTDefect, for multiclass concrete defect classification. The model integrates multiple ViT learners using Bayesian Dirichlet-based ensemble learning, which probabilistically balances the strengths of individual models and also provides an estimate of the uncertainty of each prediction. The model was evaluated with five-fold cross-validation. Experimental results show that the proposed Co-ViTDefect achieves 88.19% accuracy and 85.1% F1-score, with per-class F1-scores above 90% for cracks, efflorescence, and no-defect categories. It outperformed the best single ViT model (87.48% accuracy, 84.16% macro F1-score) as well as CNN-based and transformer-based models, including Swin-Tiny and MaxViT-Tiny. Without any post hoc calibration, Co-ViTDefect also had a lower calibration error than the uniform averaging ensemble (ECE of 0.214 against 0.285) and the lowest Brier score (0.242). Future work can improve the recognition of scaling and general defects.