Crack SAM: enhancing crack detection utilizing foundation models and Detectron2 architecture

Abstract Accurate crack detection is crucial for maintaining pavement integrity, yet manual inspections remain labor-intensive and prone to errors, underscoring the need for automated solutions. This study proposes a novel crack segmentation approach utilizing advanced visual models, specifically Detectron2 and the Segment Anything Model (SAM), applied to the CFD and Crack500 datasets, which exhibit intricate and diverse crack patterns. Detectron2 was tested with four configurations—mask_rcnn_R_50_FPN_3x, mask_rcnn_R_101_FPN_3x, faster_rcnn_R_50_FPN_3x, and faster_rcnn_R_101_FPN_3x—while SAM was compared using Focal Loss, DiceCELoss, and DiceFocalLoss. SAM with DiceFocalLoss outperformed Detectron2, achieving mean IoU scores of 0.69 and 0.59 on the CFD and Crack500 datasets, respectively. The integration of Detectron2 with faster_rcnn_R_101_FPN_3x and SAM using DiceFocalLoss involves generating bounding boxes with Detectron2, which serve as prompts for SAM to produce segmentation masks. This approach achieves mIoU scores of 0.83 for CFD dataset and 0.75 for Crack500 dataset. These results highlight the potential of combining foundation models with Detectron2 for advancing crack detection technologies, offering valuable insights for enhancing highway maintenance systems.