Deep learning-based rebar detection and instance segmentation in images

• A high-quality image benchmark is developed for rebar detection and segmentation. • The dataset covers two types of rebars under various shooting views and placement patterns. • The comparison test provides the basis for selecting deep learning models. • Six data augmentation strategies are discussed to improve the best models. • Three downstream tasks in real-world scenarios validate the developed model. Automated rebar cage assembly and quality inspection rely on reliable rebar perception. Recent studies have explored image-based rebar perception via object detection and instance segmentation algorithms. However, existing models are limited across various scenarios, especially with different rebar categories, arrangement patterns, and camera views, which limits their application. This is primarily attributed to the absence of a benchmark considering these factors. This study introduces an image benchmark designed for the effective training and selection of rebar detection and instance segmentation algorithms. It is the first to encompass two types of commonly used rebars, multiple camera views, and various rebar placement patterns at different assembly phases in a single dataset. Six object detection methods and four instance segmentation methods are evaluated to assess the applicability of the state-of-the-art methods. Additionally, a new shape-prior-based post-processing method is developed to address the merged detection problem in clustering. The experiment shows that Deformable DETR and Mask2Former achieved the highest bounding box mAP (80.4) and mask mAP (66.3) respectively. The Simple Copy-Paste technique was introduced, improving the mask mAP of Mask2Former by 2.8 points. Finally, the developed model was validated in the real-world scenarios of three downstream tasks. Notably, in the rebar spacing measurement task, the proposed post-processing method improves Mask2Former by increasing its bounding box mAP by 18.0 and mask mAP by 2.4.