Enhancing electronics grading: Detection-guided SAM inference for scratch segmentation and measurement

In every product reuse, defect detection and analysis of the severity of the detected defects are essential to correctly assess the product’s grade, which for most high-end electronics also defines to a large extent their residual value. However, the detection and measurement of small or narrow defects, such as scratches, remains a challenge for commonly adopted deep-learning segmentation approaches. In addition, pixel-level annotation that is required for the training of neural networks is highly time-consuming and labor-intensive. Therefore, to mitigate the subjectivity of purely manual defect inspection using human eyes and to reduce the labeling time required for training neural networks to perform this task, a method is proposed that combines an object detector with a Large Vision Model (LVM). This method allows the training of model with only bounding box annotations, while still producing segmentation masks that contour the scratches with high accuracies as output. Furthermore, for measuring scratch lengths, the K3M thinning method is utilized to compute the arc length of each detected scratch. To validate the effectiveness of the proposed method, a laptop dataset containing 220 images with 2,971 scratches is collected. Results demonstrate that the cascaded method adopting Faster R-CNN-FPN and the Segment Anything Model (SAM) achieves an accuracy of 83.7% for the scratch length measurement, which corresponds to a mean absolute error of 1.60mm with a measurement accuracy of 91.5%. These results indicate the great potential of the use of an LVM-driven approach for defect detection and enhancing the grading accuracy and objectivity in electronics reuse.