Quantitative estimation method for complex part surface defects based on multimodal information fusion

Surface quality is critical for the performance of high-end equipment, with defects potentially leading to severe operational failures. Current defect detection methods face challenges: 2D imaging lacks the ability to capture scratch depth, limiting quantitative damage assessment, while 3D point cloud methods are costly and time-consuming, hindering scalability in manufacturing. This study proposes a multimodal defect detection system (MDDS) that merges the benefits of 2D imaging and 3D point clouds for comprehensive defect analysis on complex parts. Utilizing a binocular vision system with high-precision industrial cameras, the system captures detailed 2D images and generates 3D point clouds through advanced reconstruction techniques. We enhance the Faster R-CNN network to improve defect localization and feature extraction, establishing a mapping between 2D images and 3D data to pinpoint defect-specific areas accurately. Additionally, we introduce a novel feature extraction approach using normal vector aggregation and the Fast Point Feature Histogram (FPFH) descriptor, combined with fuzzy C-means clustering, to detect and quantify scratch defects. This method assesses defect dimensions and depth, enabling precise damage classification. Tested on aero-engine impeller parts, our approach has proven effective in identifying and quantifying scratch defects on complex industrial components. The results demonstrate the system’s applicability and efficiency, making it a viable solution for practical implementation in industrial environments.