Multimodal comparative learning chip defect detection algorithm based on GLIP guidance

In the production of semiconductor chips, the existing process technology and the working environment have an impact on the quality of the chip, so defect detection on the chip surface is crucial. However, in real-world environments, it is challenging to collect a sufficiently large and highly representative sample of defects. In this paper, we propose a multimodal comparative learning approach with GLIP for location guidance and Multi-scale fusion modules for different multiscale fusions to localise defect locations of different shapes and sizes. In the testing phase, samples from different chip types in the training set were used to demonstrate the good generalisation ability and accuracy of our model. Data was tested on the MVTEC dataset to demonstrate the superiority of our method, where the image-level and pixel-level accuracies on our privately owned chip dataset can reach 91.3 and 92.6, and the pixel-level accuracy on the MVTEC is 92.3.