Automated Classification and Captioning for Wafer Bin Map Using Attention-based Image Captioning Approach

Analyzing defect patterns in Wafer Bin Maps (WBM) is an essential part of the semiconductor manufacturing process. In recent years, as semiconductor products have diversified and the number of chips integrated on a wafer has increased, critical dimensions have been shrinking and manufacturing processes have become more complicated. As a result, conventional defect pattern classification systems struggle to effectively analyze diverse defect variations within the same class or accurately identify the manufacturing processes responsible for them. To address these issues, this study introduces image captioning techniques and proposes a model that utilizes a Convolutional Neural Network (CNN)-based model to classify WBM defect patterns in the MixedWM38 dataset and automatically generate captions for the corresponding defect patterns. For this purpose, we divided a total of 6,000 WBM images into 4,800 training data and 1,200 test data, extracted features using a CNN encoder, and calculated weights reflecting the importance of each feature by applying an attention mechanism. Afterward, we passed them to an LSTM (Long Short-Term Memory) model along with the existing feature map to generate captions for each WBM image. By automating the classification and caption generation of WBM defect patterns, the model proposed in this study is expected to provide more consistent and reliable defect analysis results compared to conventional manual methods. In addition to classifying WBM defect classes, the model can generate captions that describe the size, length, and shape of defect patterns in detail, even without explicit subclass labels. The proposed model is expected to contribute to the optimization of semiconductor manufacturing processes and enhanced quality control.