Low-cost integrated circuit packaging defect classification system using edge impulse and ESP32CAM

Defects in integrated circuit (IC) packaging are inevitable. Several factors can cause defects in IC packaging such as material quality, errors in machine and human handling operations, and non-optimized processes. An automated optical inspection (AOI) is a typical method to find defects in the IC manufacturing field. Nevertheless, AOI requires human assistance in the event of uncertain defect classification. Human inspection often misses very tiny defects and is inconsistent throughout the inspection. Therefore, this study proposed a low-cost IC packaging defect classification system using edge impulse and ESP32-CAM. The method involves training a deep learning model (i.e., convolutional neural network (CNN)) using a dataset of non-defective and defective ICs on Edge Impulse. For defective ICs, the top surface of the ICs is deliberately scratched to imitate the cosmetic defects. ICs with scratch-free on their top surfaces are considered non-defective ICs. A successfully trained model using Edge Impulse is subsequently deployed on ESP32-CAM. The model is optimized to fit the limited resources of the ESP32-CAM. By using the built-in camera in ESP32-CAM, the trained model can perform a real-time image classification of non-defective/defective ICs. The proposed system achieves 86.1% prediction accuracy by using a 1,571 image dataset of defective and non-defective ICs.