Deploying Edge LLMs for Wafer Defect Detection in Chip Manufacturing

Wafer defect detection in chip manufacturing is critical to ensuring yield and minimizing costly production delays. However, traditional cloud-based methods face challenges related to latency and data privacy constraints. This work investigates the deployment of edge-optimized large language models (LLMs) to enable localized, low-latency anomaly detection across fabrication processes. It analyzes existing edge AI frameworks for manufacturing analytics, emphasizing their limitations in processing scanning electron microscopy (SEM) image data with dynamic process variations. The integration of lightweight LLMs with edge computing architectures is proposed as a solution, leveraging model compression and hardware-aware optimization to balance accuracy and inference speed. By integrating the widely adopted FabGPT model into edge platforms, this approach compresses the model size by 3.12× and achieves a 6.87× speedup, and achieves a model size of 2.40 GB and a response throughput of 3.06 token/s on Jetson Nano.