Industrial-Mass Mini/Micro LED Sorting Using Patch Enhanced Lightweight Self-Attention Hybrid Neural Network

Mini/Micro LEDs are becoming the next generation of displays, with sizes shrinking to less than 100 <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">μ</i>m/50 <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">μ</i>m and integration scaling increasing by more than 100-fold. Fast and precise sorting, which includes the identification and localization of mass mini/micro LED chips, has become an urgent need in the industry due to its high efficiency and reliability. However, the small size, high density, and weak defects of the mini/micro LED, combined with the industrial rapid sorting requirement, pose challenges to the sorting task. To address these issues, we propose a fast and precise visual sorting approach that includes accurate chip identification and pixel-level localization. In detail, a double-ended self-attention (SA) encoder–decoder hybrid convolutional encoder neural network framework is designed to accommodate both local features and global information. Then, two generic patch enhancement modules are constructed to compensate for the local feature modeling inefficiencies inherent in SA. Finally, the light SA encoder (Li-SA Co) and light SA covariance decoder (Li-SA-Cov Dec) basic blocks are proposed to speed up sorting. The effectiveness of the proposed approach is demonstrated by the actual mini/micro LED industrial images acquired, which have an mAP of 97.1%, a localization success of 98.6%, and a speed of 20.1FPS, all of which are higher than those of other state-of-the-art methods.