HuIV-GAN: A Human-in-the-Loop VAE-GAN for Industrial Image Synthesis in Class-Imbalanced Ball Grid Array Object Detection

In industrial inspection, severe class imbalance poses a major challenge for training accurate detection models. Generative methods offer a potential solution by augmenting minority-class images; however, most existing approaches exhibit poor generalization under extremely low-shot conditions and often produce semantically implausible results. To address these limitations, a human-in-the-loop generative framework, HuIV-GAN, is proposed. It integrates a variational autoencoder with a generative adversarial network to synthesize images under conditions of severe data scarcity. HuIV-GAN incorporates real-time human feedback to refine generation iteratively: experts assess intermediate outputs, enabling a closed-loop <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">generate–filter–retrain</i> process that ensures both structural realism and semantic accuracy. The proposed method is validated on a typical detection task to solve the scarcity of solder joint fracture data in ball grid array (BGA) packaging. The red-dye BGA dataset has a highly imbalanced label distribution, with the smallest minority class constituting less than 0.1%. Our method significantly improved minority class detection under extreme imbalance, whereas detection models trained on the original dataset failed to detect this class entirely. Furthermore, the feedback loop reduced training instability and accelerated convergence by over 200 iterations.