A Multimodal Deep Learning Framework for Groundnut Defect Detection and Classification Using GAN-Augmented U-Net R Segmentation and Vision Transformer

Arachis hypogea (groundnut) is a large-scale oil crop whose quality is directly related to nutritional value, market price, and consumer safety, but is only checked by conventional inspection techniques involving manual evaluation and basic image processing, which only reveal the outward characteristics, but does not detect the internal flaws of the oil crop, including fungal contamination, abnormal oil-contents and internal damages. This work suggests a solution to these limitations, which is a multimodal deep learning model on automated groundnut defect detection and classification through the combination of hyperspectral imaging with sophisticated computer vision methods. Conditional Generative Adversarial Network (cGAN) is utilized to overcome the problem of data scarcity and class imbalance through the creation of realistic defect samples and hence improve diversity of the dataset and robustness of the model. Accurate localization of defective regions is realized with a GAN-enhanced U-Net R segmentation architecture that is based on residual learning with a Dice Coefficient of 0.95 and a Jaccard Index (IoU) of 0.90. A Vision Transformer (ViT) model is then used to classify defects and it uses self-attention to encode global contextual information on each segmented seed but the results are an overall classification accuracy of 97.2, a precision of 96.8, a recall of 97.0, and an F1-score of 96.9. Explainable artificial intelligence (XAI) systems such as Grad-CAM are added to increase transparency by drawing attention to the areas of defects that a model uses to make its decision. Moreover, a Digital Twin environment would also allow visualizing the results of the inspection in real-time, monitoring, and analyzing them in scenarios. The experimental evidence confirms that the suggested framework provides higher defect localization, high classification rate, and excellent generalization in various groundnut variety and image processing settings, which can be considered a scalable and reliable choice in terms of the intelligent inspection of groundnut quality.