Conditional generative adversarial network for segmentation and image reconstruction of grape berries in variable illumination conditions

One of the most significant challenges faced by computer vision algorithm developers is effectively handling images captured in variable illumination scenarios, which may include poorly lit environments, object occlusions, reflections, or even partial visibility. The introduction of deep learning techniques has marked an unprecedented advance in segmentation, identification, and object reconstruction in such environments. This research presents a deep learning architecture based on a Conditional Generative Adversarial Network (cGAN) specifically designed for various grape berry segmentation varieties under any illumination conditions. It exploits a dedicated data collection for training and testing the cGAN segmentation model, that consists of 25,350 images of five grape berry varieties, divided into two sets: 20,350 for training and 5,000 for testing. To evaluate the impact of convolution kernel sizes on the segmentation process within the GAN architecture, six models were trained using kernel sizes of 2x2, 4x4, 6x6, 8x8, 10x10, and 12x12. Five metrics (accuracy, precision, recall, F 1 -score, and IoU) were employed to facilitate comparison of result with similar segmentation methods. The metrics have been calculated on a dataset with images in variable illumination conditions and in five specific scenarios with images categorized as: (a) good illumination, (b) poor illumination; (c) lack of illumination; (d) noisy image and (e) noisy and poor illumination. Among all the assessed model configurations, the cGAN model with a convolution kernel size of 6x6 exhibited the best performance on the test set, achieving the following metrics: accuracy of 0.9970, precision of 0.9862, recall of 1.0000, F 1 -score of 0.9930, and IoU of 0.9813. Notably, when compared to other similar grape segmentation studies, the Pix2Pix architecture utilized in this work consistently delivered the best results across all the evaluated metrics. Furthermore, the proposed architecture is able to reconstruct images obtained under noisy and poor illumination conditions indicating its great potential under difficult settings.