Performance analysis of modified DeepLabv3+ architecture for fruit detection and localization in apple orchards

Deep learning plays an important role in automating various operations in fruit crop production including irrigation, nutrition management, yield estimation and harvesting. Yield estimation is essential in fruit crop production as it helps farmers optimize cultivation, harvesting, logistics and marketing operations. Furthermore, fruit detection and localization is a very important step in the development of an automated fruit harvesting system. Hence, an intelligent system was proposed in this study for apple fruit detection and localization using modified DeepLabv3+, semantic segmentation based architecture. The finetuned customizations (such as modifying the activation function, optimization technique and loss function) were performed in the original architecture of DeepLabv3+ and its performance was analyzed. The modified model was trained with the training dataset of 2600 apple tree images. Images were split into 80 % of training and 20 % of validation. The modified architecture was also compared with the other variants of DeepLabv3+ architectures. After training, the model was tested with the unobserved test dataset of 101 images. The test results demonstrated the Mean Accuracy ( M A c c ) of 98.58 % and the Mean Intersection over Union ( M I o U ) of 96.66 % without compromising the inference time (i.e., 15 ms). The proposed model revealed the improved results than the original model which attained a M A c c of 92.12 % and M I o U of 88.94 % for the same dataset with the inference time of 40 ms. To ascertain further, the modified model was compared with other single stage detectors, including Fully Convolutional Network (FCN) and U-Net. FCN attained a M A c c and M I o U of 77.5 % and 77.27 %, respectively whereas U-Net resulted a M A c c and M I o U of 83.95 % and 81.09 %, respectively. Results demonstrated that the modified DeepLabv3+ with ResNet18 is capable of detecting the apple fruits by mitigating the effects of class imbalance which is the major drawback in single stage detectors. Further, better detection and localization of apple fruits can lead to the precise picking by the robotic system.