Fusion of RGB and Near Infrared Image for Semantic Segmentation of Apple Defects Using Improved DeepLabv3+

Accurate detection of apple defects, including the location, area, and type of defects, can help prevent the spread of defects and assist in fruit quality management. Therefore, we proposed a semantic segmentation network ADS-DeepLabv3+ for apple defects semantic segmentation. We introduce CBAM between the backbone network and ASPP to measure the channel and spatial weights of the feature maps output by the backbone network and introduce a feature fusion branch based on FPN in the backbone network, and finally construct ADS-DeepLabv3+ for semantic segmentation of apple defects. ADS-DeepLabv3+ got the 3.8% and 3.81% increase in MIoU and MPA than DeepLabv3+. RGB images struggle to detect invisible defects. Considering that near infrared (NIR) imaging is sensitive to changes in object structure and material. We integrate RGB and NIR images with ADS-DeepLabv3+ for semantic segmentation of apple defects. ADS-DeepLabv3+ combined with RGB and NIR images for apple defects segmentation outperformed RGB images alone, with a higher MIoU of 3.57 % and a higher MPA of 2.83%. The proposed method can effectively segment apple defects.