Combined-net: modifying error in semantic segmentation based on bounding box annotations

It is vital to improve the performance of semantic segmentation networks, since they play an important role in many fields, such as self-driving cars, medical image analysis, robotic vision and road traffic detection. Some researchers combine object detection and semantic segmentation together, so that both networks can enjoy the benefits of each other. However, their proposed networks involve retraining, and some traditional semantic segmentation networks have yet to realize their full potential. In this research, we propose Combined-Net which incorporates existed object detection and semantic segmentation to improve the performance of semantic segmentation without resorting to retrain. It locates and revises the error segmentation based on information of bounding box annotations, and hence improves mIoU of traditional semantic segmentation. The following experiment in PASCAL VOC 2012 dataset shows that combining DeeplabV3+ and SSD in basic version of Combined-Net improves the mIoU of original network by 4.94%, while the improved version brings the mIoU of this combination to 74.37%. In best combination, Combined-Net increases this metrics by 11.68% with U-Net and SSD.