HISNet: a hybrid instance segmentation network for urban street scenes combining top-down and bottom-up approaches

Instance segmentation is an important task in computer vision with wide applications in autonomous driving. Instance segmentation in autonomous driving aims to associate each pixel in an image with its corresponding object instance, enabling precise segmentation and recognition of different objects. These objects can include pedestrians, vehicles, bicycles, traffic signs, and more. In instance segmentation, common methods include top-down and bottom-up approaches. The top-down approach first performs object detection to generate candidate proposals and then performs pixel-level segmentation on each proposal. It is accurate and flexible, capable of handling objects of different sizes and shapes. However, it has high computational complexity and relies on the accuracy of object detection. The bottom-up approach first performs pixel-level clustering or segmentation and then combines candidate instances to obtain the final segmentation result. It can handle overlapping instances and has lower computational complexity but may not accurately localize and segment instances and may have coarser segmentation granularity.This paper proposes a hybrid model called HISNet that leverages the advantages of both top-down and bottom-up strategies. In the prediction stage, the paper introduces an innovative dual-branch design. One branch is the bounding box aggregation branch, which generates high-dimensional information such as the shape and pose of the bounding boxes based on the FCOS Head. The other branch is the mask decoding branch, which generates mask prediction results. These two branches are fused using the Mask FCN Header. Additionally, the model adopts EfficientNet as the backbone to improve accuracy and inference efficiency, and the Neck module incorporates the MPAFPN network module to enhance feature fusion. With these improvements, HISNet achieves an improvement of approximately 1.95% to 5.0% over the baseline model on the COCO dataset. This indicates that the proposed model performs better in urban street scenes, further enhancing object detection and segmentation tasks. The HISNet model holds the potential to provide more reliable and efficient solutions for applications in autonomous driving, intelligent transportation, urban security, and other fields.