TAPNet: Tunnelled Asymmetric Pyramid Network for Achieving a Pragmatic Semantic Segmentation Solution for Autonomous Driving

Semantic segmentation, vital in autonomous driving, is a pixel-level prediction task that is used to partition images into distinct regions. Effective models must capture multi-scale context to enhance segmentation quality but also need to be computationally efficient for practical deployment on resource-constrained devices. Hence, the challenge is to maximize accuracy and speed while minimizing model complexity to transform semantic segmentation into a successful practical reality. In response, we propose a Tunneled Asymmetric Pyramid Network (TAPNet) that combines short and long-range context with high efficiency. It is based on the TAP module, which extracts multi-scale short-range context in a highly efficient manner. Unlike conventional methods, our feature tunneling technique reinforces feature maps with finer details, enhancing segmentation accuracy. To efficiently capture long-range context, we propose the Pair-wise Aggregated Reinforced Scale-Enlarged Spatial Pyramid Pooling (PARSE-SPP) module, designed for ultra-low computational overhead. Based on the TAP and PARSE-SPP modules, we are able to achieve a high-performing segmentation model that strikes an optimal balance between accuracy and efficiency. This makes TAPNet a strong candidate for autonomous driving in real-time scenarios. More specifically, with only 0.29 million parameters, it achieves 68.75% and 68.6% mIoU on Cityscapes validation and test set, respectively. More interestingly, it achieves 161.55 FPS on a single RTX 3090 card. The related code is available at https://github.com/NadeemAtif-Alig/TAPNet.