Context-Guided Multiscale Attention for Real-Time Semantic Segmentation of Road Scene

Lightweight deep neural networks have played a pivotal role in real-time semantic segmentation for autonomous driving in resource-constrained devices, which need to effectively learn the local semantics and global context at multiple scales due to varying object sizes. Recent methods design shallow and lightweight backbones with a small receptive field for faster inference, along with additional mechanisms such as attention to compensate for the accuracy loss due to the lightweight design. While some methods have exploited multi-scale feature learning by attaching pyramid modules at the encoder end, it is often neglected at the fundamental block level due to increased inference time. Furthermore, the attention weights are mostly generated at a single object scale by only using the high-level feature representations. To solve the first problem, a key module for the basic block, the Fast Hybrid Module (FHM), has been proposed. This module uses a hybrid approach to learn multi-scale features by combining dilated kernels and downsampling operations in a parallel three-branch structure. To solve the second problem, a novel attention module, the Multi-Scale Attention Module (MSAM), is proposed. MSAM uniquely generates context weights at varying scales from the low-level features rich in object boundary and edge information and multiplies them by the high-level semantic features obtained from the encoder. With these modules, a novel encoder-decoder network named Context Guided Multi-scale Attention Network (CGMA-Net) is proposed. With only 0.54 Million parameters, the network achieves 73.4% and 68.1% mean IoU accuracy at 128.24 and 85.5 FPS on the Cityscapes and CamVid datasets, respectively. In addition, the network can run in real-time on embedded GPUs with resource constraints. Through extensive ablation studies, the effectiveness of the proposed modules and network is shown. The qualitative results on unseen data demonstrate the robustness of the method. Code available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/saquibmazhar/SemanticSegmentation-CGMANet</uri>.