SS-DeepSeg: An Efficient Deeplab with Smart Scaling for Robust Semantic Segmentation

Semantic segmentation is a critical task applied for scene understanding in computer vision with widespread applications in autonomous driving, remote sensing, and more. Despite DeepLabV3's strong segmentation capabilities, further improvements can enhance its computational efficiency and accuracy in complex scenarios, while handling the demands of realtime operation in resource-constrained environments. To address this, we propose three key enhancements to the DeepLabV3: (i) a lightweight backbone to reduce computational overhead while preserving feature extraction capabilities, (ii) a smart scaling strategy to enhance multi-scale feature aggregation and contextual understanding, and (iii) an atrous spatial pyramid pooling (ASPP) module augmented with attention for dynamic receptive field-based feature refinement. Comprehensive experiments on benchmark datasets validate the effectiveness of our approach, achieving a mean intersection over union (mIoU) of 73.78 % on the Cityscapes validation set -a 6.23% improvement over the baseline model. It also achieves a competitive 76.96 % on the CamVid test set, and 51.71 % on the LoveDA test set.