Extended Receptive Field UDA Semantic Segmentation Based on Spatial Alignment and Knowledge Distillation

In recent years, unsupervised domain adaptation (UDA) has significantly advanced, addressing the issue of requiring large amounts of labeled data in deep learning. Some UDA strategies have effectively alleviated domain shift, but they are still struggling to tackle challenges such as the model’s inability to continuously extract features, the difficulty to achieve better segmentation boundaries in the target domain, and tend to neglect previously acquired knowledge. To address these issues, we propose an extended receptive field UDA semantic segmentation based on spatial alignment and knowledge distillation (ERF). Firstly, based on the idea of combining serial and parallel, we design a novel large continuous receptive field decoder (largeCF) to extract large continuous receptive field features. This approach alleviates the bias of model in feature extraction between objects of different sizes and simultaneously reduces model complexity. Secondly, we propose an edge consistency strategy that aligns edge features and matches the spatial arrangement between predicted and ground truth labels, improving edges segmentation accuracy of the target domain. Finally, we employ a knowledge distillation module to achieve an optimized student-teacher framework, where the teacher effectively guides the student to retain previously learned information, resulting in more accurate segmentation of the target domain. Experimental results demonstrated the effectiveness of the proposed approach, which achieved mIoU of 76.5% and 68.8% on UDA benchmark tasks GTA<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\rightarrow $ </tex-math></inline-formula>CityScapes and SYNTHIA<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\rightarrow $ </tex-math></inline-formula>CityScapes, respectively. The code is available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/fz-ss/ERF</uri>. Note to Practitioners—This paper focuses on the challenges of semantic segmentation in autonomous driving, particularly facing the issue of extensive manual annotation required for dense semantic labels. We propose a novel extended receptive field UDA semantic segmentation based on spatial alignment and knowledge distillation. The article begins by outlining the initial implementation process of UDA, laying the foundation for subsequent in-depth discussions. Subsequently, we conduct a theoretical analysis of the Large Continuous Decoder, Boundary Consistency Strategy, and Knowledge Distillation Scheme, which constitute the core components of our method. Finally, experiments on two UDA benchmark tasks demonstrate the feasibility of our approach. However, the performance gap between UDA and supervised semantic segmentation still exists. In future research, we will focus on reducing the feature gap between different domains. Additionally, we will strive to fully use image features and distilled features to make greater progress, thereby driving advancements in semantic segmentation for autonomous driving.