Leci: Learnable Evolutionary Category Intermediates for Unsupervised Domain Adaptive Segmentation

To avoid the laborious annotation process for dense prediction tasks like semantic segmentation, unsupervised domain adaptation (UDA) methods have been proposed to leverage the abundant annotations from a source domain, such as virtual world (e.g., 3D games), and adapt models to the target domain (the real world) by narrowing the domain discrepancies. However, because of the large domain gap, directly aligning two distinct domains without considering the intermediates leads to inefficient alignment and inferior adaptation. To address this issue, we propose a novel learnable evolutionary Category Intermediates (CIs) guided UDA model named Leci, which enables the information transfer between the two domains via two processes, i.e., Distilling and Blending. Starting from a random initialization, the CIs learn shared category-wise semantics automatically from two domains in the Distilling process. Then, the learned semantics in the CIs are sent back to blend the domain features through a residual attentive fusion (RAF) module, such that the category-wise features of both domains shift towards each other. As the CIs progressively and consistently learn from the varying feature distributions during training, they are evolutionary to guide the model to achieve category-wise feature alignment. Experiments on both GTA5 and SYNTHIA datasets demonstrate Leci's superiority over prior representative methods.