Learning Domain Generalized Remote Sensing Image Segmentation by Multiscale Instance Disentanglement

Remote sensing image segmentation is a fundamental task in Earth observation. Rapid development has been made in the past decade owing to the deep learning techniques. Most of the existing methods assume that the training and inference remote sensing images hold the identical and independent distribution (i.i.d.). However, in practical cases, the pre-trained segmentation model has to deal with remote sensing images that have not seen before. In this paper, we advance this bottleneck by introducing the domain generalization paradigm to remote sensing image segmentation. It does not need to involve any images from the target domain during training, compared with the existing domain adaptation paradigm. This paper focuses on the key challenges from the various instance size and dramatic image appearance from cross-domain remote sensing images, and proposes a novel multi-scale instance disentanglement (MID) learning scheme. Technically, it consists of three key components, namely, multi-scale instance encoding, domain-invariant instance disentanglement and domain generalized semantic decoding. The proposed multi-scale instance encoding module incorporates both depth convolution and multi-scale representing, so as to learn robust semantic representation despite the cross-domain scale variation. The domain-invariant instance disentanglement module, on the other hand, helps eliminate the impact of cross-domain styles. Finally, the domain generalized semantic decoding fuses the style-invariant representation and outputs the per-pixel semantic prediction. Extensive experiments on four cross-domain settings show its effectiveness over existing domain adaptation and domain generalization methods from both remote sensing and computer vision community, yielding an mIoU improvement of 3.3% and 6.0% on the Postdam and Vaihigen unseen target domain over the second-best, respectively.