Enhancing Collaboration and Mitigating Conflict Between Subtasks for Remote Sensing Semantic Change Detection

Semantic change detection (SCD) consists of two separate but not independent parts, i.e., binary change detection (BCD) and semantic segmentation (SS), which aims to simultaneously locate changed areas and provide their semantic categories in bi-temporal remote sensing (RS) images. Although the existing SCD methods have achieved excellent performance, they still have two challenges: 1) Insufficient collaboration between BCD and SS: They can not efficiently leverage semantic information provided by SS to further improve performance of BCD and 2) Existing conflict between BCD and SS: They usually share input between SS and BCD branches to benefit from joint optimization. However, the shared input will be constrained by task-orientations of BCD and SS, which may produce conflict. Specifically, the task-orientation of BCD aims to align the distribution of bi-temporal domains, which inevitably results in the constraint of the shared input mining domain-specific features and further causes degradation of SS performance. Similarly, the task-orientation of SS also compromises domain alignment process of BCD as the shared input mines domain-specific features guided by the task-orientation of SS for improving its performance. Therefore, we propose enhancing collaboration-mitigating conflict network (EC-MCNet). Firstly, a semantic enhancement module (SEM) is designed to enhance inter-class variation and intra-class similarity of semantic features, which are beneficial for SS and BCD. Secondly, we no longer share input but share semantic contents between SS and BCD branches for mitigating the conflict. Specifically, we build a domain style removal module (DSRM), which ensures the input of BCD branch is removed from domain-specific styles and has the same semantic contents to the inputs of SS branches. In this way, the conflict between BCD and SS can be mitigated and the advantage of joint optimization can be preserved. Thirdly, we design a difference enhancement module (DEM) to enhance collaboration between SS and BCD, which leverages not only the attention of difference features but also the semantic similarity between bi-temporal features to enhance and identify changed areas of bi-temporal features. Extensive experimental results validate that our method outperforms state-of-the-art (SOTA) performances on two benchmark datasets for the SCD. The source code is available at https://github.com/yihui1230/ECMCNet.