A triple-task learning network for high-resolution remote sensing images semantic segmentation

Conventional multi-task learning (MTL) frameworks that incorporate height estimation often suffer from limited texture representation and inadequate task interaction in high-resolution remote sensing image segmentation, primarily due to the semantic gap between 2D and 3D features and the interference caused by objects with similar spectral-spatial characteristics. To address these issues, we propose a novel triple-task learning network that contains semantic segmentation, height estimation, and masked image modelling (MIM), abbreviated as TTL-Net. The auxiliary MIM branch encourages the network to learn fine-grained texture and structural information through masked region reconstruction, thereby improving boundary delineation and structural consistency. Moreover, a cross-task attention (CTA) module is constructed to explicitly capture inter-task dependencies by enabling task-guided feature interaction. Specifically, the CTA allows the segmentation branch to selectively integrate geometric cues from height estimation and texture details from MIM based on spatial relevance, thereby reducing task conflicts and mitigating negative transfer. Experimental results demonstrate that TTL-Net achieves an mF1 score of 90.4% and an mIoU of 82.6% on the Vaihingen dataset, and 92.3% mF1 and 85.9% mIoU on the Potsdam dataset, consistently outperforming representative state-of-the-art methods.