Transfer Learning for Joint Ground Object Extraction and Height Estimation

Recent advancements in remote sensing have significantly enhanced ground object extraction capabilities from images. However, combining individual ground object extraction with accurate height estimation remains a challenging problem. With the rapid evolution of computer vision techniques, approaches leveraging convolutional neural networks (CNNs) and Transformers have demonstrated promising results in remote sensing image analysis. Despite these advancements, many existing methods superficially adapt models designed for other domains without fully addressing the complex relationships between multiple tasks inherent to remote sensing data. While transfer learning is commonly utilized to integrate multiple tasks, the variability of spectral channels, arising from the deployment of diverse sensors, introduces further complexity in the construction of robust datasets. To address these challenges, in this study, we propose a novel height estimation model that integrates multi-dimensional remote sensing images based on high-quality available datasets of city buildings. Our approach incorporates a shallow texture prompt, and combines a pre-trained model with an instance segmentation framework to better account for the distinct characteristics of remote sensing imagery. The proposed model achieves a correlation coefficient of 0.9439 on the height estimation task. Furthermore, it attains a box APso and mask APso of 73.6% and 71.4%, respectively, on the instance segmentation task, indicating its robustness and accuracy across multiple tasks. This method provides references for the potential applications in polar sea ice, iceberg, and marine ships estimations.