HENet: A Height Estimation Network for Estimating the Height of Buildings from Single Optical Image

Estimating building height from a single optical (RGB) image is a significant task with applications in urban planning and resource management. Recent advances in deep learning have shown great potential in this domain. In this paper, we propose a deep learning network named HENet, designed to estimate the height of buildings using a classification-regression paradigm. Specifically, we first employ the Visual Geometry Group (VGG16) to extract high-dimensional semantic features. These features are then fused through skip connections between the encoder and decoder. To enhance global information, we incorporate an attention mechanism within the decoder, addressing the local attention limitation of convolutional layers. The decoder generates a rough height map, refined by our proposed segmentation based Adabins (SAdabins), following the classification-regression paradigm to obtain a fine-grained result. Remote sensing images often contain extensive background, leading to a long-tailed distribution of low-height pixels and resulting in underestimated height values. To address this, we introduce the Segment Anything (SAM) to obtain segmentation maps that filter out background in SAdabins. Experiments show that HENet outperforms other baselines in visualization quality and Quantitative evaluations.