Poly R-CNN: Efficient large-scale boundary-regularized building footprint extraction from remote sensing images

Abstract Building footprint datasets are valuable for various applications, but they are challenging to prepare due to the labor-intensive and time-consuming manual delineations of building footprint labels by human image interpreters. Although deep learning-based building extraction methods have been introduced in recent years, most of them rely on a two-stage approach involving a polygon initialization network and a polygon regularization network. These methods often suffer from high computational costs and slow inference speeds, limiting their suitability for large-scale building extraction tasks. Furthermore, achieving consistent regularization across different building types remains a challenge. This paper introduces Poly R-CNN, a region-based convolutional neural network designed for efficient, large-scale, boundary-regularized building footprint extraction. Built on the Mask R-CNN model, Poly R-CNN incorporates a lightweight polygon prediction head that works alongside the existing bounding box and mask prediction heads. This new head comprises a Vertex Proposal Network (VPN) and a fast edge-tracing algorithm. The VPN predicts instance-aware building vertex proposals, while the edge-tracing algorithm selects, traces, and refines these proposals into enclosed building polygons. Poly R-CNN outperforms two state-of-the-art deep-learning-based methods, PolygonCNN and PolyMapper, with a higher F1-score (0.722 vs. 0.663 and 0.591, respectively) and faster inference speed (248.87 ms per image vs. 457.11 ms and 604.29 ms). Additionally, unlike previous methods limited to single-image processing, Poly R-CNN supports batched training and inference which further enhances its training and inference speed. Experimental results demonstrate that Poly R-CNN can effectively extract regularized and visually appealing building polygons from large-scale urban scenes.