Leveraging Physical Augmentations From Multiview Remote Sensing Images for Building Segmentation

Building segmentation from remote sensing data has been boosted in recent years by the advances in data processing algorithms and the improved ability of sensors to capture very high spatial resolution (VHSR) images. The prevailing approaches to this task are built upon deep semantic segmentation networks learned on ortho geometry, i.e., orthorectified images. In this study, we propose to deal with the building segmentation using multiview Pléiades satellite images at the perfect sensor (PS) geometry instead of the ortho geometry. This has two main advantages: 1) it frees the segmentation workflow from geometric imprecision that may arise after the image rectification step and 2) it allows the physical augmentations of the ground truth (GT) by reprojecting building objects on each of the multiview acquisitions. The GT reprojection process makes use of rational polynomial coefficients (RPCs) provided as image metadata and a fine scale digital surface model (DSM). We assess the benefit of our proposal using a U-Net encoder-decoder learned on a dataset composed of tri-stereo Pléiades acquisitions over six French cities. Experimental results demonstrate the significance of the proposal especially an enhanced generalization capability for the building segmentation.