Benchmarking Open-Access Building Footprints: A Multi-Dimensional Assessment with High-Fidelity References

The rapid proliferation of open-access vector building footprint datasets has created substantial opportunities for large-scale urban analysis, yet their reliability remains difficult to assess effectively and consistently. This challenge arises primarily from the scarcity of large-scale, high-fidelity reference benchmarks across diverse urban environments. To address these limitations, we construct a benchmark-grade reference dataset comprising over 200,000 manually annotated building footprints across 10 purposively selected cities across diverse global regions through an exhaustive curation campaign and strict manual-automated quality validation. Using this dataset, we develop a three-tier evaluation framework that assesses statistical consistency, object recovery and spatial accuracy, and morphology fidelity among matched building pairs. Evaluating 11 mainstream datasets reveals pervasive underestimation in both building area (average bias: −14.9%) and count (−34.0%). Average city-level intersection-over-union values range from 29.1% to 58.3%, indicating low-to-moderate spatial overlap. For matched building pairs, the datasets demonstrate relatively high geometric fidelity, with morphological similarity scores of 0.495 to 0.803. Pronounced regional performance variations indicate that no single dataset is universally optimal. Specifically, Microsoft Global ML Building Footprints offers the most balanced multi-city performance, Google Research Open Buildings is recommended for boundary shape fidelity, and regional datasets remain competitive for local analyses.