The impact of reference data quality on building segmentation: a nationwide comparison of 15 deep learning models and a new benchmark dataset

The effectiveness of deep learning models in detecting and segmenting buildings on orthophoto maps is usually attributed to the choice of network architecture; however, this study shows that the quality of the reference data can substantially influence measured segmentation performance. To verify this, a new dataset comprising 14,378 samples (11,878 positive and 2,500 negative) from across Poland was prepared, based on WMS services and vector data from the Land and Buildings Register (EGiB), and 15 neural network architectures were then compared on this dataset – ranging from classical convolutional networks, through transformer models, to instance segmentation from the YOLO-Seg family – evaluating them against both the cadastral labels and an independent manually annotated test set. The ranking of architectures differed markedly between the EGiB and manually annotated datasets, showing that the choice of reference labels can influence which model is identified as the best-performing architecture. The magnitude of the label effect was also substantial: across all 15 architectures, fine-tuning on more accurate manual labels increased IoU on the manually annotated test set by 0.128 ± 0.036 on average, whereas hyperparameter optimisation yielded an additional improvement of only 0.008 ± 0.010 – more than sixteen times smaller (Wilcoxon signed-rank test, p < 0.001). Under the conditions evaluated, label refinement improved performance considerably more than hyperparameter optimisation, and the reference labels used for evaluation altered architecture rankings, indicating that reference data quality should be treated as a key factor alongside model selection.