Label-Free Quantitative Evaluation of Land Cover Segmentation Models for Remote Sensing Imagery: A Context-Aware Model Based on OpenStreetMap and a Digital Surface Model

Segmentation performance is measured against pixel-wise ground truth (GT); without it, that measurement cannot be made. Remote sensing segmentation models are evaluated likewise, yet operational systems receive new unlabeled images, and the properties of those images shift with season, region, and sensor aging. A way to check performance outside the training dataset is needed. We propose quantifying segmentation output without labels, using public geospatial data: a random-forest context-aware model predicts class area composition from OpenStreetMap (OSM) and digital surface model (DSM) statistics. Quantile mapping transforms its output to match the distribution of measured compositions, yielding the expected distribution, and the Label-Free Composition Score (LFCS) is the similarity between that distribution and the composition of the segmentation output. The context-aware model is fitted only to geospatial features and GT compositions from the segmentation model’s training dataset, so the expected distribution is independent of the segmentation model’s structure, parameters, and outputs while depending on that dataset’s GT, class scheme, and composition distribution. The evaluation stage requires no GT labels. To test the method, we saved 430 segmentation model checkpoints from 75 training runs under six controlled training variations, so that the mean intersection over union (mIoU) of each, measured against GT on validation tiles held out of training, was known in advance. These GT mIoU values ranged from 0.04 to 0.69. The models were grouped by GT mIoU in steps of 0.1. Their mean LFCS rose with every step: from 0.474 to 0.937 on held-out images from the same dataset, and from 0.467 to 0.893 on 900 globally distributed unlabeled images. Existing measures that read only a model’s own output tracked GT mIoU less consistently. Rebuilding the entire method reproduced the same behavior on two public datasets with different class schemes, confirming the validity of the method under the conditions tested.