Assessing the importance of soft labels and probability calibration for sea ice classification in synthetic aperture radar imagery

The Arctic is currently experiencing unprecedented changes, including rapid reductions in sea ice that must be characterized quickly to ensure safe Arctic navigability. While the need for ice classification is well-suited for satellite synthetic aperture radar (SAR) and machine learning (ML) solutions, there is a lack of labeled datasets at the spatial and temporal resolutions needed for pairing with fine-resolution SAR imagery. Previously, we developed an approach to obtain fine resolution ice labels by exploiting polarimetric relationships in single look complex-format (SLC) Sentinel-1 (S1) SAR imagery. The probabilistic nature of this novel approach allows for uncertainty measurements (soft labels) in addition to binary water versus ice labels (hard labels). To determine the effectiveness of these labels, we use them to train ML models with S1 GRD intensity products as input. We consider Na¨ıve Bayes, Random Forest, and XGBoost classifiers, examining the trade-off in model complexity versus recall of ice when trained on hard labels versus soft labels. In addition to assessing if training on soft labels prevents overfitting, we also test the impact of probability calibration on output label probabilities. We use S1 acquisitions that overlap with the AI4Arctic data set for training and testing. We find that training on soft probabilities is beneficial as model complexity increases, emphasizing the value added by our probabilistic approach to sea ice classification. We additionally find that in the absence of soft training labels, probability calibration is important for obtaining representative label probabilities. Moving forward, we will extend this assessment to deep learning models where such effects may be even more substantial.