ITT: Long-Range Spatial Dependencies for Sea Ice Semantic Segmentation

Processing SAR images for sea ice mapping using deep learning is typically constrained to analyzing neighboring pixels rather than studying visual patterns at a large scale. This quality ignores the wide within-class variance and spatial non-stationary class statistics observed by covering complete scenes. To leverage global and local information and improve automated sea ice mapping, we introduce the Irregular Tokens on Transformers (ITT) method by uniquely combining convolutional neural networks (CNN), transformers, and unsupervised segmentation. The method is trained to capture long-range spatial dependencies to overcome non-stationary class statistics. In addition, ITT incorporates a novel loss function, including a pixel-based term that guides the semantic segmentation and a region-based term that promotes region-consistent feature maps, increasing the classifier confidence and improving the predictions along object boundaries. We evaluate the method by classifying ice and water on dual-polarized RADARSAT-2 scenes. The evaluation metrics depict the best performance, generally, when the training leverages both pixel and region level loss terms, after pre-training the CNN. In such a case, prediction maps are less sensitive to non-stationary patterns, more detailed on ice/water boundaries, and less uncertain separating the classes.