A context aware multiclass loss function for semantic segmentation with a focus on intricate areas and class imbalances

Image segmentation models play an important role in many machine vision systems by providing a more interpretable representation of images to computers. The accuracy of these models is vital, as it can directly impact the overall performance of the systems. Therefore, making any progress in this component would be very critical. To improve this aspect, we have developed a new loss function, named SPix-WCE, to boost the performance of deep neural networks in image segmentation tasks. Our primary goal is to address imbalances in image datasets by identifying complicated areas in the images and bringing them more into focus during the model training process. This was achieved by utilizing the SLIC algorithm and analyzing each superpixel to detect key regions in images, followed by implementing a weighting scheme to control the influence of each area in the loss calculation. Subsequently, we carried out a series of experiments to validate our approach. These experiments involved three different models and four multiclass datasets with various degrees of imbalance. The models were trained and tested using the proposed loss function as well as other commonly used ones. The outcomes of our experiments demonstrate that using SPix-based losses led to better results in terms of IoU, F1-Score, and pixel accuracy metrics compared to other methods.