Similarity-Weighted IoU (sIOU): A Comprehensive Metric for Evaluating Model Performance Through Similarity-Weighted Class Overlaps

Semantic segmentation is crucial for comprehending a 2D image or a 3D point cloud by categorizing each pixel/point into a specific semantic class. A key objective of machine learning models used for segmentation is to improve the model accuracy and reliability by reducing misclassifications. Existing approaches treat all misclassification errors uniformly, lacking consideration for specific application requirements. Recognizing that some misclassifications may be more acceptable than others, depending on the particular class and application, we propose a novel Similarity-weighted Intersection over Union metric (sIOU). This approach provides a comprehensive evaluation framework, particularly in challenging scenarios with indistinct class boundaries, incorporating wrongly segmented classes with a similarity score. The metric offers a more refined assessment of a model’s segmentation performance in complex, real-world environments. Validation with an in-house indoor dataset demonstrates its superiority over existing metrics, highlighting its flexibility for diverse application contexts.