Contextual evaluation of segmentation models using spatial reasoning

Image segmentation models are often evaluated using measures of overlap and boundary deviation between a ground truth and a prediction. These measures do not indicate whether a prediction is an overestimation or underestimation of the ground truth. This contextual information is critical in medical imaging applications such as tumor detection where a model’s tendency to overestimate a prediction would be preferred to avoid overlooking malignant cells. Spatial reasoning provides context on a model’s segmentation performance in terms of its tendency to over- or underestimate a region of interest. Such context can highlight a model’s decision-making trends and can be applied to inform targeted improvements. In this work, we provide a Python module 1,2 that implements a model-agnostic spatial reasoning pipeline for the contextual evaluation of segmentation methods. We apply this pipeline to the output of the Segment Anything model on 3 electron microscopy (EM) datasets and demonstrate the meaningful inferences that can be made.