Probing deep histological segmentation models to identify specific performance boundaries

Whole slide images (WSI) are the gold standard for evaluating cell and tissue states. Segmentation is a crucial first step in extracting cell morphological features and determining tissue status. WSI segmentation is difficult to automate because of the sheer size of the images, the natural imbalance in tissue and cell types present, and the variance in appearance due to differences in dye concentration between samples. Deep learning (DL) models can address these challenges, given enough compute nodes, training time, and data. However, it is difficult to understand the drivers of failure modes in DL models. This makes it challenging to create techniques to directly address such failure modes. Here we present a method for identifying the relationship between object size and structure, and the resulting segmentation class output by a DL segmenter. This insight is valuable because it can inform the development of techniques to directly improve segmentation performance. This could provide an alternative to the expensive industry approach of massively scaling data and is a promising step towards higher quality metric extraction following segmentation.