Optimizing Cell Recognition in Sarcoma Multiplex Immunohistochemistry Images via Pre-Training Strategies

Comprehensive profiling of the sarcoma immune microenvironment offers insights into varying immunotherapy responses. Multiplex immunohistochemistry (mIH <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$C$</tex>) enables staining multiple markers, allowing for studying immune cell composition and spatial interactions within the same tissue. However, the morphological complexity of mIHC images poses fundamental challenges for precise cell recognition. Moreover, obtaining large-scale annotations is difficult due to the required intensive labor and domain expertise. We address the challenge by proposing a three-step training mechanism utilizing pretraining strategies. Specifically, we first pretrain the ViT encoder on large-scale natural images, followed by pretraining segmentation decoders on the medium-scale Pan-Nuke H&E dataset, and finally, fine-tune the cell recognition model on small-scale sarcoma mIH <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$C$</tex> images. The proposed training method achieves a mean panoptic quality (mPQ) of 0.435 and an <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$F_{1}$</tex> -detection score of 0.702, outperforms training from scratch by a substantial margin, underscoring the critical value of pretraining strategies. The proposed training mechanism holds great potential for other cancer-specific, staining-unique data with limited annotations.