Comparison of Manual Versus QuPath Software–based Immunohistochemical Scoring Using Oral Squamous Cell Carcinoma as a Model

Gold standard for immunohistochemical analyses is the manual assessment by two specialist pathologists. This process is time-consuming, highly dependent on the respective evaluator and often difficult to reproduce. The use of image analysis software, such as ImageJ, QuPath, or CellProfiler, which employ machine learning and/or deep learning mechanisms to perform biomarker analyses, offers a potential solution to these problems. The objective of our study is to evaluate whether digital assessment using the open-source software QuPath is comparable to manual evaluation and to examine the inter-evaluator variability between the two manual evaluators and two software-based evaluations. Six tissue microarrays (TMAs) were constructed for a cohort of 309 patients with primary oral squamous cell carcinoma (OSCC). The tumor tissue and corresponding non-lesional squamous epithelial mucosa specimen were immunohistochemically stained for the biomarkers Ki67, as a nuclear marker; the epidermal growth factor receptor (EGF-R), as a membranous marker; and the major histocompatibility complex class I (MHC-I) heavy chain (HC) expressed on the membrane and in the cytoplasm. The staining pattern was analyzed by two experienced, independent manual evaluators and by QuPath. The percentage of positive cells, for Ki67, and the histoscore (H-score) based on the percentage of positive cells and their staining intensity, for EGF-R and MHC-I, were determined as final values. The results yielded high to excellent spearman correlation coefficients for all three biomarkers (<i>p</i><0.001) in lesional and non-lesional tissues. The Bland-Altman plots demonstrated a high degree of agreement between manual and software-based analysis, as well as inter-evaluator variability demonstrating a high comparability of the evaluation methods. However, a prerequisite for a proper software-based analysis is an accurate, time-consuming annotation of the single specimen, which requires users with a comprehensive understanding of histology and extensive training in QuPath. Once these requirements are met, the software-based analysis offers advantages for large-scale biomarker studies due to objective and reproducible comparability of the stainings leading to a greater accuracy as well as the reuse of established conditions across similar analyses without requiring further operator input.