A Transparent Search-Based Framework for Skin and Breast Cancer Diagnosis Using Siamese Networks

Computer aid diagnosis has developed digital pathology with Deep Learning (DL)-based tools to assist pathologists in decision-making. Content-Based Histopathological Image Retrieval (CBHIR) is a novel tool to seek highly correlated patches in terms of similarity in histopathological features. In this work, we proposed two CBHIR approaches on breast (Breast-twins) and skin cancer (Skin-twins) datasets for robust and accurate patch-level retrieval, integrating a custom-built Siamese network as a feature extractor. The proposed Siamese network generalizes for unseen images by focusing on the similar histopathological features of the input pairs. The proposed CBHIR approaches are evaluated on the Breast (public) and Skin (private) datasets with top <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> accuracy. Finding the optimum amount of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> is challenging; also, as much as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> increases, the dissimilarity between the query and the returned images increases, which might mislead the pathologists. This paper is tackling this issue for the first time on histopathological images by evaluating the top first retrieved images, while such approaches are underexplored in existing literature. The Breast-twins model achieves 70% of the F1-score at the top first, which exceeds the other recent methods at a higher amount of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> such as 5 and 400. Skin-twins overpasses the recently proposed Convolutional Auto Encoder by 67%, increasing the precision. Besides, the Skin-twins model tackles the challenges of Spitzoid Tumors of Uncertain Malignant Potential (STUMP) to assist pathologists with retrieving top <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> images and their corresponding labels. So, this approach can offer a more explainable tool to pathologists in terms of transparency, trustworthiness, or reliability.