Topics in Deep Learning Applied to Histopathology Images

The dissertation at hand presents the main concepts and results derived from research done as a doctoral student in Kasia Bozek's group "Data Science of Bioimages", and is presented in this thesis as a collection of scientific papers. The thesis touches on several themes, including the interpretation of deep models, multiple instance and federated learning algorithms, and language modeling. These topics are not studied standalone but boarded from the application of computer vision models in the automatic analysis of histopathology images. Emphasis is put on predictive tasks associated with the medical treatment of two diseases, namely, esophageal adenocarcinoma (EA) and cutaneous squamous cell carcinoma (CSCC), which were my main doctoral projects. The first of the presented works acts as an introduction to the discipline. It studies the prediction of a pathological grading on microarrays of esophageal tissue stained to reveal the presence of a known biomarker, the protein HER2, to identify good candidates for targeted EA therapy. The approach adopted in this paper is the training of an attention-based multiple instance learning classifier, and the explanation of its decision outputs with the aid of saliency maps. This method is the cornerstone of the analyses done in this thesis, and is refined in further chapters. The upcoming chapters deal with the more challenging problem of prediction of therapy effectiveness from pre-therapy biopsy images, in two different study cases: the response to neoadjuvant radiotherapy in EA patients, and disease progression of CSCC patients treated by tumor excision. Despite the radical differences in tumor biology and therapy procedures, these two problems share many similarities. First of all, this type of prognosis is not done by healthcare professionals, providing no human baselines, hypotheses, or plausible interpretation of results. In the second place, these tasks lack known biomarkers to look for, therefore the tissue sections are stained to reveal their general microscopic anatomy, providing fewer visual cues to learn from. Lastly, from the image analysis standpoint, both problems can be addressed with the same techniques. These two chapters extend the methodology presented in the first work by employing a transformer model as classifier, and an explainability algorithm that suits this new architecture. Additionally, a new analysis stage is added to investigate the cellular composition of relevant image regions via cell nuclei semantic segmentation, feature engineering, and statistical analysis. The last two showcased works branch from studying the aforementioned disease-specific applications, and explore visual aspects of learning from bioimages. The first of these chapters investigates the impact of pre-training a transformer model with natural language data before being applied to pathology slide classification, and how the visual information in such images can be summarized into smaller representations. The last work in this dissertation proposes a multiple instance learning algorithm incorporating the fact that coarse patterns of tissue morphology and organization are composed of smaller histological features.