PathInteract: a video analysis suite for capturing and analyzing pathology slide reviewing interactions

Digital pathology has transformed how pathologists review and interpret tissue specimens, enabling remote access, efficient storage, and advanced visualization. Systematically analyzing pathologists’ slide reviewing interactions can uncover opportunities to design AI-assisted tools that integrate seamlessly into routine practice. We present a proof-of-concept suite, PathInteract, for extracting pathology interaction signals from recorded slide-review videos. Mouse cursor movements, viewport actions (i.e., zooming and panning), and verbal narratives are extracted using multiple deep learning and computer vision moduals. Unlike prior methods requiring specialized software or equipment, our approach operates on screen recordings, enabling broader applicability. We developed PathInteract using QuPath-recorded diagnostic sessions with view-tracking logs and applied it to ten educational YouTube videos. Results revealed distinct interaction patterns between two pathologists with different levels of experience, tissue type, and use context. Cursor tracking and viewport detection achieved strong agreement with ground truth, while speech analysis highlighted differences in cell-level focus across diseases. PathInteract enables scalable analysis of reviewing interactions in digital pathology through video recording and supports repurposing of existing pathology videos towards building interpretable, multimodal pathology AI datasets.