Abstract 6407: Deep learning-driven image analysis for tracking nucleolar morphology

Abstract Introduction: The nucleolus, a membraneless nuclear organelle responsible for ribosome production, is highly sensitive to stress. A subset of chemotherapeutic drugs induce nucleolar stress, which could influence their apoptotic mechanisms. However, tracking morphological changes of the nucleolus during treatment remains challenging due to its amorphic state and nanoscale organization. A major bottleneck for drug screening and mechanistic studies is the reliance on time-intensive manual imaging and immunostaining to identify nucleolar stress. To overcome this we have developed a suite of image analysis models to rapidly identify and classify cells. Methods: The mechanisms that lead to nucleolar stress by small molecules and mechanisms that allow the nucleolus to recover from reversible stress remain unclear. To facilitate time-dependent morphological studies as well as general drug screening for nucleolar defects, we developed deep learning models leveraging Thermo Fisher Scientific’s RNA-selective dyes to automatically detect and quantify nucleolar stress from microscopy images. Using osteosarcoma cells, we successfully classified stressed and unstressed populations with SYTOTM RNASelectTM Red, a novel RNA-selective dye developed by Thermo Fisher Scientific. This dye can be implemented into rapid high-throughput imaging methods for cancer detection and drug screening. Using these deep-learning models we can rapidly screen for novel chemotherapeutics and study the efficacy of current nucleolar stress-causing drugs. Results: Osteosarcoma cells labeled with SYTOTM RNASelectTM Red and treated with chemotherapeutics display distinct nucleolar staining enabling precise segmentation and quantitative analysis. Under chemotherapeutic treatment nucleoli decrease in size and become more rounded. Our deep learning models not only identify which cells are experiencing nucleolar stress but also determine the specific stage of stress for each cell. These results have been validated through current known methods to identify nucleolar stress such as immunocytochemistry and RNA production assays. This platform now allows us to investigate how cells recover from reversible nucleolar stress, providing a high-throughput approach to dissect nucleolar function, resilience, and responses to chemotherapeutic treatment. Conclusions: SYTOTM RNASelectTM Red is an effective RNA dye because it provides a bright stain for the nucleolus, enabling clear segmentation and quantitative analysis using machine learning. Its compatibility with immunocytochemistry allows for high-resolution, dynamic studies of RNA biology and cellular stress responses, making it a versatile tool for both single cell and population-level investigations. This dye, coupled with our deep-learning methods, opens the door to broad screening protocols, allowing for a spectrum of drug candidates to be tested rapidly. Citation Format: Leif E. Lindberg, Katelyn R. Alley, Sylvia M. Kennerly, Ian Reynolds, Ryan W. Holly, Jongtae Yang, Chris L. Vonnegutt, John C. Rogers, Victoria J. DeRose. Deep learning-driven image analysis for tracking nucleolar morphology [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2026; Part 1 (Regular Abstracts); 2026 Apr 17-22; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2026;86(7 Suppl):Abstract nr 6407.