ARIPA: Agentic Reasoning for Image-based Pathology Analysis

Nuclear segmentation across patient-derived organoids (PDOs) is complicated by differences in morphology, staining, image scale, and acquisition format. We present ARIPA, an automated human-in-the-loop (HITL) system that coordinates pre-trained segmentation tools through a vision-language model (VLM)-based agentic planning, verification, and refinement loop for hematoxylin and eosin images of formalin-fixed, paraffin-embedded PDOs. In contrast to existing tools, ARIPA does not require training and achieves good performance. Tissue-aware tiling and image-derived calibration initialize segmentation; optional expert corrections guide configuration search and light-weight object-level rules without retraining or fine-tuning the pre-trained segmentation networks. We evaluated initial ARIPA predictions on more than 50,000 nuclei from 19 images of breast, head and neck, and lung patient-derived cancer PDOs, where nuclear segmentation was challenging due to its 3D structure and variation in imaging layers and malignant cell morphology. Performance was evaluated on agent-selected models before expert feedback, after feedback from the initial runs excluding direct expert edits, and in comparison to a custom-trained HoVer-Net model previously published by our group. At a minimum polygon intersection over union (IoU) of 0.5, image-macro F1 was 0.8380 before feedback, 0.9849 after feedback, and 0.4713 for HoVer-Net. Mean area under the F1 score versus IoU curve was 0.8013, 0.9700, and 0.4317, respectively. The mean paired F1 increase after feedback was 0.1468, with increases in 17 out of the total 19 images and no changes in the remaining two. Ground truth nuclei annotations were created by expert review and editing of ARIPA's post-feedback outputs. In this cohort, ARIPA outperformed the custom-trained HoVer-Net, and its performance was improved further following expert-guided HITL refinement. By automatic agentic reasoning with VLM review without training, ARIPA has potential to be rapidly deployed across organs and cancers with HITL design, accelerating the fundamental step towards downstream biomedical applications.

Authors

Institutions

Publication Details

Journal
bioRxiv (Cold Spring Harbor Laboratory)
Published
2026-11-18
DOI
https://doi.org/10.64898/2026.11.16.767159
Primary Topic
AI in cancer detection
Type
preprint

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

ARIPA: Agentic Reasoning for Image-based Pathology Analysis

Ioannis K. Zervantonakis, Aatur Dilip Singhi, Riyue Bao, Qiangqiang Gu et al.
2 links
bioRxiv (Cold Spring Harbor Laboratory)
AI in cancer detection
preprint

ARIPA: Agentic Reasoning for Image-based Pathology Analysis

Ioannis K. Zervantonakis, Aatur Dilip Singhi, Riyue Bao, Qiangqiang Gu, Brian Robert Isett, Rose Doerfler, Sahussapont Joseph Sirintrapun, Rong Xia, Arjun Singh, Jie Chen
preprint en

Abstract

Nuclear segmentation across patient-derived organoids (PDOs) is complicated by differences in morphology, staining, image scale, and acquisition format. We present ARIPA, an automated human-in-the-loop (HITL) system that coordinates pre-trained segmentation tools through a vision-language model (VLM)-based agentic planning, verification, and refinement loop for hematoxylin and eosin images of formalin-fixed, paraffin-embedded PDOs. In contrast to existing tools, ARIPA does not require training and achieves good performance. Tissue-aware tiling and image-derived calibration initialize segmentation; optional expert corrections guide configuration search and light-weight object-level rules without retraining or fine-tuning the pre-trained segmentation networks. We evaluated initial ARIPA predictions on more than 50,000 nuclei from 19 images of breast, head and neck, and lung patient-derived cancer PDOs, where nuclear segmentation was challenging due to its 3D structure and variation in imaging layers and malignant cell morphology. Performance was evaluated on agent-selected models before expert feedback, after feedback from the initial runs excluding direct expert edits, and in comparison to a custom-trained HoVer-Net model previously published by our group. At a minimum polygon intersection over union (IoU) of 0.5, image-macro F1 was 0.8380 before feedback, 0.9849 after feedback, and 0.4713 for HoVer-Net. Mean area under the F1 score versus IoU curve was 0.8013, 0.9700, and 0.4317, respectively. The mean paired F1 increase after feedback was 0.1468, with increases in 17 out of the total 19 images and no changes in the remaining two. Ground truth nuclei annotations were created by expert review and editing of ARIPA's post-feedback outputs. In this cohort, ARIPA outperformed the custom-trained HoVer-Net, and its performance was improved further following expert-guided HITL refinement. By automatic agentic reasoning with VLM review without training, ARIPA has potential to be rapidly deployed across organs and cancers with HITL design, accelerating the fundamental step towards downstream biomedical applications.

bioRxiv (Cold Spring Harbor Laboratory)
University of Pittsburgh (US), Massachusetts General Hospital (US), University of Pittsburgh Medical Center (US), New York University (US)
University of Pittsburgh
Good health and well-being
AI in cancer detection
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon