AutoPathML: Automated Machine Learning for Histology Images via Large Language Model and Multi‐Agent

ABSTRACT Automated machine learning (AutoML) for histology image analysis faces challenges because of the complexity of pathological data, including multi‐scale tissue architecture, staining heterogeneity across institutions. Existing AutoML frameworks struggle to address these domain‐specific challenges, often requiring extensive manual configuration and domain expertise. To address these limitations, we introduce AutoPathML, a novel multi‐agent framework that leverages large language models (LLMs) to automate the construction and optimisation of ML pipelines for histology image segmentation. AutoPathML comprises five agents working in coordination, namely a data characterisation agent that performs multi‐resolution feature extraction across magnification levels and analyses staining variations using colour deconvolution; a preprocessing agent that employs LLM‐guided strategy selection for stain normalisation and artefact restoration; a segmentation specialist agent that uses SegAnyPath as the backbone model with LLM‐based task classification; an optimisation strategy agent that develops dataset‐specific training protocols through LLM‐coordinated hyperparameter optimisation and a pipeline assembly agent that synthesises all components into executable implementations. We evaluated AutoPathML on 10 diverse histology datasets spanning nuclear, cell, tissue and tumour segmentation tasks. Experimental results demonstrate performance improvements over state‐of‐the‐art methods, with AutoPathML achieving 82.47% dice coefficient on nuclear segmentation compared to 78.02% for state‐of‐the‐art.