Sequential Federated Fine-Tuning of Large Language Models for Clinical Outcome Prediction in Intensive Care Units

Abstract Large language models (LLMs) show promise for clinical outcome prediction in the intensive care unit (ICU), but collaborative fine-tuning across hospitals is constrained by privacy and data-governance concerns. Federated learning (FL) enables collaborative training without exchanging raw data, but conventional parallel FL requires coordinated participation and centralized aggregation, which can be challenging when institutions differ in training availability. We develop and evaluate a novel framework called sequential federated fine-tuning (SFF) framework that enables efficient and privacy-aware collaborative clinical outcome prediction in ICUs across multiple hospitals. By transferring low-rank adapters sequentially from one hospital to the next, SFF supports asynchronous learning, allowing institutions to remain offline until their turn while another institution performs local training. Using 45,149 ICU stays from 27 hospitals in the eICU Collaborative Research Database (eICU-CRD), we applied low-rank adaptation (LoRA) to Qwen-2.5-7B-Instruct and LLaMA-3.1-8B-Instruct and compared SFF with centralized fine-tuning, local fine-tuning, zero-shot prompting, and federated averaging (FedAvg) for in-hospital mortality and three-class length-of-stay (LoS) prediction. SFF achieved performance close to centralized fine-tuning while outperforming local training and FedAvg. For mortality prediction, SFF reached AUROCs of 86.08% and 89.24% for Qwen-2.5-7B and LLaMA-3.1-8B, respectively; for LoS classification, it reached 75.32% and 76.57%. The advantage over FedAvg was observed across most hospital cohorts and remained evident when local training data were limited. Our work advances the practical, privacy-aware deployment of LLMs in multi-institutional clinical settings for collaborative clinical outcome prediction in ICUs.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-17
DOI
https://doi.org/10.21203/rs.3.rs-11602066/v1
Primary Topic
Machine Learning in Healthcare
Type
preprint

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Sequential Federated Fine-Tuning of Large Language Models for Clinical Outcome Prediction in Intensive Care Units

Tongnian Wang, Yuanxiong Guo, Yifan Gao, Yanmin Gong
4 links
Research Square
Machine Learning in Healthcare
preprint

Sequential Federated Fine-Tuning of Large Language Models for Clinical Outcome Prediction in Intensive Care Units

Tongnian Wang, Yuanxiong Guo, Yifan Gao, Yanmin Gong
preprint en

Abstract

Abstract Large language models (LLMs) show promise for clinical outcome prediction in the intensive care unit (ICU), but collaborative fine-tuning across hospitals is constrained by privacy and data-governance concerns. Federated learning (FL) enables collaborative training without exchanging raw data, but conventional parallel FL requires coordinated participation and centralized aggregation, which can be challenging when institutions differ in training availability. We develop and evaluate a novel framework called sequential federated fine-tuning (SFF) framework that enables efficient and privacy-aware collaborative clinical outcome prediction in ICUs across multiple hospitals. By transferring low-rank adapters sequentially from one hospital to the next, SFF supports asynchronous learning, allowing institutions to remain offline until their turn while another institution performs local training. Using 45,149 ICU stays from 27 hospitals in the eICU Collaborative Research Database (eICU-CRD), we applied low-rank adaptation (LoRA) to Qwen-2.5-7B-Instruct and LLaMA-3.1-8B-Instruct and compared SFF with centralized fine-tuning, local fine-tuning, zero-shot prompting, and federated averaging (FedAvg) for in-hospital mortality and three-class length-of-stay (LoS) prediction. SFF achieved performance close to centralized fine-tuning while outperforming local training and FedAvg. For mortality prediction, SFF reached AUROCs of 86.08% and 89.24% for Qwen-2.5-7B and LLaMA-3.1-8B, respectively; for LoS classification, it reached 75.32% and 76.57%. The advantage over FedAvg was observed across most hospital cohorts and remained evident when local training data were limited. Our work advances the practical, privacy-aware deployment of LLMs in multi-institutional clinical settings for collaborative clinical outcome prediction in ICUs.

Research Square
University of Tennessee at Chattanooga (US), The University of Texas at San Antonio (US), Texas A&M University (US)
National Science Foundation
Good health and well-being
Machine Learning in Healthcare
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon