AI-Assisted Generation of Long-Term Follow-Up Recommendations for Survivors of Childhood Cancer and Hematopoietic Stem Cell Transplantation

BACKGROUND: Survivors of childhood cancer and hematopoietic stem cell transplantation (HSCT) require individualized long-term follow-up (LTFU) care to monitor for treatment-related late effects. Development of survivorship care plans is labor-intensive and requires integration of complex guideline-based recommendations. We evaluated the feasibility of using a large language model (LLM) to generate draft LTFU care recommendations for childhood cancer and HSCT survivors. METHODS: Survivors seen in the UCSF Benioff Children's Hospitals Survivorship Program between January 2021 and December 2023 were eligible. Clinicians created individualized LTFU plans using institutional standards and Children's Oncology Group Long-Term Follow-Up Guidelines (version 5). Deidentified treatment summaries were provided to OpenAI GPT-4o to generate AI-assisted draft recommendations using structured prompts. Prompt refinement was performed iteratively in 2 development cohorts of 20 patients each. Final evaluation used an independent validation cohort of 40 survivors. AI-generated recommendations were compared with clinician-generated recommendations using clinician-generated plans as the operational reference standard. Agreement between AI-generated and clinician-generated recommendations was assessed by determining the proportion of clinician-generated recommendations identified by AI and the proportion of AI-generated recommendations also present in clinician-generated plans. RESULTS: Among 40 survivors, AI-generated recommendations contained 467 items compared with 446 in clinician-generated recommendations. Among 446 clinician-generated recommendations, 385 (86.3%) were also identified in the AI-generated recommendations. Conversely, 385 (82.4%) of the 467 AI-generated recommendations were also present in clinician-generated plans. Most discordant recommendations involved radiation-related exposures and survivorship scenarios requiring nuanced clinical interpretation. CONCLUSIONS: AI-assisted generation of survivorship care recommendations demonstrated substantial concordance with clinician-generated recommendations and may support clinician-supervised survivorship care workflows.