Multi-Agent Learning for Adaptive Prompt Optimization in Heterogeneous Clinical Narratives

Abstract Free-text narratives within electronic health records, particularly echocardiography reports, contain vital physiological parameters that are frequently absent from structured flowsheet tables. While modern foundation language models offer strong textual comprehension, manual prompt engineering remains an empirical and non-robust procedure. Small prompt changes can cause formatting errors, missed unit conversions, and numerical hallucinations. To address these issues, we formulate prompt optimization as a discrete-action Markov Decision Process and evaluate it on 120,555 echocardiography notes from 38,022 adult ICU patients in the MIMIC-IV database. Unlike conventional rule-based parsers and naive prompts, the proposed reinforcement learning agents optimize extraction using a numeric reward with a strict \pm 2.5% relative tolerance. Both the off-policy Dueling Double Deep Q-Network (D3QN) and on-policy Deep SARSA agents converged within 15 training episodes, achieving a 7.24% relative improvement in extraction accuracy over conventional NLP methods.