Aligning language models with clinical expertise: Direct preference optimization for heart failure nursing documentation in critical care

Nursing documentation in intensive care units (ICUs) provides essential clinical intelligence but often suffers from inconsistent terminology, informal styles, and lack of standardization, challenges that are particularly critical in heart failure care. This study applies Direct Preference Optimization (DPO) to adapt Mistral-7B, a locally deployable language model, using 8838 heart failure nursing notes from the MIMIC-III database and 21,210 preference pairs derived from expert-verified GPT outputs, model generations, and original notes. Evaluation across BLEU, ROUGE, BERTScore, Perplexity, and expert qualitative assessments demonstrates that DPO markedly enhances documentation quality. Specifically, BLEU increased by 84% (0.173 → 0.318), BERTScore improved by 7.6% (0.828 → 0.891), and expert ratings rose across accuracy (+14.4 points), completeness (+14.5 points), logical consistency (+14.1 points), readability (+11.1 points), and structural clarity (+6.0 points). These results indicate that DPO can align lightweight clinical language models with expert standards, supporting privacy-preserving, AI-assisted documentation within electronic health record systems to reduce administrative burden and improve ICU patient safety. • Present the first application of direct preference optimization to align a locally deployable 7B language model with expert-verified ICU heart failure nursing documentation standards. • Construct 21,210 structured preference pairs from expert-checked GPT outputs, original nursing notes, and model generations using 8838 MIMIC-III samples. • Demonstrate measurable improvements in BLEU, ROUGE, BERTScore, and perplexity compared with the untuned Mistral-7B baseline. • Show through blinded clinical assessment that DPO reduces omissions, contradictions, and structural inconsistencies while improving accuracy and completeness. • Provide a novel error taxonomy and qualitative analysis showing how DPO reshapes narrative structure and clinical clarity in nurse documentation.