The impact of prompting on ChatGPT’s adherence to status epilepticus treatment guidelines

This study assessed ChatGPT's adherence to established management guidelines for status epilepticus (SE) from major neurological societies (NCS, AES, EFNS) and examined how prompt specificity affected the quality of its recommendations. Four prompts varying in detail were each submitted four times, and the generated recommendations were analyzed for consistency with guidelines, along with an assessment of source relevance and accuracy. ChatGPT consistently recommended securing the airway and a breathing check (100% of responses) and always suggested benzodiazepines as first-line treatment. However, it rarely recommended key measures such as side positioning (25%) to prevent potential aspiration and neurological assessments (0-25%). Likewise, alternative administration routes for benzodiazepines were mentioned inconsistently (0-100%). While second- and third-line antiseizure medications were suggested consistently, proper dosage guidance was lacking when unprompted. EEG monitoring was recommended in 50-100%. More specific and detailed prompts increased guideline adherence most markedly regarding vital sign assessment (from 33 to 91%), correct dosages of second-line (from 50 to 100%), third-line drugs (from 0 to 100%), and screening for complications (from 0 to 100%). The findings underscore that Chat-GPT shows promise as a clinical support tool but requires structured prompts for accuracy and should not replace clinical judgment or professional oversight.