The effectiveness of ChatGPT in pediatric simulation-based tests of nursing courses in Taiwan: A descriptive study

• GPT-4.0 demonstrated a greater comprehension of complex clinical scenarios than GPT-3.5. • Both GPT models could misinterpret detailed medical data and fail to respond accurately in pediatric simulated-based situations. • In complicated clinical situations, ChatGPT should be done cautiously, especially when requiring clinical reasoning and judgment. ChatGPT is a generative language model that enhances personalized learning, encourages critical thinking, and supports problem-based learning. However, its use in nursing education requires further validation. This study aimed to assess the performance of ChatGPT models on pediatric simulation-based assessment tests, and compare the test scores with nursing students. A descriptive study was conducted. The ChatGPT-3.5 and 4.0 were used to complete pediatric simulation-based assessment tests, analyze the content of the responses, and compare scores to those of nursing students. The test included four pediatric simulated scenarios and consisted of 40 items. A passing grade was an average score of 60 or above. A total of 267 fourth-year Associate Degree in Nursing (ADN) program students were recruited for this study. The scores of ChatGPTs were significantly different ( p < .05). GPT-4.0 outperformed GPR-3.5, and more effectively conveyed and interpreted images. There are four domains to explain the performance of GPTs: Good Assist, Answer inconsistencies between ChatGPTs, Fabricated information, and Inability to provide comprehensive assessments. Compared to students, the scores of all sections of nursing students were higher than GPTs, except the second round GPT-4.0. ChatGPT is a useful tool in nursing education. However, ChatGPT could not pass all the pediatric simulation-based assessment tests. It is cautious when utilizing ChatGPT to detect real clinical situations.