The Use of ChatGPT to Write Assessment Questions

It has been suggested that ChatGPT can be used to write assessment questions. This study aimed to evaluate student pharmacists’ academic performance on ChatGPT-generated questions in summative assessments, explore their perceptions regarding the use of ChatGPT to write assessment questions, and characterize the questions’ Bloom’s taxonomy levels. ChatGPT was used to generate assessment questions aligned with the lecture objectives. After selection by a content expert, ChatGPT-generated questions were used for all examinations in the Pharmaceutics I and Pharmaceutics II courses administered in 2024 and 2025. A survey administered to student pharmacists investigated their perceptions of using artificial intelligence (AI) to generate assessment questions. Ninety-one students participated in the study (response rate 93%). Students were graded on approximately 100 ChatGPT-generated questions for the combined courses. In both 2024 and 2025, in the Pharmaceutics I course, the average percent correct was higher on the ChatGPT-generated questions compared to instructor-generated questions, whereas in the Pharmaceutics II course, the percent correct was lower on ChatGPT-generated questions. Most students thought that instructors should not use ChatGPT to generate most (≥50%) or all their assessment questions. The average success rate of students in distinguishing between 20 assessment questions that were evenly split between ChatGPT- and instructor-generated questions was 53%. CompetencyGenie™, an AI-powered tool, assigned mostly Bloom’s taxonomy levels of Remember and Understand to the ChatGPT-generated questions. Student performance was similar across ChatGPT-generated questions from the two levels. ChatGPT can generate questions that are usable in summative assessments in pharmaceutics courses.