Detecting AI use in lower-division A&P course work

Learning anatomy and physiology (A&P) at the undergraduate level requires remembering facts and understanding basic bodily mechanisms. The easily available, user-friendly, and time-saving artificial intelligence (AI) tools threaten the teaching and learning of fundamental A&P concepts and instructors may unknowingly conflate human- and AI-written work. Here, we examined the veracity of online AI checkers to detect student- vs. AI-generated work for fact-heavy science writing, compared detection rates with human coders, and surveyed participants about AI work. We hypothesized that detection of AI-generated work would be better by human coders than with AI checkers and that participants would rate the quality of AI work as better than their own. Participants (n=268) wrote a 150±15-word essay to a prompt on plasma membrane structure and function and then instructed a generative AI program of their choosing to write an essay to the same prompt. After reading each essay, participants completed a survey on the quality of the essays and their general use of AI. Randomly chosen essays were assessed by 4 online AI checkers (n=50 essays / group) and by human coders (n=20 essays). Average false positive (human as AI) and false negative (AI as human) rates for three of four AI checkers was 2.7±1.8 and 2.0±1.2%, respectively; the fourth AI checker rates were 16.3 and 24%, respectively. Generally, the accuracy of the AI checkers was variable if assessing hybrid essays that contained combined sentences from AI- and human-generated essays, which may be of concern if students are using AI for grammar or punctuation assistance. False positive and false negative rates for human coders were 4.9±2.7 and 16.2±6.1%, respectively, with substantial agreement among coders (κ=0.62), indicating that AI checkers are better at discriminating AI- and human-generated work than humans (p<0.05). A majority of participants completing the survey (n=190) believed that AI generated the better answer (63.2%), was better written (75.1%), answered the prompt with greater depth (68.1%), and had fewer grammatical errors (57.4%) (p<0.05). Although 43.7% of participants thought AI had fewer content errors, 43.2% believed that their essay and the AI essay were equal for content errors (p>0.05). Preliminary qualitative content analysis indicated that the verbiage used in human and AI essay was not substantively different, suggesting that undergraduate students can express similar content as AI, but lack confidence in that expression. Overall, ~77% of undergraduate participants believed that they would be able to identify a human- and an AI-generated essay (p<0.05). Taken together, if instructors suspect students of using generative AI for general and fact-based A&P content, then the essays should be cross-checked with multiple online AI checkers to eliminate the chance of false positives. This abstract was presented at the American Physiology Summit 2025 and is only available in HTML format. There is no downloadable file or PDF version. The Physiology editorial board was not involved in the peer review process.