Whose appraisal? A Freire-informed, think-aloud study of allied health students’ use of Generative AI in evidence appraisal

Background Generative AI (GenAI) can now produce artefacts such as essays, literature reviews, and worked rationales which challenge assessment designs that reward the products of thinking over the practice of it (Bearman et al., 2024; Corbin et al., 2025; Dawson et al., 2024). In evidence-based practice (EBP) education, students are required to appraise study quality, interpret effect sizes, and integrate findings into clinical decisions (Lehane et al., 2019). GenAI can now generate convincing outputs for each of these tasks. Observational data on what students do cognitively when they use GenAI in EBP tasks are scarce. Depending on how students use it, GenAI may scaffold the reasoning that EBP education is designed to develop or may bypass it (Vendrell & Johnston, 2026). Aim This study explores how allied health students engage with GenAI, how much they rely on it, and how they reason about its use when completing an EBP evidence-appraisal task. Methods This is an exploratory qualitative study using a concurrent think-aloud protocol with allied health students who have completed an EBP unit. The unit must have included critical reasoning as a learning outcome. Participants complete a structured 60-minute EBP task involving a clinical scenario. During this task, participants verbalise their reasoning and rationale for using or declining GenAI. A researcher facilitates this process through non-directive questions. Sessions are audio-recorded, transcribed verbatim, and analysed in two stages. In the first stage, transcripts are coded deductively and inductively against the six critical thinking processes in Vendrell and Johnston’s (2026) framework: conceptual interpretation, inferential reasoning, evaluative judgement, metacognitive regulation, intellectual curiosity, and epistemic integrity. This stage identifies whether and how reasoning is handed over to GenAI. In the second stage, an interpretive analysis informed by critical pedagogy (Freire, 2005) examines how students position themselves in relation to AI-generated knowledge. Freire described a banking model of education, in which an authority deposits knowledge into passive students. This model offers a lens for examining uncritical acceptance of GenAI output, should that be present (Freire, 2005). Freire located authority in the school system and in the teacher who deposits knowledge into students (Freire, 2005). GenAI is a further candidate for that position in higher education. Unlike a teacher or an institution, it is designed by companies whose incentive is engagement and commercial return rather than student learning (Bartsch et al., 2025). Findings Data collection is in progress. This study will generate think-aloud data on how allied health students engage cognitively with GenAI during EBP appraisal tasks, mapping where GenAI use aligns with or diverges from the reasoning processes EBP education is designed to develop. The interpretive analysis will explore the degrees of agency students exercise over AI-generated knowledge and the conditions in which they exercise or relinquish it, which process-level coding alone cannot capture. Conclusion Findings may inform teaching and assessment design in EBP units and provide a think-aloud protocol that can be adapted to examine GenAI use in reasoning tasks across health professions education.

Authors

Institutions

Publication Details

Journal
ASCILITE Publications
Published
2026-11-17
DOI
https://doi.org/10.65106/apubs.2026.3235
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Whose appraisal? A Freire-informed, think-aloud study of allied health students’ use of Generative AI in evidence appraisal

Merrolee Penman, Leanne Chalmers, Daniel Cury Ribeiro, Ryan Lopez
4 links
ASCILITE Publications
Artificial Intelligence in Healthcare and Education
article

Whose appraisal? A Freire-informed, think-aloud study of allied health students’ use of Generative AI in evidence appraisal

Merrolee Penman, Leanne Chalmers, Daniel Cury Ribeiro, Ryan Lopez
article en

Abstract

Background Generative AI (GenAI) can now produce artefacts such as essays, literature reviews, and worked rationales which challenge assessment designs that reward the products of thinking over the practice of it (Bearman et al., 2024; Corbin et al., 2025; Dawson et al., 2024). In evidence-based practice (EBP) education, students are required to appraise study quality, interpret effect sizes, and integrate findings into clinical decisions (Lehane et al., 2019). GenAI can now generate convincing outputs for each of these tasks. Observational data on what students do cognitively when they use GenAI in EBP tasks are scarce. Depending on how students use it, GenAI may scaffold the reasoning that EBP education is designed to develop or may bypass it (Vendrell & Johnston, 2026). Aim This study explores how allied health students engage with GenAI, how much they rely on it, and how they reason about its use when completing an EBP evidence-appraisal task. Methods This is an exploratory qualitative study using a concurrent think-aloud protocol with allied health students who have completed an EBP unit. The unit must have included critical reasoning as a learning outcome. Participants complete a structured 60-minute EBP task involving a clinical scenario. During this task, participants verbalise their reasoning and rationale for using or declining GenAI. A researcher facilitates this process through non-directive questions. Sessions are audio-recorded, transcribed verbatim, and analysed in two stages. In the first stage, transcripts are coded deductively and inductively against the six critical thinking processes in Vendrell and Johnston’s (2026) framework: conceptual interpretation, inferential reasoning, evaluative judgement, metacognitive regulation, intellectual curiosity, and epistemic integrity. This stage identifies whether and how reasoning is handed over to GenAI. In the second stage, an interpretive analysis informed by critical pedagogy (Freire, 2005) examines how students position themselves in relation to AI-generated knowledge. Freire described a banking model of education, in which an authority deposits knowledge into passive students. This model offers a lens for examining uncritical acceptance of GenAI output, should that be present (Freire, 2005). Freire located authority in the school system and in the teacher who deposits knowledge into students (Freire, 2005). GenAI is a further candidate for that position in higher education. Unlike a teacher or an institution, it is designed by companies whose incentive is engagement and commercial return rather than student learning (Bartsch et al., 2025). Findings Data collection is in progress. This study will generate think-aloud data on how allied health students engage cognitively with GenAI during EBP appraisal tasks, mapping where GenAI use aligns with or diverges from the reasoning processes EBP education is designed to develop. The interpretive analysis will explore the degrees of agency students exercise over AI-generated knowledge and the conditions in which they exercise or relinquish it, which process-level coding alone cannot capture. Conclusion Findings may inform teaching and assessment design in EBP units and provide a think-aloud protocol that can be adapted to examine GenAI use in reasoning tasks across health professions education.

ASCILITE Publications
Curtin University (AU)
Quality education, Good health and well-being
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon