Knowledge-Informed Trust in LLMs Among Medical and Dental Students in Germany: A Multicenter Mixed-Methods Study (Preprint)

BACKGROUND Large language models (LLMs) are reshaping how medical and dental students acquire knowledge by providing immediate, personalized explanations of unfamiliar concepts. Yet understanding an explanation does not necessarily mean being able to judge its accuracy or reliability. Students may turn to AI precisely when they lack the knowledge needed to evaluate its responses. Much prior research has assessed AI literacy through self-reported knowledge or perceived competence, which may not reflect objectively demonstrable knowledge. Whether trust in LLM outputs reflects objectively assessed GenAI knowledge remains unclear. OBJECTIVE To investigate whether trust in LLM outputs aligns with objectively assessed GenAI literacy among medical and dental students, and to examine how literacy and trust relate to LLM use and students' evaluations of AI benefits and risks. METHODS We conducted a secondary analysis of a multicenter cross-sectional survey of medical and dental students in Germany. GenAI literacy was measured using the 20-item Generative AI Literacy Assessment Test (GLAT), alongside overall trust in LLM outputs and reported use frequency. Adjusted regression analyses examined associations between literacy and trust. Two independent human coders classified open-ended descriptions of GenAI benefits and risks, with disagreements reviewed by a third independent adjudicator. Exploratory analyses examined the number of themes mentioned and the specificity of students' explanations in relation to literacy and trust. RESULTS Among 802 students with paired GLAT and trust data, 752 were included in the primary adjusted analysis. Greater trust in LLM outputs was associated with lower GenAI literacy (adjusted difference, −0.966 GLAT points per trust level; 95% CI −1.347 to −0.585). This inverse association was directionally consistent across all four GLAT knowledge areas, although differences between areas were not statistically conclusive. More frequent LLM use was associated with greater trust (Spearman ρ=0.417; P<.001), but not clearly with GLAT scores (ρ=−0.051; P=.147). Among open-ended responses (579 benefit and 642 risk responses), higher GLAT scores were associated with mentioning more benefit and risk themes and providing more specific explanations in both domains. These associations involved learning support and interaction, answer quality and task fit, and information and evidence risks. Greater trust was associated with mentioning more benefit themes, but showed no clear association with risk theme counts or explanatory specificity. CONCLUSIONS Greater trust in LLM outputs was associated with more frequent use but lower objectively assessed GenAI literacy, whereas use frequency itself was not clearly associated with literacy. Higher literacy was associated with broader and more specific evaluations of AI benefits and risks. These findings highlight the importance of fostering knowledge-informed trust in medical education. Objectively assessed GenAI knowledge is an important foundation for educational preparedness, supporting students' understanding of what these systems can and cannot reliably do and their ability to evaluate AI-generated information in context.

Authors

Publication Details

Published
2026-11-12
DOI
https://doi.org/10.2196/preprints.116711
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Knowledge-Informed Trust in LLMs Among Medical and Dental Students in Germany: A Multicenter Mixed-Methods Study (Preprint)

Jingyu Zhou, Chao Tu, Shalong Wang, Wenyu Li et al.
Artificial Intelligence in Healthcare and Education
preprint

Knowledge-Informed Trust in LLMs Among Medical and Dental Students in Germany: A Multicenter Mixed-Methods Study (Preprint)

Jingyu Zhou, Chao Tu, Shalong Wang, Wenyu Li, Kun Chen, Zequn Zhang, Qiyuan Wang
preprint en

Abstract

BACKGROUND Large language models (LLMs) are reshaping how medical and dental students acquire knowledge by providing immediate, personalized explanations of unfamiliar concepts. Yet understanding an explanation does not necessarily mean being able to judge its accuracy or reliability. Students may turn to AI precisely when they lack the knowledge needed to evaluate its responses. Much prior research has assessed AI literacy through self-reported knowledge or perceived competence, which may not reflect objectively demonstrable knowledge. Whether trust in LLM outputs reflects objectively assessed GenAI knowledge remains unclear. OBJECTIVE To investigate whether trust in LLM outputs aligns with objectively assessed GenAI literacy among medical and dental students, and to examine how literacy and trust relate to LLM use and students' evaluations of AI benefits and risks. METHODS We conducted a secondary analysis of a multicenter cross-sectional survey of medical and dental students in Germany. GenAI literacy was measured using the 20-item Generative AI Literacy Assessment Test (GLAT), alongside overall trust in LLM outputs and reported use frequency. Adjusted regression analyses examined associations between literacy and trust. Two independent human coders classified open-ended descriptions of GenAI benefits and risks, with disagreements reviewed by a third independent adjudicator. Exploratory analyses examined the number of themes mentioned and the specificity of students' explanations in relation to literacy and trust. RESULTS Among 802 students with paired GLAT and trust data, 752 were included in the primary adjusted analysis. Greater trust in LLM outputs was associated with lower GenAI literacy (adjusted difference, −0.966 GLAT points per trust level; 95% CI −1.347 to −0.585). This inverse association was directionally consistent across all four GLAT knowledge areas, although differences between areas were not statistically conclusive. More frequent LLM use was associated with greater trust (Spearman ρ=0.417; P<.001), but not clearly with GLAT scores (ρ=−0.051; P=.147). Among open-ended responses (579 benefit and 642 risk responses), higher GLAT scores were associated with mentioning more benefit and risk themes and providing more specific explanations in both domains. These associations involved learning support and interaction, answer quality and task fit, and information and evidence risks. Greater trust was associated with mentioning more benefit themes, but showed no clear association with risk theme counts or explanatory specificity. CONCLUSIONS Greater trust in LLM outputs was associated with more frequent use but lower objectively assessed GenAI literacy, whereas use frequency itself was not clearly associated with literacy. Higher literacy was associated with broader and more specific evaluations of AI benefits and risks. These findings highlight the importance of fostering knowledge-informed trust in medical education. Objectively assessed GenAI knowledge is an important foundation for educational preparedness, supporting students' understanding of what these systems can and cannot reliably do and their ability to evaluate AI-generated information in context.

Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon