AI through the looking glass: a controlled feasibility study of wearable AI assistance on plastic surgery in-service examination questions

Purpose Large language models (LLMs) have demonstrated substantial accuracy on specialty in-service examinations. Commercially available smart glasses now make real-time AI assistance wearable, raising concerns about testing integrity. We developed and evaluated a smart glasses application that captures an examination question, submits it to an AI model, and delivers the answer directly to the wearer, to assess the feasibility and limitations of this workflow in a controlled simulation. Methods In Phase 1, three LLMs (Google Gemini 2.5 Flash, OpenAI GPT-4o, and Anthropic Claude Sonnet 4.6) and a majority-vote ensemble were benchmarked on 743 Plastic Surgery In-Service Training Examination (PSITE) question texts from the 2021–2023 examination cycles. Clinical images were not supplied. In Phase 2, the model with the highest observed accuracy was deployed through Rokid AI Glasses, a companion phone, and a cloud service. One study author tested the system on a 50-question subset selected from Phase 1. Outcomes were accuracy, first-attempt success, eventual completion, and successful-attempt latency. Results Gemini 2.5 Flash achieved the highest individual accuracy (82.8%), followed by Claude Sonnet 4.6 (80.9%) and GPT-4o (76.9%). The majority-vote ensemble achieved 81.6% accuracy and did not outperform Gemini. In Phase 2, the wearable system answered 76.0% of questions correctly (38/50; 95% CI, 62.6%–85.7%). Answers were returned on the first attempt for 45/50 questions (90.0%); all 50 were completed after 56 attempts. Six attempts failed because of application timeouts requiring restart. Mean successful-attempt latency was 8.9 s (SD 4.2), excluding failed attempts and restart time. Conclusions Commercially available smart glasses and AI services can be connected into a workflow that answers plastic surgery examination questions in real time, although errors and retries limit reliability. This controlled simulation illustrates a potential route to unauthorized assistance. The findings aim to encourage broader discussion of emerging risks to testing integrity and the future of education and assessment.

Authors

Publication Details

Journal
Global Surgical Education - Journal of the Association for Surgical Education
Published
2026-11-19
DOI
https://doi.org/10.1007/s44186-026-00637-w
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

AI through the looking glass: a controlled feasibility study of wearable AI assistance on plastic surgery in-service examination questions

Mustafa Tamim Alam Khan, Joao Bombardelli, Mohin A. Bhadkamkar, Erik Marques et al.
Global Surgical Education - Journal of the Association for Surgical Education
Artificial Intelligence in Healthcare and Education
article

AI through the looking glass: a controlled feasibility study of wearable AI assistance on plastic surgery in-service examination questions

Mustafa Tamim Alam Khan, Joao Bombardelli, Mohin A. Bhadkamkar, Erik Marques, Mahwish Khan
article en

Abstract

Purpose Large language models (LLMs) have demonstrated substantial accuracy on specialty in-service examinations. Commercially available smart glasses now make real-time AI assistance wearable, raising concerns about testing integrity. We developed and evaluated a smart glasses application that captures an examination question, submits it to an AI model, and delivers the answer directly to the wearer, to assess the feasibility and limitations of this workflow in a controlled simulation. Methods In Phase 1, three LLMs (Google Gemini 2.5 Flash, OpenAI GPT-4o, and Anthropic Claude Sonnet 4.6) and a majority-vote ensemble were benchmarked on 743 Plastic Surgery In-Service Training Examination (PSITE) question texts from the 2021–2023 examination cycles. Clinical images were not supplied. In Phase 2, the model with the highest observed accuracy was deployed through Rokid AI Glasses, a companion phone, and a cloud service. One study author tested the system on a 50-question subset selected from Phase 1. Outcomes were accuracy, first-attempt success, eventual completion, and successful-attempt latency. Results Gemini 2.5 Flash achieved the highest individual accuracy (82.8%), followed by Claude Sonnet 4.6 (80.9%) and GPT-4o (76.9%). The majority-vote ensemble achieved 81.6% accuracy and did not outperform Gemini. In Phase 2, the wearable system answered 76.0% of questions correctly (38/50; 95% CI, 62.6%–85.7%). Answers were returned on the first attempt for 45/50 questions (90.0%); all 50 were completed after 56 attempts. Six attempts failed because of application timeouts requiring restart. Mean successful-attempt latency was 8.9 s (SD 4.2), excluding failed attempts and restart time. Conclusions Commercially available smart glasses and AI services can be connected into a workflow that answers plastic surgery examination questions in real time, although errors and retries limit reliability. This controlled simulation illustrates a potential route to unauthorized assistance. The findings aim to encourage broader discussion of emerging risks to testing integrity and the future of education and assessment.

Global Surgical Education - Journal of the Association for Surgical EducationVol. 5(1)
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon