The effects of oral versus written knowledge task prompt delivery in medical device usability testing

Knowledge tasks (KTs) are commonly used in medical device human factors (HF) testing to assess participant understanding that cannot be evaluated through direct observation. However, it is unclear whether presenting KTs orally or in writing influences participant behavior or task performance. This study compared oral and written KT prompt presentation in 38 lay adults who each completed 10 KTs for two consumer medical devices. Participants responded verbally in both groups and had access to the device instructions for use (IFU). The Written KT delivery resulted in statistically higher accuracy than the Oral delivery for the blood pressure monitor (65% vs. 47%) and nebulizer (85% vs. 72%), as well as greater frequency of IFU consultation (64% vs. 32% and 65% vs. 36%, respectively). The Written KT Group also had longer pre-response and total task times and fewer moderator interventions, whereas response duration did not differ between groups. To our knowledge, this is the first study to directly examine KT prompt delivery in medical device HF testing. The findings indicate that oral and written KT prompts may lead to different results and participant behavior. Accordingly, whether KT prompts are presented orally or in writing should be treated as a deliberate study design choice, standardized within studies, and clearly reported.