Qualitative and quantitative evaluation of the usability of transport ventilators using eye tracking

Deficiencies in the usability of transport ventilators may lead to operating errors and patient harm. Intuitively operable devices and optimized ergonomics can reduce cognitive workload and critical events. In order to identify shortcomings of specific devices we compared the user interfaces of four modern transport ventilators. We hypothesized that the task-specific usability of user interfaces differs between transport ventilators. Therefore, we investigated the usability of modern transport ventilators from four manufacturers by means of subjective and objective usability measures, including eye-tracking analysis in 20 typical subsequent tasks. Thirty-four anesthesia residents without extensive experience in the use of transport ventilation were asked to perform a series of tasks on each of four transport ventilators of different manufacturers (Hamilton-T1, Medumat Transport, Dräger Oxylog 3000 plus and Monnal T60). Number of assistances and task failure rates were counted, and task completion times were measured. Further the systems' usability was assessed via questionnaires and rating scales. In addition, eye-tracking was used to analyze gaze direction, time to first fixation and heatmaps superimposed to snapshots. Data are presented as means ± SD and compared by Chi Square tests or two-way ANOVA where appropriate. Task failure rate was highest with the Monnal T60 (10.9%), followed by Hamilton-T1 (8.6%), Oxylog 3000 plus (4.2%) and Medumat Transport (3.0%). Cumulative task completion times differed between the devices. To render all tasks took longest with the Monnal T60 (368 ± 228 s), followed by the Hamilton-T1 (294 ± 136 s), the Oxylog 3000 plus (214 ± 116 s) and the Medumat Transport (204 ± 111 s). System usability scores were comparable between the Oxylog 3000 plus, the Hamilton-T1 and the Medumat Transport, but significantly lower for the Monnal T60. Eye movement analysis showed that, depending on the device and task, some functions were recognized quickly and tasks were solved immediately. Heatmaps showed marked differences in operators´ visual focuses. The systems under test revealed relevant task and design related differences in their usability. The comparison of the four devices highlights the importance of intuitive user guidance to reduce potential sources of error.