Rethinking the usability and safety assessment of medical devices – an assessment paradigm involving triangulation of deliberate misuse testing, heuristic evaluation, and simulated-use testing and practical validation through a smart infusion pump ca

We propose a novel assessment triangulation paradigm in which deliberate misuse testing, heuristic evaluation, and simulated-use testing are triangulated to complement each other’s methodological strengths in detecting usability and safety deficiencies in medical devices. Step-by-step details for implementing the triangulation paradigm were practically validated through a case study of nurses using smart infusion pumps and are presented. Smart infusion pumps were used as contextual clinical testbeds for practical validation. First, deliberate misuse testing was conducted by intentionally introducing abnormal operations and errors designed to replicate misuse and error conditions that pose potential safety risks to examine how the device responds to these disruptions. Heuristic evaluation was then conducted by evaluators with domain knowledge who selected and utilized general and device-specific usability and safety heuristics to identify deficiencies. Simulated-use testing was conducted by the nurse representatives of end users who performed tasks relevant to the clinical use of medical devices to assess user–device–task interactions. Detailed steps for implementing the triangulation paradigm and its complementary functions were validated. Deliberate misuse testing was devised to allow the discovery of device operation problems that might not be found through standard testing, where only expected and correct inputs and operations are used. Heuristic evaluation allowed the detection of device interface design deficiencies. Simulated-use testing provided a realistic setting for objectively evaluating user–device–task interaction performance. We recommend the adoption of the assessment triangulation paradigm in usability and safety evaluation because triangulating multiple assessment methods enables the generation of complementary information on the different dimensions of the device. The complementary information essentially provides a more comprehensive and deeper understanding of the usability and safety issues of the products/devices under evaluation. What is already known on the topic • Identifying usability and safety deficiencies in medical devices before use is critical, yet single-method evaluations often fail to detect a broad range of design and use-related issues, risking patient safety. What this study adds to our knowledge • A novel assessment paradigm for the assessment of medical devices that involves the triangulation of three usability and safety assessment methods is proposed, and step-by-step instructions for applying this paradigm are offered. • The three triangulated assessment methods, deliberate misuse testing, heuristic evaluation, and simulated-use testing, complement each other’s strengths and thus generate complementary information on the different dimensions of the devices being assessed. How this study might affect research, practice or policy • As single-method assessments present significant gaps, our assessment triangulation paradigm, which encourages a more comprehensive assessment process, may reshape how relevant entities in the medical device safety and patient safety fields (e.g., such as the FDA, ISO, and leading practitioners) approach their evaluations.