Usability Testing and the System Usability Scale Effectiveness Assessment on Different Sensing Devices of Prototype and Live Web System Counterpart

During the process of digital-system development from prototype to live implementation, differences in user interactions, perceived usability, and overall satisfaction can emerge. These differences often arise due to various factors, which may include the fidelity of the software prototype, the limitations of the prototyping tool, and the complexity of the live digital system. Recognizing these potential usability discrepancies between prototypes and live digital systems, assessment of how well user experience (UX) test approaches, such as usability testing and the System Usability Scale (SUS), reflect the UX in using the digital-system prototype and its counterpart deployed live system emerged as an important research gap. To address this gap, this study compares usability testing and SUS results among a Figma web prototype and its counterpart live web digital system, for the telecom service extension process as a representative digital-system case study. The research study involved a testing process with a total of 10 participants across the Figma prototype and live-web-system test environments, in which different sensing devices that included versatile types of mobile phones were utilized. The research study presents usability testing results related to the overlap in perceived usability issues for the same digital-product developments in both testing environments, which are experienced on different types of mobile sensing devices. The usability testing results are presented as reports on the frequency of occurrence of web system usability issues and corresponding severity levels. The obtained results demonstrated that prototype testing is highly effective for detecting a wide range of usability issues early in the digital-product development phase. The paper also evaluates the predictive capabilities of SUS assessment for the case of the Figma web prototype and its counterpart live web system in the phase of digital-product development. The results show that the SUS evaluation, when applied to digital-system prototype testing, can provide early in the development process a reliable indication of the perceived usability of its counterpart digital system, once it is developed and deployed. The findings presented in the paper offer valuable guidance for software designers and developers seeking to make prototypes and their counterpart real digital-product deployments with improved digital-product overall user experience.