System Usability Scale: A Meta-Analysis of How SUS Relates to Workload, Task Time, and Error Rate

The system usability scale (SUS) is a widely used instrument for measuring perceived usability. This study meta-analyzes 105 studies that reported SUS, workload, task time, and error rate for pairs of systems. Workload was restricted to self-report metrics. The meta-analysis resulted in three main findings. First, SUS and workload were strongly correlated. Second, SUS and performance, as measured by task time and error rate, were related but somewhat independent. Third, task time and error rate correlated with SUS to similar extents. In only 10% of the studies, the system that received the higher SUS score imposed higher workload. In contrast, the system that received the higher SUS score resulted in poorer performance in 24% (task time) and 23% (error rate) of the studies. To conclude, SUS complements performance measures and it largely does so by also being sensitive to perceived workload.