Deriving Usability Evaluation Criteria for Threat Modeling Tools

As the domestic and international landscape rapidly changes, the importance of implementing security measures in response to the growing threats that businesses face has increased. In this context, the need for Security by Design (SbD), integrating security into the earlier phases of software development lifecycle, is becoming more obvious, with threat modeling recognized as a fundamental element of SbD. In particular, as part of the Shift Left strategy—which focuses on saving costs and time by detecting and resolving security threats early—personnel with limited security expertise, such as software developers, are required to engage in threat modeling. Although various automated threat modeling tools have been released, their lack of usability for users with limited security expertise poses challenges in effectively conducting threat modeling. To address this, we analyzed the usability of threat modeling tools based on criteria derived from the GQM (Goal-Question-Metric) approach. An expert survey was conducted to derive the importance of each criterion and utilize it as a weighting factor. We performed usability evaluations of five threat modeling tools (MS TMT, OWASP TD, PyTM, IriusRisk, SPARTA), and concluded that IriusRisk is the most usable one among them. This study proposes criteria of usability for software to assist personnel with limited security expertise in effectively performing threat modeling, thereby fostering a supportive environment.