Validating a Set of Candidate Criteria for Evaluating Software Tools and Data Sources for National CSIRTs’ Cyber Incident Responses

National Computer Security Incident Response Teams (CSIRTs) are established worldwide to coordinate responses to cyber security incidents at the national level. It is known that software tools (including open source ones) and public data are routinely used to facilitate incident response in national CSIRTs. However, there is a lack of an authoritative set of criteria that can be used for a systematic evaluation to decide which software tools and data sources should be used by national CSIRTs for incident response. A prior study identified a set of potential candidate criteria for such an evaluation. The study presented in this article aims to validate these candidate criteria empirically by asking staff members of several national CSIRTs how they perceive the candidate criteria’s practical usefulness and readiness for deployment in national CSIRTs’ operations. The study involved online semi-structured interviews with nine interviewees from nine national CSIRTs in Asia-Pacific, Africa and Europe. After validating the candidate criteria using semi-structured interviews with these nine interviewees, we applied the criteria to evaluate a selection of software tools and data sources by converting each criterion into one or more relevant metrics, such as ‘measuring the time taken by a tool to produce results’. Results from the study led to the following main findings: (1) all interviewees perceived the candidate criteria as practically useful for evaluating tools and data sources in the operations of national CSIRTs; (2) all interviewees agreed that the candidate criteria could be deployed in national CSIRTs and other types of CSIRTs and (3) the candidate criteria can be applied relatively easily in practice. These criteria are envisaged to help national CSIRTs select the most appropriate tools and data sources to facilitate effective incident response, improve their operational practices and improve the quality of wider security operations.