Probabilistic Measurement of CTI Quality for Large Numbers of Unstructured CTI Products

This paper addresses the critical challenge of evaluating the quality of Cyber Threat Intelligence (CTI) products, particularly focusing on their relevance and actionability. As organizations increasingly rely on CTI to make cybersecurity decisions, the absence of CTI quality metrics challenges the assessment of intelligence quality. To address this gap, the article introduces two innovative metrics. Relevance (Re) and Actionability (Ac) are designed to evaluate CTI products in relation to organizational information needs and defense mechanisms. Using probabilistic algorithms and data structures, these metrics provide a scalable approach for handling large numbers of unstructured CTI products. Experimental findings demonstrate the effectiveness of metrics in filtering and prioritizing CTI products, offering organizations a tool to prioritize their cybersecurity resources. Furthermore, experimental results demonstrate that, using the metrics, organizations can reduce candidate CTI products by several orders of magnitude, understand weaknesses in defining information needs, guide the application of CTI products, assess CTI products’ contribution to defense, and select CTI products from information sharing communities. In addition, the study has identified certain limitations, which open avenues for future research, including the real-time integration of CTI into organizational defense mechanisms. This work significantly contributes to standardizing the quality evaluation of CTI products and enhancing organizations’ cybersecurity posture.