Harnessing the Power of LLMs: LLM Summarization for Human-Centric DAST Reports

Dynamic Application Security Testing (DAST) tools test web application security by simulating attacks on its front end and evaluating it externally like a malicious attacker. DAST tools aim to identify vulnerabilities and provide recommendations for improving security. However, the security alerts generated by these tools are lengthy and contain numerous details that may not be relevant to a software practitioner seeking a quick overview of the results. To solve this challenge, we propose using Large Language Models (LLMs) to summarize the alerts generated by DAST tools. We generated security alerts using two popular DAST tools: Burp Suite and ZAP. Then, we generated various summaries of these alerts using five different LLMs. We surveyed 48 software practitioners to understand the challenges software practitioners face when specifically dealing with DAST reports and determine the effectiveness of the LLM-generated summaries in understanding the security issue. The results bring to light various challenges software practitioners face and indicate that the LLM-generated summaries are clearer and more comprehensible in understanding the security issue and, hence, more preferred. This approach can significantly improve the security of software products by making the security alerts more accessible to different stakeholders, making the software product more robust and resilient to cyber threats.