LLM-based Crowdsourced Test Report Clustering

The openness of crowdsourced testing introduces diversity in testing results. However, it also leads to a large volume of test reports, many of which highlight the same recurring issues. While these reports provide valuable feedback, their redundancy makes it inefficient for developers to review the reports and identify bugs. Crowdsourced test report clustering has been proposed to mitigate this problem, allowing developers to focus only on the representative reports from each cluster. However, existing methods primarily rely on embedding features extracted from reports for clustering, which limits their ability to generate accurate and interpretable clusters due to a lack of deeper semantic understanding of the reports. To address the aforementioned challenge, we propose LLMCluster , a novel method for crowdsourced test report clustering based on large language models (LLMs). LLMCluster employs an iterative clustering strategy. In each iteration, LLMCluster processes a subset of reports by instructing the LLM to disregard surface-level variations in expression, analyze the core issue in each report, and group reports addressing the same issue into new or existing clusters. After the iterative clustering process, LLMCluster applies correction algorithms to ensure the completeness and validity of the clustering result. Finally, LLMCluster utilizes the LLM to generate concise summaries for each cluster, making the results more intuitive and interpretable. Experimental results show that LLMCluster outperforms state-of-the-art methods across six commonly used clustering evaluation metrics. Additionally, the cluster summaries generated by LLMCluster semantically align well with manually written summaries.