RPerf: Mining User Reviews Using Topic Modeling to Assist Performance Testing: An Industrial Experience Report

Software performance affects the user-perceived quality of software. Therefore, it is important to analyze the performance issues that users are concerned with. In this paper, we document our experience working with our industry partner on analyzing user reviews to identify and analyze performance issues users are concerned with. In particular, we designed an approach, RPerf, which automatically analyzes unstructured user reviews and generates a performance analysis report that can assist performance engineers with performance testing. In particular, RPerf uses BERTopic to uncover performance-related topics in user reviews. RPerf then maps the derived topics to performance KPIs (key performance indicators) such as response time. Such performance KPIs better help performance test design and allocate performance testing resources. Finally, RPerf extracts user usage scenarios from user reviews to help identify the causes. Through a manual evaluation, we find that RPerf achieves a high accuracy (over 93%) in identifying the performance-related topics and performance KPIs from user reviews. RPerf can also accurately extract usage scenarios in over 80% of user reviews. We discuss the performance analysis report that is generated based on RPerf. We believe that our findings can assist practitioners with analyzing performance-related user reviews and inspire future research on user review analysis.