The Influence of Using LLMs on the Activities of a Software Testing Team: An Industry Case

Context: In this study, we investigated the integration of Large Language Models (LLMs) in test teams, aiming to understand how these tools can improve the test process and improve productivity. The main goal was to determine if the implementation of LLMs can enhance the productivity of test professionals. Methodology: We performed an empirical study with 14 participants, and the results were analysed quantitatively and qualitatively. For the quantitative analysis, we analysed the frequency of activities, and for the qualitative analysis, we analysed the participants’ perception of productivity through a scoring system, in which participants gave a score between 0 to 5 for their perception of learning from before and after the implementation of LLM in the team. Results: The results show a significant improvement in productivity (p-value = 0,0193) after the implementation of LLM. Furthermore, the qualitative analysis provided data on the impact of LLM in learning, use experience, trust perception and productivity. Conclusion: It was noted that the integration of LLMs in test teams can be a valuable tool to improve productivity and efficiency of test professionals.