Ensuring Test Case Coverage through Multilabel Requirements Classification: A Feasibility Study with Pre-Trained Models

Context: In a software development project, the developers’ main task is to understand the requirements to then implement them without bugs. Particularly, this paper is inserted in the context of a test team from a software institute that receives new requirements daily. With the goal of verifying if the requirements were correctly implemented, test cases are created based on these requirements, to then confirm if the software is working. Problem: Each requirement must be associated to one or more test cases, thus allowing tracking and ensuring the validation during the test process. However, manually classifying requirements to their associated test cases demands time and effort. Solution: This work details a study about the viability of using the pre-trained models: BERT, Electra, RoBERTa, DeBERTa and XLNet in the process of multi-label classification of requirements, identifying the test cases associated to each requirement. Method: For this study, we considered requirements received by the test team from 2024 to April 2025, and 38 associated test cases. After collecting and preparing the data, we trained the models and measured their performance through the metrics: Hamming Loss, Jaccard, Precision, Recall and F1-Score. Results: Results showed that all models had low Hamming Loss values, specially the BERT model, with 0.012 Hamming Loss. After performing the Friedman and Conover tests, it was determined that there is a significative discrepany between the models. Conclusions: Therefore, this research’s results show that it is possible to use pre-trained models to classify requirements with associated test cases.