Automated Classification and Identification of Non-Functional Requirements in Agile-Based Requirements Using Pre-Trained Language Models

Non-functional requirements (NFRs) are critical factors for software quality and success. A frequently reported challenge in agile requirements engineering is that NFRs are often neglected due to the focus on functional requirements (FRs) and the limited capability of agile requirements documented as user stories to represent NFRs. With the emergence of transfer learning and large pre-trained language models, various applications in requirements engineering have become feasible, alleviating several longstanding challenges. This study evaluates transformer-based models for the automated identification and classification of NFRs. We leveraged transfer learning with pre-trained transformer models to automate the identification and classification of NFRs in agile textual requirements documented as user stories. A dataset of over 10k user stories was collected and labeled, and pre-trained transformer models, including BERT, RoBERTa, XLNet, and DistilBERT, were fine-tuned to automate the identification of NFRs. We incorporated Focal Loss during training to mitigate the dominance of functionally driven requirements and class imbalances. In addition, thorough experiments on hyperparameter optimization were employed using Bayesian hyperparameter optimization to obtain the combination of hyperparameters that best correlated with the aim of enhancing each model’s performance. Our evaluation demonstrated that the finetuned pre-trained models significantly outperformed comparable prior approaches relying on rule-based techniques or traditional machine learning, with a fine-tuned BERT model achieving an F1 Score of 93.4 %. These findings highlight the potential of pre-trained language models in agile requirements engineering, enabling more efficient NFRs identification, reducing manual review burden, and facilitating a viable and efficient approach to address the neglect of NFRs in agile development processes.