Automated Ethical Requirements Classification in AI: The ERC4AI Approach

Context: The operationalization of AI ethical principles during the requirements engineering phase is paramount for creating ethically aligned AI systems. However, ethical requirements classification remains a significant challenge due to the abstract nature of ethical principles and the lack of practical tools to support developers. Objective: We introduce the EthicalRequirements4AI, a dataset comprising 1,091 annotated requirements, including ethical and non-ethical requirements. We also provide the Ethical Requirements Classification for AI (ERC4AI), a model for classifying requirements based on AI ethical principles. Method: Passenger Flow and PROMISE datasets were used to construct the EthicalRequirements4AI dataset, encompassing \textcolor{blue}{385 requirements aligned with 11 AI ethical principles and 706 non-ethical requirements.} Three language models – XLM-RoBERTa, BERT, and DistilBERT – were evaluated for multi-label classification task. Results: BERT demonstrated the strongest overall performance among the evaluated models (XLM-RoBERTa, BERT, and DistilBERT) for multi-label classification, achieving a Macro F1-score of 0.77 and a Weighted F1-score of 0.83 on the test set. While BERT showed the best aggregate results, performance varied across different ethical principles.} Conclusions: ERC4AI offers a novel approach for researchers and developers to address ethical concerns early in AI development. The dataset and models are publicly available to foster further research and advancements in practical AI ethics.