Artifacts - BDD4ML: A Behavior-Driven Development Framework for Machine Learning Models

Abstract: Context / Problem: Machine Learning (ML) systems present unique testing challenges due to their non-deterministic nature and lack of formal specifications. Aims: We introduce BDD4ML, a framework that adapts Behavior-Driven Development (BDD) principles for testing of ML models. Method: We designed BDD4ML to support both classification and regression model testing through natural language clauses in Gherkin syntax, and implemented it using the Python Behave framework, allowing model expectations to be explicit and executable. The framework was evaluated through an academic observational study and an industrial focus group discussion, both assessing technology acceptance. In the observational study, graduate students used the framework to automate tests for an industrial ML model. In the focus group, the ML experts responsible for developing the ML model were presented with the BDD4ML solution and discussed the usage of the framework within real projects. Results: In the classroom setting, participants were able to specify BDD4ML scenarios for successfully testing the industrial model with minimal mistakes. A majority of the students agreed on the framework’s usefulness (84.62%), ease of use (76.92%), and expressed intention to adopt it (76.92%). In the industrial focus group, participants also considered the BDD4ML useful and easy to use, and showed interest in adopting it. In the discussions, they highlighted BDD4ML’s value for transparency, decision support, model monitoring, and collaborative requirement specification. Overall, The results indicate BDD4ML as a promising approach for bringing BDD practices to ML model testing. The framework is openly available for use and extension. Our next steps include conducting another focus group with more experienced machine learning developers to identify additional areas for improvement.