Artifacts - BDD4ML: A Behavior-Driven Development Framework for Machine Learning Models

Abstract—[Context] Machine Learning (ML) systems presentunique testing challenges due to their non-deterministic natureand lack of formal specifications. [Goal] In this paper, we intro-duce BDD4ML, a framework that adapts Behavior-Driven Devel-opment (BDD) principles for testing of ML models. [Method] Wedesigned BDD4ML to support both classification and regressionmodel testing through natural language clauses in Gherkinsyntax, and implemented it using the Python Behave framework,allowing model expectations to be explicit and executable. Theframework was evaluated through an academic observationalstudy and an industrial focus group discussion, both assessingtechnology acceptance. In the observational study, graduatestudents used the framework to automate tests for an industrialML model. In the focus group, the ML experts responsible fordeveloping the ML model were presented with the BDD4MLsolution and discussed the usage of the framework within realprojects. [Results] In the classroom setting, participants wereable to specify BDD4ML scenarios for successfully testing theindustrial model with minimal mistakes. A majority of thestudents agreed on the framework’s usefulness (84.62%), ease ofuse (76.92%), and expressed intention to adopt it (76.92%). In theindustrial focus group, participants also considered the BDD4MLuseful and easy to use, and showed interest in adopting it. In thediscussions, they highlighted BDD4ML’s value for transparency,decision support, model monitoring, and collaborative require-ment specification. [Conclusion] The results indicate BDD4ML asa promising approach for bringing BDD practices to ML modeltesting. The framework is openly available for use and extension.