Towards reproducible machine learning-based process monitoring and quality prediction research for additive manufacturing

Additive manufacturing (AM) is increasingly adopted across industries for its ability to support design flexibility, rapid prototyping, and mass customization. Machine learning (ML)-based cyber-physical systems (CPSs) have been extensively developed to improve the print quality of AM. However, the reproducibility of these systems has not been thoroughly investigated due to a lack of formal evaluation methods. Reproducibility, a critical component of trustworthy artificial intelligence, is achieved when an independent team can replicate the findings or artifacts of a study using a different experimental setup and achieve comparable performance. In many publications, critical information necessary for reproduction is often missing due to a lack of comprehensive AM and ML domain knowledge, resulting in systems that fail to replicate the reported performance. Integrating AM and ML domain knowledge, this paper proposes a reproducibility investigation pipeline and a reproducibility checklist for ML-based AM process monitoring and quality prediction systems. Based on the CRoss Industry Standard Process (CRISP) methodology, the pipeline guides researchers through the key steps required to reproduce a study, while the checklist systematically extracts information relevant to reproducibility from the publication. We validated the proposed approach through two case studies: reproducing a fused filament fabrication warping detection system and a laser powder bed fusion melt pool area prediction model. Both case studies confirmed that the pipeline and checklist successfully identified missing information, improved reproducibility, and enhanced the performance of reproduced systems. Based on the proposed checklist and leveraging large language models, a reproducibility survey was conducted to assess the current reproducibility status within this research domain.