Overview and status of the Machine Learning Data Platform project

The Machine Learning Data Platform (MLDP) is a product providing full-stack support for data science, Artificial Intelligence, and Machine Learning (AI/ML) applications at particle accelerator and large experimental physics facilities. It supports AI/ML applications from front-end, high-speed acquisition of heterogeneous, time-series data, through data archiving and management, to back-end analysis. The MLDP represents a “data-science ready” platform for the diagnosis, modeling, control, and optimization of these facilities. It offers a consistent, data-centric interface to archived data, standardizing implementation and deployment of AI/ML algorithms for different facility configurations, or between facilities. The MLDP is also deployable for experimental data collection, archiving, and analysis. It can acquire and archive heterogeneous data from experimental diagnostics (e.g., images, arrays, etc.) along with control system configurations, and any metadata required for provenance. Thus, the MLDP can manage experimental data through its entire lifecycle, from acquisition and archiving, through analysis and investigation, to release and final publication. The MLDP is a public-domain product and available to the community. The archive management system is fully independent with an installation and deployment utility (see https://github.com/osprey-dcs/data-platform). We present a brief MLDP project overview then detail the status and notable achievements.