dpdata: A Scalable Python Toolkit for Atomistic Machine Learning Data Sets

Seamless management of atomistic data sets is a critical prerequisite for the successful development and deployment of machine learning potentials (MLPs). Here, we present dpdata, an open-source Python library designed to streamline every aspect of MLP data handling. Built upon a flexible, plugin-based architecture, dpdata supports reading, writing, and converting between a broad range of file formats─from popular quantum-chemistry packages and molecular-dynamics engines to specialized MLP frameworks. Users may define custom data types, formats, drivers, and minimizers, enabling effortless extension to emerging software. Key utilities include automated train-test splitting, coordinate perturbation for active learning, outlier-energy removal, Δ-learning data set generation, error-metric computation, and unit conversion. Through efficient NumPy-backed storage and system-level operations, dpdata achieves significant memory saving and inference speedups over configuration-by-configuration tools such as ASE. We also highlight practical impact, with dpdata used across published studies, for format conversion, data storage, coordinate perturbation, and utilization in other projects for data processing.

Authors

Institutions

Publication Details

Journal
Journal of Chemical Information and Modeling
Published
2025-10-20
DOI
https://doi.org/10.1021/acs.jcim.5c01767
Citations
1
Primary Topic
Machine Learning in Materials Science
Type
article
Field-Weighted Citation Impact
0.52
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

dpdata: A Scalable Python Toolkit for Atomistic Machine Learning Data Sets

Cao Thang Nguyen, Ping Tuo, Rundong Liao, Anyang Peng et al.
1 citations
Journal of Chemical Information and Modeling
Machine Learning in Materials Science
0.52
article

dpdata: A Scalable Python Toolkit for Atomistic Machine Learning Data Sets

Cao Thang Nguyen, Ping Tuo, Rundong Liao, Anyang Peng, Logan Ward, Chenxing Luo, Wanrun Jiang, Pan Xiang, Yixiao Chen, Liping Liu, Fengbo Yuan, Jinzhe Zeng, Jiaxin Zhu, Chengqian Zhang, R. Zhou, Renxi Liu, Marián Rynik, Junjie Wang, Xuyuan Zhou, Denan Li, Yong‐Bin Zhuang, Kaiwei Wan, Bowen Li, Wenshuo Liang, Haidi Wang, Jiahao Fan, Haoxi Li, Yifan Li, Handong Wang, Duo Zhang, Yuzhi Zhang, Linfeng Zhang, Weihong Xu, Tao Chen, Jiameng Huang, Xulong Peng, Zezhong Zhang, Jinchao Zhang, Yingze Wang
article en
1 citations

Abstract

Seamless management of atomistic data sets is a critical prerequisite for the successful development and deployment of machine learning potentials (MLPs). Here, we present dpdata, an open-source Python library designed to streamline every aspect of MLP data handling. Built upon a flexible, plugin-based architecture, dpdata supports reading, writing, and converting between a broad range of file formats─from popular quantum-chemistry packages and molecular-dynamics engines to specialized MLP frameworks. Users may define custom data types, formats, drivers, and minimizers, enabling effortless extension to emerging software. Key utilities include automated train-test splitting, coordinate perturbation for active learning, outlier-energy removal, Δ-learning data set generation, error-metric computation, and unit conversion. Through efficient NumPy-backed storage and system-level operations, dpdata achieves significant memory saving and inference speedups over configuration-by-configuration tools such as ASE. We also highlight practical impact, with dpdata used across published studies, for format conversion, data storage, coordinate perturbation, and utilization in other projects for data processing.

Journal of Chemical Information and ModelingVol. 65(21)
University of North Carolina at Chapel Hill (US), University of Science and Technology of China (CN), Institute of Applied Physics (UA), Argonne National Laboratory (US), Microsoft (United States) (US), Hefei University of Technology (CN), City University of Hong Kong (HK), Iowa State University (US), Xiamen University (CN), Chinese Academy of Sciences (CN), Princeton University (US), Peking University (CN), Baker Hughes (United States) (US), Art Institute of Portland (US), Suzhou University of Science and Technology (CN), Westlake University (CN), University of Alabama at Birmingham (US), Laboratory of Molecular Genetics (PL), Microsoft (Finland) (FI), IRD Fuel Cells (Denmark) (DK), Beijing National Laboratory for Molecular Sciences (CN), FZU ‒ Institute of Physics of the Academy of Sciences of the Czech Republic (CZ), National Test Pilot School (US), Center for Interdisciplinary Studies (IN), Computational Physics (United States) (US), Collaborative Innovation Center of Advanced Microstructures (CN), SP Technology (South Korea) (KR), Institute of Applied Physics and Computational Mathematics (CN), Institute of Semiconductors (CN), Collaborative Innovation Center of Chemistry for Energy Materials (CN), Institute of Physics (CN), Ningbo Institute of Industrial Technology (CN), Centre for Innovation Studies (CA), Preferred Networks (Japan) (JP), Ulsan National Institute of Science and Technology (KR), East China Normal University (CN), Comenius University Bratislava (SK), Columbia University (US), Beihang University (CN), Nanjing University (CN), University of California, Berkeley (US)
Openalex Percentile: Top 31%
Machine Learning in Materials Science
0.52
DOISourceOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.