DPA-2: a large atomic model as a multi-task learner

The rapid advancements in artificial intelligence (AI) are catalyzing transformative changes in atomic modeling, simulation, and design. AI-driven potential energy models have demonstrated the capability to conduct large-scale, long-duration simulations with the accuracy of <i>ab initio</i> electronic structure methods. However, the model generation process remains a bottleneck for large-scale applications. We propose a shift towards a model-centric ecosystem, wherein a large atomic model (LAM), pre-trained across multiple disciplines, can be efficiently fine-tuned and distilled for various downstream tasks, thereby establishing a new framework for molecular modeling. In this study, we introduce the DPA-2 architecture as a prototype for LAMs. Pre-trained on a diverse array of chemical and materials systems using a multi-task approach, DPA-2 demonstrates superior generalization capabilities across multiple downstream tasks compared to the traditional single-task pre-training and fine-tuning methodologies. Our approach sets the stage for the development and broad application of LAMs in molecular and materials simulation research.

Authors

Institutions

Publication Details

Journal
npj Computational Materials
Published
2024-12-19
DOI
https://doi.org/10.1038/s41524-024-01493-2
Citations
105
Primary Topic
Machine Learning in Materials Science
Type
article
Field-Weighted Citation Impact
12.30

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

DPA-2: a large atomic model as a multi-task learner

Junhan Chang, Zhicheng Zhong, Fu‐Zhi Dai, Jun Cheng et al.
105 citations
npj Computational Materials
Machine Learning in Materials Science
12.30
article

DPA-2: a large atomic model as a multi-task learner

Junhan Chang, Zhicheng Zhong, Fu‐Zhi Dai, Jun Cheng, Mengchao Shi, Anyang Peng, Linshuang Zhang, Wanrun Jiang, Jiyuan Yang, Jinzhe Zeng, Chengqian Zhang, Fu‐Qiang Gong, Guolin Ke, Jianchuan Liu, Manyi Yang, Xiaoshan Luo, Xinzijian Liu, E Weinan, Tong Zhu, Mohan Chen, Darrin M. York, Siyuan Liu, Zhenyu Wang, Xiangyu Zhang, Bowen Li, Xinyan Wang, Yifan Li, Handong Wang, Duo Zhang, Shi Liu, Yuzhi Zhang, Linfeng Zhang, Yiming Du, Yudi Yang, Jiameng Huang, Weile Jia, Chun Cai, Jing Wu, Shuo Zhou, Jian Yi Lv, Xuejian Qin, Yifan Shan, Hangrui Bi
article en
105 citations

Abstract

The rapid advancements in artificial intelligence (AI) are catalyzing transformative changes in atomic modeling, simulation, and design. AI-driven potential energy models have demonstrated the capability to conduct large-scale, long-duration simulations with the accuracy of <i>ab initio</i> electronic structure methods. However, the model generation process remains a bottleneck for large-scale applications. We propose a shift towards a model-centric ecosystem, wherein a large atomic model (LAM), pre-trained across multiple disciplines, can be efficiently fine-tuned and distilled for various downstream tasks, thereby establishing a new framework for molecular modeling. In this study, we introduce the DPA-2 architecture as a prototype for LAMs. Pre-trained on a diverse array of chemical and materials systems using a multi-task approach, DPA-2 demonstrates superior generalization capabilities across multiple downstream tasks compared to the traditional single-task pre-training and fine-tuning methodologies. Our approach sets the stage for the development and broad application of LAMs in molecular and materials simulation research.

npj Computational MaterialsVol. 10(1)
Rutgers, The State University of New Jersey (US), Xihua University (CN), Xiamen University (CN), Jilin University (CN), Chinese Academy of Sciences (CN), Princeton University (US), Peking University (CN), New York University Shanghai (CN), Italian Institute of Technology (IT), Westlake University (CN), Institute of Computing Technology (CN), Quantitative BioSciences (US), Institute of Applied Physics and Computational Mathematics (CN), Collaborative Innovation Center of Chemistry for Energy Materials (CN), University of Chinese Academy of Sciences (CN), Ningbo Institute of Industrial Technology (CN), State Key Laboratory of High Pressure and Superhard Materials (CN), East China Normal University (CN)
National Science Foundation, U.S. Department of Energy, National Natural Science Foundation of China, National Institutes of Health, National Science Fund for Distinguished Young Scholars, Natural Science Foundation of Zhejiang Province
Quality Education
Openalex Percentile: Top 0%
Machine Learning in Materials Science
12.30
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.