Achieving GPT-4o level performance in astronomy with a specialized 8B-parameter large language model
AstroSage-Llama-3.1-8B is a domain-specialized natural-language AI assistant tailored for research in astronomy, astrophysics, cosmology, and astronomical instrumentation. Trained on the complete collection of astronomy-related arXiv papers from 2007 to 2024 along with millions of synthetically-generated question-answer pairs and other astronomical literature, AstroSage-Llama-3.1-8B demonstrates remarkable proficiency on a wide range of questions. AstroSage-Llama-3.1-8B scores 80.9% on the AstroMLab-1 benchmark, greatly outperforming all models-proprietary and open-weight-in the 8-billion parameter class, and performing on par with GPT-4o. This achievement demonstrates the potential of domain specialization in AI, suggesting that focused training can yield capabilities exceeding those of much larger, general-purpose models. AstroSage-Llama-3.1-8B is freely available, enabling widespread access to advanced AI capabilities for astronomical education and research.
Authors
- T. de Haan (ORCID: https://orcid.org/0000-0001-5105-9473)
- Alberto Accomazzi (ORCID: https://orcid.org/0000-0002-4110-3511)
- Nesar Ramachandra (ORCID: https://orcid.org/0000-0001-7772-0346)
- Azton I. Wells (ORCID: https://orcid.org/0000-0002-1183-7143)
- Tirthankar Ghosal (ORCID: https://orcid.org/0000-0002-2358-522X)
- Yuan-Sen Ting (ORCID: https://orcid.org/0000-0001-5082-9536)
- Zechang Sun
- Tuan Dung Nguyen (ORCID: https://orcid.org/0000-0002-1105-005X)
- Rui Pan (ORCID: https://orcid.org/0000-0003-4302-9543)
Institutions
- Argonne National Laboratory (US)
- Oak Ridge National Laboratory (US)
- High Energy Accelerator Research Organization (JP)
- Hong Kong University of Science and Technology (HK)
- Center for Astrophysics Harvard & Smithsonian (US)
- The Ohio State University (US)
- University of Pennsylvania (US)
- Tsinghua University (CN)
Publication Details
- Journal
- Scientific Reports
- Published
- 2025-04-21
- DOI
- https://doi.org/10.1038/s41598-025-97131-y
- Citations
- 4
- Primary Topic
- Topic Modeling
- Type
- article
- Field-Weighted Citation Impact
- 9.92
Funders
- National Science Foundation
- U.S. Department of Energy
- University of Chicago
- Ministry of Education, Culture, Sports, Science and Technology
- Office of Science
- Argonne National Laboratory
- Oak Ridge National Laboratory