pathfinder: A Semantic Framework for Literature Review and Knowledge Discovery in Astronomy
Abstract The exponential growth of astronomical literature poses significant challenges for researchers navigating and synthesizing general insights or even domain-specific knowledge. We present pathfinder , a machine learning framework designed to enable literature review and knowledge discovery in astronomy, focusing on semantic searching with natural language instead of syntactic searches with keywords. Utilizing state-of-the-art large language models (LLMs) and a corpus of 385,166 peer-reviewed papers from the Astrophysics Data System, pathfinder offers an innovative approach to scientific inquiry and literature exploration. Our framework couples advanced retrieval techniques with LLM-based synthesis to search astronomical literature by semantic context as a complement to currently existing methods that use keywords or citation graphs. It addresses complexities of jargon, named entities, and temporal aspects through time-based and citation-based weighting schemes. We demonstrate the tool’s versatility through case studies, showcasing its application in various research scenarios. The system’s performance is evaluated using custom benchmarks, including single-paper and multipaper tasks. Beyond literature review, pathfinder offers unique capabilities for reformatting answers in ways that are accessible to various audiences (e.g., in a different language or as simplified text), visualizing research landscapes, and tracking the impact of observatories and methodologies. This tool represents a significant advancement in applying artificial intelligence to astronomical research, aiding researchers at all career stages in navigating modern astronomy literature.
Authors
- Christine Ye (ORCID: https://orcid.org/0000-0002-8559-0788)
- Anjalie Field (ORCID: https://orcid.org/0000-0002-6955-746X)
- Mugdha Polimera (ORCID: https://orcid.org/0000-0001-6162-3963)
- Sanjib Sharma (ORCID: https://orcid.org/0000-0002-0920-809X)
- Jesse Cranney (ORCID: https://orcid.org/0000-0002-3015-9130)
- Alberto Accomazzi (ORCID: https://orcid.org/0000-0002-4110-3511)
- M. Jabłońska (ORCID: https://orcid.org/0000-0001-6962-4979)
- Mikaeel Yunus (ORCID: https://orcid.org/0000-0002-9851-2850)
- M. Ginolfi (ORCID: https://orcid.org/0000-0002-9122-1700)
- Sandor Kruk (ORCID: https://orcid.org/0000-0001-8010-8879)
- Kartheik G. Iyer (ORCID: https://orcid.org/0000-0001-9298-3523)
- Sergio José Rodríguez Méndez (ORCID: https://orcid.org/0000-0001-7203-8399)
- J. E. G. Peek (ORCID: https://orcid.org/0000-0003-4797-7030)
- Michael J. Smith (ORCID: https://orcid.org/0000-0003-0220-5125)
- Mike Walmsley (ORCID: https://orcid.org/0000-0002-6408-4181)
- Kevin Schawinski (ORCID: https://orcid.org/0000-0001-5464-0888)
- John F. Wu (ORCID: https://orcid.org/0000-0002-5077-881X)
- Tirthankar Ghosal (ORCID: https://orcid.org/0000-0002-2358-522X)
- Jill Naiman (ORCID: https://orcid.org/0000-0002-9397-6189)
- Yuan-Sen Ting (ORCID: https://orcid.org/0000-0001-5082-9536)
- Alina Hyk
- Simone Astarita
- Huiling Liu (ORCID: https://orcid.org/0009-0008-5660-1282)
- Ioana Ciucă (ORCID: https://orcid.org/0000-0001-6823-5453)
- R. C. Mistry
- Charles O’Neill (ORCID: https://orcid.org/0000-0003-2586-6874)
- Kiera McCormick
- Rishabh Chakrabarty
- Gabriel Marchidan
Publication Details
- Journal
- The Astrophysical Journal Supplement Series
- Published
- 2024-11-29
- DOI
- https://doi.org/10.3847/1538-4365/ad7c43
- Citations
- 8
- Primary Topic
- Semantic Web and Ontologies
- Type
- article
- Field-Weighted Citation Impact
- 2.78
Funders
- National Aeronautics and Space Administration
- Microsoft Research
- Johns Hopkins University
- Space Telescope Science Institute
- Kavli Institute for Theoretical Physics, University of California, Santa Barbara