Large language models surpass human experts in predicting neuroscience results

Scientific discoveries often hinge on synthesizing decades of research, a task that potentially outstrips human information processing capacities. Large language models (LLMs) offer a solution. LLMs trained on the vast scientific literature could potentially integrate noisy yet interrelated findings to forecast novel results better than human experts. Here, to evaluate this possibility, we created BrainBench, a forward-looking benchmark for predicting neuroscience results. We find that LLMs surpass experts in predicting experimental outcomes. BrainGPT, an LLM we tuned on the neuroscience literature, performed better yet. Like human experts, when LLMs indicated high confidence in their predictions, their responses were more likely to be correct, which presages a future where LLMs assist humans in making discoveries. Our approach is not neuroscience specific and is transferable to other knowledge-intensive endeavours.

Authors

Institutions

Publication Details

Journal
Nature Human Behaviour
Published
2024-11-27
DOI
https://doi.org/10.1038/s41562-024-02046-9
Citations
95
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
33.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Large language models surpass human experts in predicting neuroscience results

Jessica Dafflon, Michael Gaebler, Kangjoo Lee, Xiaoliang Luo et al.
95 citations
Nature Human Behaviour
Topic Modeling
33.00
article

Large language models surpass human experts in predicting neuroscience results

Jessica Dafflon, Michael Gaebler, Kangjoo Lee, Xiaoliang Luo, Nianlong Gu, Pasquale Minervini, Alexandra O. Cohen, Anna Behler, Justin M. Ales, Leyla Loued‐Khenissi, Mikail Khona, Sebastian Musslick, Anton Pashkov, Sherry Dongqi Bao, Jennifer K. Bizley, Rui Mata, Roberta Rocca, Valentina Borghesani, Daniele Marinazzo, Nicholas E. Myers, Ilia Sucholutsky, Martin Ferianc, N. Apurva Ratan Murty, Isil Poyraz Bilgin, Bradley C. Love, Felipe Yáñez, Guiomar Niso, Alessandro Salatiello, Guangzhi Sun, Pui-Shee Lee, Kaustubh R. Patil, Chloe M. Hall, Jonathan Nicholas, Elkhan Yusifov, Akilles Rechardt, Kevin Kermani Nejad, Tereza Okalova, S. Morteza Razavi, Bati Yilmaz
article en
95 citations

Abstract

Scientific discoveries often hinge on synthesizing decades of research, a task that potentially outstrips human information processing capacities. Large language models (LLMs) offer a solution. LLMs trained on the vast scientific literature could potentially integrate noisy yet interrelated findings to forecast novel results better than human experts. Here, to evaluate this possibility, we created BrainBench, a forward-looking benchmark for predicting neuroscience results. We find that LLMs surpass experts in predicting experimental outcomes. BrainGPT, an LLM we tuned on the neuroscience literature, performed better yet. Like human experts, when LLMs indicated high confidence in their predictions, their responses were more likely to be correct, which presages a future where LLMs assist humans in making discoveries. Our approach is not neuroscience specific and is transferable to other knowledge-intensive endeavours.

Nature Human BehaviourVol. 9(2)
University of Geneva (CH), Turing Institute (GB), Georgia Institute of Technology (US), University of Nottingham (GB), Novosibirsk State Medical University (RU), Emory University (US), University of St Andrews (GB), Bilkent University (TR), Osnabrück University (DE), University of Basel (CH), Johannes Gutenberg University Mainz (DE), Princeton University (US), University of Zurich (CH), Aarhus University (DK), Instituto Cajal (ES), University of Cambridge (GB), Brown University (US), University College of Osteopathy (GB), Ghent University (BE), Yale University (US), University of Bristol (GB), University of Oxford (GB), University Medical Center of the Johannes Gutenberg University Mainz (DE), Quantitative BioSciences (US), Institut Universitaire de Gériatrie de Montréal (CA), The Alan Turing Institute (GB), Max Planck Institute for Human Cognitive and Brain Sciences (DE), Centre Hospitalier de Valence (FR), Federal Research Center of Fundamental and Translational Medicine (RU), Federal State Budgetary Institution "Federal Center For Cardiovascular Surgery" Ministry of Health of The Russian Federation (RU), The London College (GB), Federal Neurosurgical Center Novosibirsk (RU), National Institute of Mental Health (US), Max Planck Institute for Neurobiology of Behavior – caesar (DE), Heinrich Heine University Düsseldorf (DE), University College London (GB), New York University (US), Technical University of Munich (DE), Massachusetts Institute of Technology (US), University of Newcastle Australia (AU), University of Pennsylvania (US), University of Tübingen (DE), Ludwig-Maximilians-Universität München (DE), Novosibirsk State Technical University (RU), University of Lausanne (CH), University of Edinburgh (GB)
Royal Society, Economic and Social Research Council
Quality Education
Openalex Percentile: Top 0%
Topic Modeling
33.00
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.