A community effort to optimize sequence-based deep learning models of gene regulation

A systematic evaluation of how model architectures and training strategies impact genomics model performance is needed. To address this gap, we held a DREAM Challenge where competitors trained models on a dataset of millions of random promoter DNA sequences and corresponding expression levels, experimentally determined in yeast. For a robust evaluation of the models, we designed a comprehensive suite of benchmarks encompassing various sequence types. All top-performing models used neural networks but diverged in architectures and training strategies. To dissect how architectural and training choices impact performance, we developed the Prix Fixe framework to divide models into modular building blocks. We tested all possible combinations for the top three models, further improving their performance. The DREAM Challenge models not only achieved state-of-the-art results on our comprehensive yeast dataset but also consistently surpassed existing benchmarks on Drosophila and human genomic datasets, demonstrating the progress that can be driven by gold-standard genomics datasets.

Authors

Institutions

Publication Details

Journal
Nature Biotechnology
Published
2024-10-11
DOI
https://doi.org/10.1038/s41587-024-02414-w
Citations
26
Primary Topic
Genomics and Chromatin Dynamics
Type
article
Field-Weighted Citation Impact
6.08

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

A community effort to optimize sequence-based deep learning models of gene regulation

Abdul Muntakim Rafi, Zhixiu Lu, P W Greaves, Lars Feuerbach et al.
26 citations
Nature Biotechnology
Genomics and Chromatin Dynamics
6.08
article

A community effort to optimize sequence-based deep learning models of gene regulation

Abdul Muntakim Rafi, Zhixiu Lu, P W Greaves, Lars Feuerbach, Sebastian Röner, Ashwin Narayanan, Aviv Regev, Jake Albrecht, Sangyeup Kim, Pratik Dutta, Daria Nogina, Jacob Bradford, Yuanfang Guan, Yanrong Ji, Il‐Youp Kwak, G. A. Meshcheryakov, Payman Yadollahpour, Ramana V. Davuluri, Pyaree Mohan Dash, Tin Nguyen, Scott Emrich, Chih-Han Huang, Jingzhe Wang, Yuedong Yang, Gabija Kavaliauskaite, Carl G. de Boer, Danyeong Lee, N. S. KIM, Wuwei Zhang, Edward S.C. Shih, Onuralp Söylemez, Dohoon Lee, Carl Schmitz, Owen Queen, Andreas Møller, Maria‐Anna Trapotsi, Jhih-Yu Chen, Ashley Babjac, Dmitry Penzar, Pablo Meyer, Ashok Palaniappan, Hsuan-Kai Wang, Konstantinos Kyriakidis, Jesper Grud Skat Madsen, Ibrahim Alsaggaf, Gunjan Dixit, Benedikt Brors, Cen Wan, Max Schubach, Andigoni Malousi, Peter K. Koo, Ivan V. Kulakovskiy, Robert Salomone, Wei Qiu, Krishna Kant Gupta, Wuming Gong, Tsai‐Min Chen, Eeshit Dhaval Vaishnav, Taein Kang, Arsenii Zinkevich, Susanne Bornelöv, Xiaoting Chen, Maolin Ding, Byeongchan Kim, Andrey Lando, Yeojin Shin, Jiayu Wen, Xinming Tu, Michele Tinti, Sz-Hau Chen, Ke Ding, Fredrik Svensson, Chih‐Hsun Wu, Dimitri Perrin, Konstantinos Kardamiliotis, Prasanna Kumar Saravanam, Ethan Labelson, Rohan Ghotra, Nicholas Allen Baclig Abad, Han Liu, Juhyun Lee, Ken Chen, Sun Kim, Jiaxing Chen, Yiyang Yu, Dohyeon Kim, Yichao Li, Duc Tran, Zhihan Zhou, Carl Barton, Pallavi Surana, Yu Hiratsuka, Yang AiWei, Cindy Körner, David T. Jones, Kuei-Lin Huang, Aayush Prakash, Rekha Sathian, Mao Takatsu
article en
26 citations

Abstract

A systematic evaluation of how model architectures and training strategies impact genomics model performance is needed. To address this gap, we held a DREAM Challenge where competitors trained models on a dataset of millions of random promoter DNA sequences and corresponding expression levels, experimentally determined in yeast. For a robust evaluation of the models, we designed a comprehensive suite of benchmarks encompassing various sequence types. All top-performing models used neural networks but diverged in architectures and training strategies. To dissect how architectural and training choices impact performance, we developed the Prix Fixe framework to divide models into modular building blocks. We tested all possible combinations for the top three models, further improving their performance. The DREAM Challenge models not only achieved state-of-the-art results on our comprehensive yeast dataset but also consistently surpassed existing benchmarks on Drosophila and human genomic datasets, demonstrating the progress that can be driven by gold-standard genomics datasets.

Nature BiotechnologyVol. 43(8)
Broad Institute (US), Northwestern University (US), Australian National University (AU), Beijing Normal-Hong Kong Baptist University (CN), Cincinnati Children's Hospital Medical Center (US), University of British Columbia Hospital (CA), University of Minnesota (US), St. Jude Children's Research Hospital (US), Sage Bionetworks (US), University of Nevada, Reno (US), IBM (United States) (US), Seoul National University (KR), Hong Kong Baptist University (HK), University of British Columbia (CA), Sun Yat-sen University (CN), Queensland University of Technology (AU), National Taiwan University (TW), University of Southern Denmark (DK), German Cancer Research Center (DE), China Medical University (TW), Lomonosov Moscow State University (RU), University of Washington (US), Aristotle University of Thessaloniki (GR), Heidelberg University (DE), University of Cambridge (GB), Beijing Normal University (CN), Cold Spring Harbor Laboratory (US), University of Michigan (US), Friends United (US), Research Center for Information Technology Innovation, Academia Sinica (TW), Institute of Biomedical Sciences, Academia Sinica (TW), National Centre for Cell Science (IN), Syosset Hospital (US), Global Blood Therapeutics (United States) (US), Wellcome Centre for Anti-Infectives Research (GB), Development Center for Biotechnology (TW), Freenome (United States) (US), Institute of Protein Research (RU), Berlin Institute of Health at Charité - Universitätsmedizin Berlin (DE), Vavilov Institute of General Genetics (RU), University College London (GB), Yandex (Russia) (RU), Stony Brook University (US), Chung-Ang University (KR), Niigata University (JP), University of Tennessee at Knoxville (US), Academia Sinica (TW), National Chengchi University (TW), SASTRA University (IN), National Yang Ming Chiao Tung University (TW), Birkbeck, University of London (GB)
Stem Cell Network, Michael Smith Health Research BC, National Research Foundation, Seoul National University, Canadian Network for Research and Innovation in Machining Technology, Natural Sciences and Engineering Research Council of Canada, National Research Foundation of Korea, University of British Columbia, Russian Science Foundation, Iran Telecommunication Research Center, Ministry of Science and ICT, South Korea, Alliance de recherche numérique du Canada, National Institutes of Health, Natural Sciences and Engineering Research Council of Canada
Openalex Percentile: Top 3%
Genomics and Chromatin Dynamics
6.08
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.