Lessons learned from a Kaggle challenge for particle picking in cryo-electron tomography
Abstract The difficulty of particle picking in cryo-electron tomography remains a barrier to routine in situ structure determination. Machine learning is well-suited to overcome this bottleneck with efficient algorithms that generalize across molecular species. To spur new algorithm development, we held a three-month Kaggle challenge that tasked contestants with annotating five molecular species across hundreds of experimental tomograms. This competition successfully engaged >1000 participants from diverse fields and delivered particle pickers that outperformed existing state-of-the-art. Systematic comparisons of the contestants’ submissions revealed the tolerance of subtomogram averaging to moderate but not severe over-picking and underscored the need for more robust measures of annotation quality. The winning models also highlighted the importance of data augmentation to overcome limited training data. All competition tomograms along with the ground truth and winning teams’ annotations have been released on the CryoET Data Portal as a resource to benchmark current and future tools for particle picking.
Authors
- Mohammadreza Paraan (ORCID: https://orcid.org/0000-0002-8402-0134)
- Kira Evans
- Joshua Hutchings (ORCID: https://orcid.org/0000-0001-6841-8583)
- David List
- Bridget Carragher (ORCID: https://orcid.org/0000-0002-0624-5020)
- Zhuowen Zhao (ORCID: https://orcid.org/0000-0002-2355-2284)
- Dari Kimanius (ORCID: https://orcid.org/0000-0002-2662-6373)
- Dannielle G. McCarthy (ORCID: https://orcid.org/0000-0002-8810-0836)
- Yue Yu (ORCID: https://orcid.org/0000-0002-3248-9678)
- Walter Reade (ORCID: https://orcid.org/0000-0002-9215-700X)
- Sergio A. Silva (ORCID: https://orcid.org/0000-0001-5270-370X)
- Zun Shi Wang
- Clinton S. Potter (ORCID: https://orcid.org/0000-0002-3287-121X)
- Saugat Kandel (ORCID: https://orcid.org/0000-0002-7324-2559)
- Shawn Zheng (ORCID: https://orcid.org/0000-0001-9517-3075)
- Utz H. Ermel (ORCID: https://orcid.org/0000-0003-4685-037X)
- Kyle Harrington (ORCID: https://orcid.org/0000-0002-7237-1973)
- Jonathan Schwartz (ORCID: https://orcid.org/0000-0002-8063-6951)
- Ariana Peck (ORCID: https://orcid.org/0000-0002-5940-3897)
- David A. Agard (ORCID: https://orcid.org/0000-0003-3512-695X)
- Suzette McCanny
- Jing Ni (ORCID: https://orcid.org/0009-0001-4110-9403)
- Kandarp Khandwala (ORCID: https://orcid.org/0009-0003-2343-2105)
- Jessica Gadling
- Manasa Venkatakrishnan
- Jeremy Asuncion
- Janeece Pourroy
- Brendan Artley
Institutions
- Universidade Estadual de Maringá (BR)
- Google (United States) (US)
- University of British Columbia (CA)
- University of California, San Francisco (US)
- Chan Zuckerberg Biohub San Francisco (US)
- Chan Zuckerberg Initiative (United States) (US)
- Denver School of Nursing (US)
Publication Details
- Journal
- Nature Methods
- Published
- 2026-08-14
- DOI
- https://doi.org/10.1038/s41592-026-03198-4
- Citations
- 2
- Primary Topic
- Advanced Electron Microscopy Techniques and Applications
- Type
- article
- Field-Weighted Citation Impact
- 9.06