Automatic genetic phenotype normalization from dysmorphology physical examinations: an overview of the BioCreative VIII—Task 3 competition

We present here an overview of the BioCreative VIII Task 3 competition, which called for the development of state-of-the-art approaches to automatic normalization of observations noted by physicians in dysmorphology physical examinations to the Human Phenotype Ontology (HPO). We made available for the task 3136 deidentified and manually annotated observations extracted from electronic health records of 1652 paediatric patients at the Children's Hospital of Philadelphia. This task is challenging due to the discontinuous, overlapping, and descriptive mentions of the observations corresponding to HPO terms, severely limiting the performance of straightforward strict matching approaches. Ultimately, an effective automated solution to the task will facilitate computational analysis that could uncover novel correlations and patterns of observations in patients with rare genetic diseases, enhance our understanding of known genetic conditions, and even identify previously unrecognized conditions. A total of 20 teams registered, and 5 teams submitted their predictions. We summarize the corpus, the competing systems approaches, and their results. The top system used a pre-trained large language model and achieved a 0.82 F1 score, which is close to human performance, confirming the impact that recent advances in natural language processing can have on tasks such as this. The post-evaluation period of the challenge, at https://codalab.lisn.upsaclay.fr/competitions/11351, will be open for submissions for at least 18 months past the end of the competition. Database URL: https://codalab.lisn.upsaclay.fr/competitions/11351.

Authors

Institutions

Publication Details

Journal
Database
Published
2025-01-01
DOI
https://doi.org/10.1093/database/baaf051
Primary Topic
Genomics and Rare Diseases
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Automatic genetic phenotype normalization from dysmorphology physical examinations: an overview of the BioCreative VIII—Task 3 competition

Goran Nenadić, Zhiquan Feng, Jin‐Hun Sohn, Graciela Gonzalez‐Hernandez et al.
Database
Genomics and Rare Diseases
article

Automatic genetic phenotype normalization from dysmorphology physical examinations: an overview of the BioCreative VIII—Task 3 competition

Goran Nenadić, Zhiquan Feng, Jin‐Hun Sohn, Graciela Gonzalez‐Hernandez, Tim Beck, Hongfei Lin, Karen O’Connor, Hajung Kim, Viktor Schlegel, ChanMin Kim, Ying-Jia Lin, Davy Weissenbacher, Areej Alhassan, A Laín, Riza Batista-Navarro, Zhihao Yang, Jessica Priestley, Sarah F. Schmidt, Xinwei Zhao, Ian M. Campbell, Joram M. Posma, Ekin Soysal, Jaewoo Kang, Monira Aloud, T. Ian Simpson, Marek Rei, Ling Luo, Kirk Roberts, Mujeen Sung, Katherine M. Szigety, Hung Yu Kao, Jiewei Qi
article en

Abstract

We present here an overview of the BioCreative VIII Task 3 competition, which called for the development of state-of-the-art approaches to automatic normalization of observations noted by physicians in dysmorphology physical examinations to the Human Phenotype Ontology (HPO). We made available for the task 3136 deidentified and manually annotated observations extracted from electronic health records of 1652 paediatric patients at the Children's Hospital of Philadelphia. This task is challenging due to the discontinuous, overlapping, and descriptive mentions of the observations corresponding to HPO terms, severely limiting the performance of straightforward strict matching approaches. Ultimately, an effective automated solution to the task will facilitate computational analysis that could uncover novel correlations and patterns of observations in patients with rare genetic diseases, enhance our understanding of known genetic conditions, and even identify previously unrecognized conditions. A total of 20 teams registered, and 5 teams submitted their predictions. We summarize the corpus, the competing systems approaches, and their results. The top system used a pre-trained large language model and achieved a 0.82 F1 score, which is close to human performance, confirming the impact that recent advances in natural language processing can have on tasks such as this. The post-evaluation period of the challenge, at https://codalab.lisn.upsaclay.fr/competitions/11351, will be open for submissions for at least 18 months past the end of the competition. Database URL: https://codalab.lisn.upsaclay.fr/competitions/11351.

DatabaseVol. 2025
Cedars-Sinai Medical Center (US), University City Science Center (US), Children's Hospital of Philadelphia (US), University of Nottingham (GB), Korea University (KR), Dalian University of Technology (CN), King Saud University (SA), University of Manchester (GB), Dalian University (CN), Korea Institute of Science & Technology Evaluation and Planning (KR), Intelligent Health (United Kingdom) (GB), Korea University (JP), Imperial College London (GB), University of Pennsylvania (US), National Cheng Kung University (TW), The University of Texas Health Science Center at Houston (US), University of Edinburgh (GB)
National Institute of Allergy and Infectious Diseases, National Institute of Child Health and Human Development, U.S. National Library of Medicine, Eunice Kennedy Shriver National Institute of Child Health and Human Development
Openalex Percentile: Top 72%
Genomics and Rare Diseases
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.