Pt II: Evaluating the Natural Language Processing Algorithm Elastex for the Phenotype-Guided Genomic Diagnosis Platform, Genomediver

This study evaluates the performance of Elastex, a natural language processing (NLP) algorithm in extracting Human Phenotype Ontology (HPO) terms for integration into GenomeDiver, a phenotype-guided genomic diagnosis platform. Using a manually curated gold standard (GS) dataset derived from 100 neurodevelopmental patient records, we assessed Elastex’s precision, recall, and overall effectiveness in extracting clinically relevant phenotypic information. The GS identified 538 unique HPO mapped terms, while Elastex extractions identified 527 with an average precision of 43% and a recall of 22%. Recall was notably low, highlighting Elastex’s preference for specificity over breadth. Error analysis revealed common issues such as misinterpretation of negations and redundant extractions resulting in a pooled kappa value of -0.38 (agreement between GS vs. Elastex). Inter-annotator agreement was also measured using pooled kappas and was poor (-0.34 to -0.40). However, this supports that discordance between the GS and Elastex was not due to random error, but rather systematic differences in how extraction was done. This emphasizes not only the variability among different annotators, but also the inherent subjectivity in manual extraction processes even with the implementation of a standard set of guidelines. Despite its limitations, Elastex demonstrates improving potential as a scalable and reproducible tool for preliminary HPO extraction when used in conjunction with human oversight. Its integration into GenomeDiver could reduce diagnostic delays and streamline phenotypic curation by surfacing reliable candidate terms for expert validation. Future improvements to enhance context awareness and synonym recognition may further increase its clinical utility in rare disease diagnostics.