Static and Dynamic Embedding Approaches to Identify Is-A Relations in SNOMED CT

Systematized Nomenclature of Medicine - Clinical Terms (SNOMED CT) is a comprehensive clinical terminology with over 350,000 unique concepts and 1 million relationships. It serves as an essential resource for both clinical practice and medical research. Approximately 40% of these relationships are of the is-a type, which is key to classifying concepts as subtypes or subclasses within broader categories and is fundamental for decision support systems and automated reasoning in clinical environments. Identifying is-a relationships in SNOMED CT is crucial for enhancing the accuracy of patient cohort queries, which form the backbone of clinical and research applications. Our study aims to use deep learning-based neural networks to determine whether a relationship between two concepts falls under the is-a or non-is-a categories. This approach can help detect misclassified or undefined is-a concept pairs, ultimately improving the quality of SNOMED CT ontology. We construct a node-edge-node structure for concept pairs and their relationships, which we then split into two categories: is-a and non-is-a classes. In this study, we propose two classification approaches. In the first approach using embeddingBag, the data are mapped into vector representations and used as input for a fully connected neural network to learn the patterns of the is-a relationship. During model training, 80% of the data is used for training and the remaining 20% for testing.We achieved a precision of 0.903, recall of 0.894, and F1 Score of 0.898 in predicting the is-a relation among the associated medical concepts. The second approach involved transformer-based models like BERT(Bidirectional Encoder Representations from Transformers), introduced by Devlin et al. [1], which dramatically advanced various natural language processing (NLP) tasks by offering deeply contextualized word embeddings. And RoBERTa(Robustly optimized BERT approach)[2], both BERT and RoBERTa models have significantly advanced the field of NLP. This work uses them as encoders connected to a classifier head to predict is-a or non-is-a categories. We fine-tune our models using a historical SNOMED CT dataset from 2017, while evaluation is carried out on new data introduced after 2023 in the 2024 release. With a recall of 0.933, precision of 0.933, F1 score of 0.932, accuracy of 0.933, and ROC of 0.972, RoBERTa outperforms the other baseline models (Support Vector Machine(SVM), K-Nearest Neighbors(KNN), Naive Bayes, and Multilayer Perceptron(MLP)) across all metrics. This work demonstrates that our implementation can effectively identify unknown is-a relations in future datasets.