Automated Phenotype Annotation in Spanish Clinical Text: A Large Language Model Approach with Retrieval-Augmented Generation

This work addresses the challenge of automatically annotating phenotypes from unstructured Spanish clinical text. Phenotype coding from free text is essential for standardizing health information and improving interoperability between health information systems. Currently, this task is often done manually by geneticists, increasing workload and delaying patient care. The objective of this work is to optimize the current annotation process by developping an Articial Intelligencebased platform that automates phenotype extraction and mapping. The system analyzes Spanish clinical notes, identifies relevant phenotypes, and links them to corresponding codes from the Human Phenotype Ontology (HPO). The proposed approach combines Large Language Model (LLM) with Retrieval Augmented Generation (RAG), as this method has shown high accuracy in similar tasks in English. Additional benefits include adaptability to ontology updates and easy scalability to other codification systems. The methodology followed in this work involves translating the HPO into Spanish, building a reference vector database, testing prompt strategies, and creating a user interface for the validation and annotation of clinical notes. The entire solution is built on an open-source, microservice-based architecture. In addition, the results of different RAG and LLMs configurations are compared, revealing their effect on accuracy and computational cost. Results show that the LLMs alone (i.e. OpenAI’s o4-mini) performs poorly (F1-score of 0.2), often generating incorrect or unrelated codes. Integrating RAG significantly improves results. The highest-performing configuration achieved reaches an F1-score of 0.64 (0.68 precision and 0.62 recall) on the RAGHPO dataset.