The RECUR corpus: annotated German radiology reports for urolithiasis research
Background Advances in clinical NLP depend on high-quality datasets that reflect the complexity of real-world documentation. In response, a fine-grained, relation-annotated corpus for urinary stone diagnostics is introduced. The corpus addresses a gap in German medical NLP resources and provides a reference dataset for developing and evaluating clinical information extraction models. Methods We extracted and de-identified 262 radiology reports and 251 corresponding short findings concerning kidney and ureter stones from the hospital information system of the University Medical Center Freiburg. Using a detailed annotation schema comprising nine entity types and five relation types, three medical doctoral students annotated the corpus in Label Studio, followed by expert adjudication to establish a gold standard. For evaluation, we fine-tuned several German general-domain and medical language models for named entity recognition (NER) and relation extraction (RE) using a fixed train–development–test split, a unified Hugging Face pipeline, and NNI-based hyperparameter tuning. Results The resulting corpus comprises 513 radiology documents with 3,223 annotated entities, dominated by stone- and anatomy-related categories. Inter-annotator agreement was consistently high for both entity and relation annotations (NER F1 = 82.8%; RE F1 = 81.8%), supporting the reliability of the annotation schema. The evaluated models achieved macro F1-scores of up to 89.45% for NER and 99.24% for RE. While NER performance varied across models and fine-grained entity categories, RE performance was consistently high across all evaluated encoders. Conclusion RECUR establishes a fine-grained reference corpus and evaluation setting for automated information extraction from German radiology reports. Its adjudicated annotations, strong inter-annotator agreement, and comprehensive encoder baselines support the systematic evaluation of clinical NER and RE methods.
Authors
- Tabea Walther (ORCID: https://orcid.org/0000-0001-7152-5689)
- Luise Modersohn (ORCID: https://orcid.org/0000-0002-8258-4366)
- Martin Schoenthaler (ORCID: https://orcid.org/0000-0002-3783-4047)
- Maximilian Glienke (ORCID: https://orcid.org/0000-0002-6051-4404)
- Maximilian Ferry von Bargen
- Lidija Ujkic
- Raphael Schmitt
Institutions
- Universidade dos Açores (PT)
- University of Freiburg (DE)
- University Medical Center Freiburg (DE)
- TUM Klinikum (DE)
- Technical University of Munich (DE)
Publication Details
- Journal
- BMC Medical Informatics and Decision Making
- Published
- 2026-11-19
- DOI
- https://doi.org/10.1186/s12911-026-03982-4
- Primary Topic
- Biomedical Text Mining and Ontologies
- Type
- article
- Field-Weighted Citation Impact
- 0.00