AI-supported Metadata Annotation of PILOT Data Sets

All research data produced by TRR 359, Perinatal Development of Immune Cell Topology (PILOT), is kept in an integrated data storage and dissemination system. Its central component is the Freiburg Research Data Tool (fredato), providing native data storage besides linking to external data storage sites. Ultimately, it will be possible to search and find all data – wherever stored – through fredato and obtain information about data access. To this end, all datasets and publications need to be annotated with metadata, i. e. information describing their contents and the methods employed to produce them. Data annotation is a joint effort by all PILOT participants, coordinated by the Information infrastructure for research data management (INF) project which is part of PILOT's project area C, central projects. Currently, a metadata schema is developed in collaboration with PILOT members. It covers the organisms and organs targeted by experiments, sample preparation and processing, interventions and machine readouts. If possible, annotation keywords are taken from well-established domain ontologies, e. g. Medical Subject Headings (MESH), the Data Analysis and Management Ontology (EDAM) for concepts in bioinformatics and the Experimental Factor Ontology (EFO). We present the current state of data schema development. The INF project uses annotation of PILOT outcomes to explore the potential of metadata annotation prediction based on Large Language Models (LLMs). We employ a multistep iterative approach between a LLM for full-text evaluation, the PubTator 3.0 database and the predefined metadata schema. Cross-checking using PubTator 3.0 enables us to restrict findings by the LLM to scientifically grounded concepts. The two-part approach makes a distinction between schema-related and non-schema-related predictions. Comparison of the two allows us to visualize the practical limitations of the schema-concept for facilitating metadata annotation within a TRR. Finally, predicted annotation terms will be presented to PILOT researchers for re-examination in order to calculate the precision of the chosen approach.