FAIR Data at Germany's Largest Supercomputing Centres - A Building Block for the NFDI

The "InHPC-DE" project for the integration of the three largest German supercomputing centres (JSC – Jülich, HLRS – Stuttgart, LRZ – Garching near Munich) has reinforced its work (in WP2) towards homogeneous FAIR Research Data Management (RDM) standards at the centres. Data publication in the supercomputing sector has traditionally been a challenge, as typical lighthouse projects generate outputs too large for research data repositories. Data publication via special portals, running on top of the infrastructure where the data has been stored, solves this problem – but only few projects have the manpower to implement this sustainably on their own. Therefore, the three centres – working together in InHPC-DE in the context of the Gauss Centre for Supercomputing e.V. (GCS) – have been aiming at a unified approach and prototype service to make supercomputing results FAIR. During the past few years, the focus has been (i) on a unified, domain-agnostic (DataCite-based) metadata format for storing metadata as "sidecar files" with datasets, and (ii) on facilitating manual and semi-automatic creation (via a input-mask portal and a crawler) of these sidecar files. The presence of these files with datasets guarantees that metadata are automatically stored or archived together with the data. In the current funding period of InHPC-DE, we facilitate pushing such metadata to research data portals, generating landing pages and minting persistent identifiers (PIDs, e.g. DOIs following ISO 26324:2025). In particular, we aim at supporting the portal frameworks InvenioRDM, EUDAT-B2SHARE and Dataverse – which in this case will act as a pure presentation layer, while the data will remain on the supercomputing storage/archive systems and be accessible through a link from the portal. Such a link will utilise file-sharing and file-publication mechanisms made available within GCS/InHPC-DE (e.g. gridftp, uftp, GLOBUS). With the combination of file-retrieval mechanisms, landing pages and PIDs, a full basic "FAIR layer" is implemented in the supercomputing environment. The FAIR layer is developed and evolved in the context of our various collaborations between GCS centres and NFDI consortia, so that special add-ons can be realised (e.g. metadata according to domain-specific ontologies, registration of datasets in domain-specific registries). Semi-automated metadata crawling has been implemented with help of NFDI4Ing, and registration to a domain-specific registry is planned with PUNCH4NFDI. Also, harvesting of relevant (meta-)datasets from a data-portal prototype at LRZ (based on InvenioRDM) into Earth Data and the NFDI4Earth services has been facilitated. We are presenting our approach and looking forward to collaboratively maturing and automating our dataset-publication mechanism in the best way possible. This will hopefully help to encourage larger and larger parts of the High-Performance Computing community to consider comprehensive FAIR dataset publication. With our general-purpose/domain-agnostic basic strategy, we enable FAIR data in general supercomputing wherever no other supply is available. Domain-specific modifications of the approach can be developed with NFDI consortia as sketched. Thus, we aim to significantly contribute to a best-practice "FAIR supercomputing" methodology in German and European context.