Deploying DataFed for Scientific Data Management: Lessons Learned

This Developer Presentation shares our experience using DataFed, a novel scientific data management system, in research projects at Drexel University. We will address three main questions: What is DataFed, and why might Open Repositories participants find it useful? What challenges have we faced in deploying DataFed endpoints, and how did we overcome those? What should researchers know about the technical requirements and process of implementing DataFed in the university setting? DataFed, developed on an open-source basis at Oak Ridge National Laboratory (ORNL), is a distributed system for managing research data and metadata. It simplifies handling large datasets in alignment with the FAIR principles, providing researchers with scalable storage, a customizable metadata infrastructure, and optional federated sharing capabilities. Deploying a DataFed repository presents certain challenges—for example, a server with a public static IP address and open ports for ingress, which institutional network administrators may be reluctant to allow. Additionally, DataFed depends on Globus Connect Server for secure data transfer, introducing further complexities. Despite such obstacles, we have found that the benefits of DataFed more than justify the effort. We are also collaborating with ORNL to streamline the deployment process for future users.