Local Data, Global Reach: The Aruna Engine and the Future of Scalable Fully Federated Research Data Ecosystems

Research data infrastructure faces increasing complexity with distributed environments, very large and diverse data types, and cross-disciplinary collaboration requirements. Studies show researchers spend up to 80 % of time on data management rather than analysis [1], with significant barriers to data sharing across institutions [2]. The upcoming release of our Aruna Data Orchestration Engine addresses these issues by introducing a new federation-first approach that completely eliminates the need for central coordination. As a result, institutions can maintain full control over their data while participating in the broader research data ecosystem, an approach in line with the recommendations of the Research Data Alliance for distributed data management [3]. To accomplish this, the system makes use of peer-to-peer network technologies enabling seamless resource discovery across organizational boundaries while preserving institutional autonomy. Conflict-free replication mechanisms are used to synchronize metadata, eliminating the need for complex consensus mechanisms and allowing both local and remote instances to participate equally. The system efficiently handles large volumes of data, both in terms of file size and quantity, while maintaining performance and integrity across distributed environments, aligning with the Base4NFDI White Paper's discussion of potential NFDI services that could complement EOSC [4]. Governance plays a pivotal role in the effective functioning of distributed research data ecosystems, establishing the foundation for trust, compliance, and interoperability across institutional boundaries [5]. Our multi-tiered governance structure organizes instances of common domains into "realms" that adhere to consistent rules. These rules standardize critical aspects of data management, including metadata requirements, specifications, format constraints, and validation methodologies, ensuring data quality and usability throughout the entire network. By integrating compute considerations directly into the platform, Aruna can optimize research workflows by tracking data locality across its federated network. This allows the system to make intelligent decisions about where to perform computations. For large datasets, it will attempt to place a containerized compute job as close to the storage nodes as possible. For smaller datasets, it efficiently moves data to appropriate compute resources. All while complying with technical, regulatory, and governance requirements and policies. The Aruna system is designed to integrate with a wide range of existing data storage solutions, extending their capabilities while preserving established infrastructures. Using open standards such as the GA4GH endorsed Data Repository Service (DRS, O'Connor et al. [6]) and Task Execution Service (TES) and parts of the Gaia-X DataSpaces protocol, the platform interfaces with established research ecosystems, including national and international initiatives such as NFDI, ELIXIR or Gaia-X. This commitment to open standards and community-driven development positions Aruna as both a technical answer to data management challenges and a unifying element within the fragmented landscape of research data infrastructure. As a core service layer, Aruna provides the foundation upon which more sophisticated data platforms and domain-specific solutions can be built, abstracting away most of the technical challenges of deploying and maintaining a highly scalable and fully federated distributed system. By providing a low barrier to entry and integration with existing systems, it is possible to make storage infrastructures ready for federated access with minimal configuration effort.