An Analysis of Research Data Storage Systems

The scientific protocols, experiments and instruments that generate data are an integral part of the research lifecycle. Consequently, almost every scientific research institution requires a Research Data Storage System (RDSS). However, RDSS implementations vary significantly due to factors that include cost, geography, workloads, policy, risk tolerance and available technical skills. A RDSS may be on premises, in the public cloud or a mixture of both. Previously we identified 10 key high level features of a RDSS and defined an abstract high-level Research Data Reference Architecture (RDRA). Together, these features enable data fabrics, creating repeatable and consistent structures for low friction, highly efficient data movement and near real time data analysis for decision making and scientific workflows. We build on this earlier work in this paper to present a new Research Data Implementation Architecture (RDIA) that meets the RDRA and can guide implementations without locking in any specific product or service. This paper presents and compares six significant RDSSs and shows how they meet both the RDIA and therefore the RDRA. We identify a new structure – the Research Data Storage System Aggregator (RDSS-A) that describes clusters of RDSSs, and survey five such instances. Finally, we provide a real-world validation of the RDIA by documenting the technical specification of a real RDSS. The new work both clarifies a complex landscape, and aids groups building new systems or adopting existing systems.