Ceph at CERN in the multi-datacentre era

The recent commissioning of CERN’s Prévessin Data Centre (PDC) brings with it the opportunity for multi-data-centre Ceph deployments which in turn have advantages for Business Continuity and Disaster Recovery. However, the simple extension of a single cluster across data centres is impractical due to the impact of latency on Ceph’s strong consistency requirements. Dedicated techniques for such an environment are thus required. This paper reports on our research towards building a multi-datacentre Ceph deployment in production. Due to different transaction semantics for blocks, objects and files, geo-distributing a Ceph cluster needs a different approach for each protocol in use. We here detail the challenges with Ceph across data centres, the various solutions we evaluated and a road-map for the future at CERN.