Secure and Efficient RDMA NIC Cryptography Offloading for Memory Disaggregation

Confidentiality and integrity are essential for secure communication. In RDMA-based systems, however, they are often neglected because security mechanisms impose significant performance overheads. In this work, we design a secure RDMA data path that ensures confidentiality and integrity by protecting data in transit and at the remote side using encryption and checksums. To reduce security overhead, our system offloads cryptographic operations to the RDMA NIC (RNIC). However, hardware limitations and relatively slow cryptography performance in RNICs make a secure and efficient design challenging. We address these challenges with three key techniques: dynamic key reconfiguration, configuration-data batching, and context pooling. We also conduct a case study with RDMA memory disaggregation systems to evaluate when offloading is beneficial. We find that naive offloading degrades performance and that overlapping CPU's computation with RNIC's cryptography is essential to realizing its benefits. Our evaluation shows that overlapping achieves up to 9.63x lower P99.9 latency than a CPU-based secure RDMA data path on RocksDB.