CHash: A High Cost-Performance Hash Design for CXL-based Disaggregated Memory System

The exponential growth in demand for high-performance computing systems has created an urgent requirement for innovative memory technologies that can provide higher bandwidth and enhanced capacity scalability. In particular, Compute Express Link (CXL) has emerged as a promising solution for memory expansion and system acceleration. Hash-based index structures are widely recognized as fundamental components of in-memory database systems, and they are commonly used for indexing in-memory key-value stores due to their capability for rapid lookup performance. How to design and maintain a hash index structure in a CXLbased disaggregated memory system, with comprehensive consolidation, is a challenging topic of in-depth research. Specifically, the architecture and characteristics of the CXL memory device are significantly distinct from those of DRAM and PMEM. Simply applying prior hashing approaches without considering the differences would not fully exploit the characteristics presented by CXL memory devices. This leads to the performance inefficiencies of conventional indexing designs and operation processes that overlook the characteristics of the CXL memory device. In this paper, we conduct extensive experiments on two real CXL memory devices based on the 4thgeneration Intel® Xeon® Scalable Processor with CXL 1.0 support. Specifically, we run various microbenchmarks not only to evaluate the performance characteristics of CXL memory devices, but also to measure the performance impact of different memory allocation methods. We observe significant performance disparities and allocation inefficiencies, motivating us to innovate the hash design in the CXL-based disaggregated memory system. Upon in-depth analysis, we propose CHash, as shown in Figure 1, which consists of several key designs, as shown below: Stage buckets for KV batching. The experimental results show that real CXL memory devices from different vendors exhibit diverse performance characteristics. Moreover, there is still a significant performance gap between CPU-attached DRAM and CXL memory. To address this difference, CHash first stores the KV records in a DRAM buffer, called the stage bucket in our design, to batch the KVs, avoiding frequent and small writes to CXL memory devices. SAcceleration techniques for fast lookups. Although memory access latencies for read and write operations are comparable, the end-to-end performance of reads is lower than that of writes due to multiple caching layers, including the CPU cache and the write buffer in the device's memory controller, shorten the write latency. In response, CHash implements several acceleration techniques designed to minimize costly key-value comparisons, thereby reducing the number of reads to CXL memory. Furthermore, CHash stores the metadata which is the most frequently accessed components of the hash table in DRAM to enhance the overall performance. Some alternative optimizations for the effective CXL memory management, including two-level filtering, hash multiplexing and so on, are also further adopted to improve the overall performance. Extensive experimental results validate the efficiency of CHash in different CXL memory devices, demonstrating that it achieves a 2.2× to 9.4× speedup for insertions, and a 1.2× to 4.5× speedup for lookups, all the while maintaining a high load factor (~90%), compared to state-of-the-art hashing schemes.