Implementing Content-Defined Chunking for Deduplication in Host-Managed SSDs

Although conventional solid-state drives (SSDs) are widely utilized in enterprise storage solutions because of their superior I/O performance, SSDs still face serious challenges such as limited lifespan and write amplification because of the inherent limitation of flash memory. To address these issues, data deduplication has been used to discard redundant writing data by generating and checking the unique fingerprint of each data chunk. The traditional deduplication strategy for SSDs is to identify replicate data chunks in the fixed-size unit of the flash memory page. However, the page size increases with the advancement of flash memory technology, which makes it harder to find duplicate data. To enhance deduplication efficiency, deduplication with content-defined chunking (CDC) technique has been proposed and applied in various storage mediums such as hard disk drives and phase-change memory. However, deploying CDC-based deduplication in SSDs is not straightforward due to the various sizes of chunks, which brings the challenge of fingerprint management and data misalignment. This paper explores the integration of CDC-based deduplication with host-managed SSDs (e.g., OCSSD). By exposing detailed features of flash memory to the host, we introduce CDC-aware Fingerprint Store and Space Management to support CDCbased deduplication on host-based SSDs. Our approach not only boosts overall deduplication efficiency but also reduces the space inefficiencies associated with varying chunk sizes. Our experiments show that this approach boosts the deduplication rate by up to 30.35% and enhances write performance by 32.8% compared to the traditional method.