<scp>JumpBackHash</scp>: Say Goodbye to the Modulo Operation to Distribute Keys Uniformly to Buckets

ABSTRACT Introduction Distributed data processing and storage systems require efficient methods to distribute keys across buckets. While simple and fast, the traditional modulo‐based mapping is unstable when the number of buckets changes, leading to spikes in system resource utilization, such as network or database requests. Consistent hash algorithms minimize remappings but are either significantly slower, require floating‐point arithmetic, or are based on a family of hash functions rarely available in standard libraries. This work introduces JumpBackHash, a consistent hash algorithm that overcomes those shortcomings. Methodology JumpBackHash applies the concept of active indices borrowed from consistent weighted sampling, which inherently leads to consistency. It generates the active indices in reverse order, which avoids floating‐point operations, enables the minimization of consumed random values and the use of a standard pseudorandom generator, and finally leads to a very efficient algorithm. Results Theoretical analysis shows that JumpBackHash has an expected constant runtime. The expected value and the variance of the number of consumed random values perfectly agree with the experiments. Empirical tests also confirm the consistency. Conclusion JumpBackHash offers a fast and efficient solution for uniformly distributing keys across buckets in distributed systems. Its simplicity, performance, and the availability of a production‐ready Java implementation as part of the Hash4j open source library make it a viable replacement for the modulo‐based approach for improving assignment and system stability.

Authors

Institutions

Publication Details

Journal
Software Practice and Experience
Published
2024-11-05
DOI
https://doi.org/10.1002/spe.3385
Citations
2
Primary Topic
Peer-to-Peer Network Technologies
Type
article
Field-Weighted Citation Impact
0.70
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

JumpBackHash: Say Goodbye to the Modulo Operation to Distribute Keys Uniformly to Buckets

Otmar Ertl
2 citations
Software Practice and Experience
Peer-to-Peer Network Technologies
0.70
article

JumpBackHash: Say Goodbye to the Modulo Operation to Distribute Keys Uniformly to Buckets

Otmar Ertl
article en
2 citations

Abstract

ABSTRACT Introduction Distributed data processing and storage systems require efficient methods to distribute keys across buckets. While simple and fast, the traditional modulo‐based mapping is unstable when the number of buckets changes, leading to spikes in system resource utilization, such as network or database requests. Consistent hash algorithms minimize remappings but are either significantly slower, require floating‐point arithmetic, or are based on a family of hash functions rarely available in standard libraries. This work introduces JumpBackHash, a consistent hash algorithm that overcomes those shortcomings. Methodology JumpBackHash applies the concept of active indices borrowed from consistent weighted sampling, which inherently leads to consistency. It generates the active indices in reverse order, which avoids floating‐point operations, enables the minimization of consumed random values and the use of a standard pseudorandom generator, and finally leads to a very efficient algorithm. Results Theoretical analysis shows that JumpBackHash has an expected constant runtime. The expected value and the variance of the number of consumed random values perfectly agree with the experiments. Empirical tests also confirm the consistency. Conclusion JumpBackHash offers a fast and efficient solution for uniformly distributing keys across buckets in distributed systems. Its simplicity, performance, and the availability of a production‐ready Java implementation as part of the Hash4j open source library make it a viable replacement for the modulo‐based approach for improving assignment and system stability.

Software Practice and ExperienceVol. 55(3)
Voestalpine (Austria) (AT)
Openalex Percentile: Top 27%
Peer-to-Peer Network Technologies
0.70
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.