Poster: Vortex: Efficient Decentralized Vector Overlay for Similarity Search and Delivery
Nearest-neighbor search over embeddings has become a core primitive for AI and LLM-centric workloads. However, prevailing vector databases remain centralized or cluster-bound, introducing single control points, privacy vulnerabilities, and cost/latency bottlenecks. We present Vortex, a decentralized vector overlay that delivers planet-scale approximate nearest neighbor (ANN) search without a centralized control plane. Vortex integrates three key components: (1) Distributed Learned Hashing (DLH), which collaboratively learns piecewise similarity-preserving hash functions to map semantically related vectors to nearby key ranges while balancing load; (2) a Distributed Hash Table (DHT) for scalable, fault-tolerant routing and churn resilience; and (3) a co-designed Distributed HNSW (DHNSW) index for high-recall, low-latency search on each peer. Preliminary results show that Vortex matches the accuracy and latency of leading centralized systems while reducing per-peer index memory requirements by two orders of magnitude and eliminating any central coordinator—enabling fully decentralized, self-organizing ANN overlay for next-generation AI systems.
Authors
- Yi Liu (ORCID: https://orcid.org/0000-0002-8839-9508)
- Shengze Wang (ORCID: https://orcid.org/0000-0001-9496-158X)
- Chen Qian (ORCID: https://orcid.org/0000-0002-6882-9590)
Institutions
- University of California, Santa Cruz (US)
Publication Details
- Published
- 2025-09-22
- DOI
- https://doi.org/10.1109/icnp65844.2025.11192399
- Primary Topic
- Time Series Analysis and Forecasting
- Type
- article
- Field-Weighted Citation Impact
- 0.00