A High-Speed FPGA Implementation for IVF-PQ Index Construction

The Inverted File with Product Quantization (IVF-PQ) is a widely used method for Approximate Nearest Neighbor Search (ANNS), playing a critical role in AI-driven applications such as search engines, recommendation systems, and advertising platforms. With the advent of Large Language Models (LLMs), the demand for efficient and real-time index construction has significantly increased, especially for edge-side personal applications. In this paper, we propose a scalable and high-speed FPGA implementation of IVF-PQ index construction, significantly reducing indexing latency and making it feasible for edge scenarios. First, we optimize the original index construction algorithm by introducing batch-mode centroid updates and replacing floating-point division with hardware-efficient operations, while maintaining competitive recall performance (with less than 5% degradation and up to 12.5% improvement compared to the original algorithm). Next, based on the modified algorithm, we design a flexible and scalable hardware architecture that supports two distance metrics (L2 and Inner Product), six PQ configurations, and input data with up to 1024 dimensions, all without necessitating hardware recompilation. Our implementation maximizes computational efficiency through finely tuned parallelism and dataflow, ensuring full pipeline utilization. Finally, we implement our design in Verilog and evaluate it on the Xilinx XCU280-FSVH2892-2L-E FPGA platform. Experimental results show that our accelerator achieves up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$30\times $</tex-math> </inline-formula> speedup over a high-end server CPU (Intel Xeon Gold 6248R), reducing the indexing time from hours to minutes.