An Empirical Analysis of Embedded versus Dedicated Client-Server Architectures for High-Throughput Vector Retrieval-Augmented Generation Systems
(1) Background: The emergence of the Retrieval-Augmented Generation (RAG) family of approaches in enterprise AI has made vector similarity search one of the key infrastructure pillars, requiring a choice between an in-process storage engine and dedicated client-server vector database nodes. (2) Methods: We present a thorough empirical comparison between representative vector storage architectures including embedded storage (Chroma), lightweight client-server frameworks based on Rust (Qdrant), and cloud-native distributed databases (Milvus) across vector dimensions (384 to 1536), scale of datasets (up to 50k vectors), and multi-client concurrency queries (up to 100 virtual users). (3) Results: We found that embedded storage architectures perform well as a single-tenant solution, having low latency and memory footprint during startup, yet they experience significant throughput decrease in the presence of concurrent write/read activity. Client-server architectures provide better tail latency stability (\texorpdfstring{$P_{99}$}{P99}) and scalable throughput but introduce greater complexity and higher memory cost. (4) Conclusions: We summarize our results in a formal framework for selecting architecture by software engineers developing RAG backend solutions.
Authors
- Md. Zahid Hasan
Institutions
- Czech University of Life Sciences Prague (CZ)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-17
- DOI
- https://doi.org/10.5281/zenodo.24325410
- Primary Topic
- Advanced Database Systems and Queries
- Type
- preprint