Hybrid Query of Boolean Filter and Vector Similarity Search: Benchmark, Comparison and Direction

The growing use of vectors for unstructured data has made efficient hybrid queries-combining boolean filters with vector similarity searches-essential. However, publicly available datasets for evaluating DBMS performance on such queries are limited. To address this, we introduce two datasets: the singletable fungi dataset (STFD) with 295,938 base entities and the multi-table movie dataset (MTMD) with 284,713 base vectors, available at this link. These datasets provide a framework for evaluating DBMS performance on hybrid queries. Our experiments with Milvus, pgvector, ClickHouse, and Elasticsearch show that current DBMSs struggle with hybrid queries, each system exhibiting strengths and weaknesses. QPS varies up to <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$50 \times$</tex>, and recall differences reach 55 %. These results underscore the importance of query optimization and index utilization. We outline future research directions, including specialized data structures, algorithms for hybrid queries, techniques for estimating query difficulty and predicate selectivity, and enhanced query optimization strategies, to support the development of more efficient DBMS solutions.