Interacting with vector databases by means of domain-specific language

Abstract Vector database management systems have been recognized as a crucial innovation in the era dominated by artificial intelligence, where vast and high-dimensional datasets are generated at unprecedented scales. These systems are designed to efficiently handle, store, retrieve, and analyze high-dimensional vector data, while uncovering patterns within unstructured and heterogeneous datasets. The ability of vector database management systems to perform fast and accurate similarity searches allows contextual data retrieval. Access to vector databases is often facilitated through application code tailored to proprietary application programming interfaces and query languages, varying in syntax and terminology used among vector database management systems of different vendors. A state of tight coupling, interoperability challenges, and difficulties during transitions between vector database management systems that ultimately affect usability is thereby produced. To address these issues, we propose a model-driven software development solution that incorporates vecDSL – a domain-specific language serving as its central component – to provide a uniform approach to accessing vector databases. The goal of the proposed solution is to have vector database management simplified and its interactions streamlined, thereby ensuring that end-user efficiency is enhanced through the utilization of vecDSL. Concepts uniformly used in vecDSL are expected to ease learning and eliminate database-specific adjustments, while abstraction provided by the language aims to simplify testing and enable efficient performance assessments across different vector database management systems. In this article, we describe the syntax and usage of vecDSL, as well as the application of the MDSD-based solution in supporting the interaction with diverse vector databases. We also include the evaluation of the proposed vecDSL syntax, to examine its ability in addressing current issues and explore its potential for further development.