Adapting General-Purpose Embedding Models to Private Datasets Using Keyword-based Retrieval

Text embedding models play a cornerstone role in AI applications, such as retrieval-augmented generation (RAG).While general-purpose text embedding models demonstrate strong performance on generic retrieval benchmarks, their effectiveness diminishes when applied to private datasets (e.g., company-specific proprietary data), which often contain specialized terminology and lingo.In this work, we introduce BMEmbed, a novel method for adapting general-purpose text embedding models to private datasets.By leveraging the wellestablished keyword-based retrieval technique (BM25), we construct supervisory signals from the ranking of keyword-based retrieval results to facilitate model adaptation.We evaluate BMEmbed across a range of domains, datasets, and models, showing consistent improvements in retrieval performance.Moreover, we provide empirical insights into how BM25-based signals contribute to improving embeddings by fostering alignment and uniformity, highlighting the value of this approach in adapting models to domain-specific data.We release the source code 1 for the research community.