GMR-229 Semantic Search using Sentence Transformers

Traditional keyword-based search engines struggle to accurately capture the semantics of user queries in today's enormous digital resources. Our research study focuses on creating a semantic search engine that uses Sentence Transformers to improve information retrieval by understanding the context of queries and documents. Our method creates sentence embeddings for documents and user queries, allowing retrieval based on semantic similarity rather than keyword matching. The project involves data collection and preprocessing, feature extraction with Sentence Transformers, and implementation of a search engine that ranks documents based on cosine similarity to query embeddings. According to preliminary testing, this method greatly improves search relevancy and accuracy, we have compared the results with a baseline algorithm, BM25, to assess the effectiveness of Sentence Transformers in enhancing retrieval relevance. This work opens the door for future refinements in retrieval systems based on natural language processing and shows how semantic search engines can deliver results that are more contextually aligned.