SEEBot: Leveraging Open-Source LLMs and RAG for Secure and Economical Enterprise Chatbots

In this research, we introduce SEEBot, a secure and economical enterprise chatbot designed for local deployment within organizations to address data privacy and security concerns. SEEBot leverages a PrivateGPT architecture, integrating open-source large language models (LLMs) with retrieval-augmented generation (RAG) for enhanced performance. We evaluated three open-source LLMs including: Llama2 7B, Llama2 13B, and Mistral 7B, using a benchmark dataset of 44 questions from the Paul Graham Dataset, assessed through LlamaIndex. Key performance metrics include faithfulness, relevance, and inference time. Additionally, we compare these models with GPT-3.5 Turbo, using two embedding models: (i) open source BAAI General Embedding (BGE), and (ii) paid OpenAI's text-embedding-ada-002. Experimental results indicate that Mistral 7B outperforms Llama2 7B, Llama2 13B, and GPT-3.5 Turbo in faithfulness and relevance, while GPT-3.5 Turbo achieves faster inference times. In addition, the performance of Mistral 7B was nearly same using both the OpenAI's and BAAI's embedding models. Moreover, our cost analysis demonstrates that deploying SEEBot reduces annual expenses by nearly 45% compared to cloud-based solutions. These findings support SEE-Bot as a viable, cost-effective, and privacy-conscious solution for enterprise chatbot deployment within secure, locally controlled infrastructures.