Privacy-Centric Offline Chatbot Using Large Language Models

From ELIZA in the 1960s mimicking a psychotherapist to ChatGPT helping us to write code, chatbots have come a long way and are dominating the digital era. Chatbots, which come under Conversational AI make it possible for humans and machines to converse. They are developed significantly through the latest advances in AI, becoming more complex and intelligent but some downfalls exist. Many current solutions depend on cloud-based models which need internet connectivity. They also might collect and store the data leading to privacy breaches. This research paper focuses on these problems. This proposed methodology equips locally managed Large Language Models (LLMs) using tools like Ollama and bge-m3 model for embedding. Operating offline ensures that no data is collected and shared and it can also work in remote places with no internet connectivity. This integrates Retrieval- Augmented Generation (RAG) to generate context- aware, accurate answers and it also employs vector indexing search for visually related images and files to extract information. Keywords: Artificial Intelligence (AI), Large Language Models (LLMs), Local Models, Offline Chatbot, Retrieval-Augmented Generation (RAG).