Retrieval Augmented Generation for Document Query Automation using Open source LLMs

Ollama provides access to powerful open-source language models that can be integrated into various applications. It supports local hosting, controlling the model's usage and data privacy. LLMs are large language models also known as deep learning models which are pre-trained on a vast amount of data. Integrating with retrieval augmented generation (RAG) can improve the efficiency of the LLM applications by retrieving custom data. The specific website or input of a particular document file can be integrated into the system. The document queries are automatically added to the Excel file using UiPath Automation. Firstly, the proposed system prompts by directly passing through the two LLM models: the Phi3 model by Microsoft with 3 billion parameters and the Llama 3.1 model by Meta with 8 billion parameters for text input and output, which can be accessed from Ollama released by Meta. To achieve a desired output, the system can also prompt by passing through retrieval augmented generation (RAG). Finally, analyze the results of the two outputs by directly using LLM models and embedding them with RAG. According to the evaluation result, RAG integration with llama3.1 improves response quality and relevance for custom data. Phi3 is better in the latency evaluation result.