Usage of Chatbot Techniques for Automation of Knowledge Tests

AI chatbot is often used for user support. It helps end-users in specialized tasks, diminishing the need for human assistance in support centers. Two mechanisms are used for chatbot implementation: RAG (Retrieval Augmented Generation) and LLM (Large Language Model) fine-tuning. RAG mechanism employs a prompt for serving relevant context, selected from the vector database to the LLM with the user query. After that, LLM constructs informative replies and returns them to the end-user. It is important to note that LLM has not been trained on significant materials in the area where the chatbot is constructed. Similar procedures can be used to automate knowledge tests. When the student answers the question relevant documents are retrieved from a specialized vector database with teaching materials and serve as a context with the student's answer to LLM through the prompt. LLM decides if the answer is correct based on the context and the student's answer. The procedures of vector database creation, test creation, and student answering can be automated, so that minimal interaction is needed from the teacher. In the paper, we explain software architecture and the implementation of such a system, with some recommendations. At the end of the paper, we present the proof-of-concept and directions for future development of the tool.