SiJesi: Large Language Model Chatbot with Augmented Retrieval Approach Generation and Prompt Engineering

This research designs a chatbot based on Large Language Models (LLM) for cyber incident consultation, utilizing the Retrieval Augmented Generation (RAG) approach, integrated with GPT and prompt engineering techniques. The goal is to improve public cybersecurity awareness, especially in areas with low awareness, by enhancing community consultation services through automation. The chatbot addresses security concerns by examining measures against prompt injection attacks to ensure reliability. The research follows the Waterfall System Development Life Cycle methodology. Results show that LLM chatbots using RAG and prompt engineering generate more accurate responses compared to human-based answers and GPT 3.5, achieving a top score of 35.26% in A/B testing. Additionally, the chatbot received a high satisfaction score of 86.47<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">%</sup> in User Acceptance Testing. These findings are expected to contribute to increased cybersecurity awareness and the development of innovative, secure chatbot technology for public services.