“Design and evaluation of public chatbots for computer crime awareness: A scoping review”

Abstract Chatbots are software applications that allow users to access information and services in various public administration fields. However, as chatbot technologies advance, only certain features meet the specific needs of the public sector, namely in the justice field and computer crime awareness. With a growing number of products available, careful evaluation is critical to determining which functionalities best support these requirements. By examining key characteristics and features, we intended to identify existing research gaps and enhance the design and evaluation of chatbots to increase their effectiveness and utility in the public sector, namely for computer crime awareness. Following the PRISMA-ScR guidelines for scoping reviews, relevant studies were identified in several digital library databases during April 2023. Of the 301 studies identified, 22 were included in the review. In addition to the Architecture for Public Service Chatbots guidelines, the findings revealed other key themes, including concepts influencing chatbot design, research methodologies shaping chatbot artefacts, technologies impacting chatbots, architectural frameworks, chatbot tool development, and methods in chatbot evaluation. The design and evaluation of public chatbots should account for ethical concerns, transparency, and fairness. Design Science Research was the methodology identified for developing chatbot artefacts. Deterministic, rule-based, and goal-oriented chatbots provide auditability, while cloud-based services and AI technologies offer security, scalability, and interoperability. There are external evaluation methods, such as user feedback, performance metrics, and usability. One of the principal areas of future research should prioritise the integration of AI and cloud computing in generative models, such as ChatGPT, with a focus on data protection and ethical considerations.