A Framework for Security Testing of Large Language Models

The purpose of this paper is to present a framework for testing of large language models (LLMs) for security vulnerabilities before their implementation to production environment. The paper discusses the latest developments in the Artificial Intelligence (AI) and Generative Artificial Intelligence (Generative AI) adoption in the industry, the expectations for further accelerated adoption and evolving regulatory landscape. An overview of the most significant risks and vulnerabilities of the LLMs such as prompt injection and denial of service have been presented with their mitigation strategies. A testing approach and testing framework have been developed and implemented with simple chatbot app. The test scenarios have been executed and results have been obtained for three open-source LLMs from which two pass the test and one failed and demonstrated the application of the proposed testing framework. Source code of the application and test script are published open source for reproducibility and reuse. In conclusion the with the confirmation of the results the limitation of the reliance on semantic similarity for the responses of the models was discussed together with three areas for further development: expanding the test scenarios to significant risks, integration with popular cloud continuous development platforms and integrating blockchain for transparent publication of the final test results.