Navigating (in)Security of AI-Generated Code

The increasing use of large language models (LLMs) such as OpenAI's ChatGPT and Google's Bard in the software development industry raise questions about the security of generated code. Our research evaluates Java, C, and Python code samples that were generated by these LLMs. In our investigation, we assessed the consistency of code samples generated by each LLM, characterized the security of generated code, and asked both LLMs to evaluate and fix the weaknesses of their own generated code as well as the code of the other LLM. Using 133 unique prompts from Google Code Jam competitions, we produced 3,854 code samples across three distinct programming languages. We found that the code produced by these LLMs is frequently insecure and prone to weaknesses and vulnerabilities. This concerns human developers who must exercise caution while employing these LLMs.