From Manual to Automated Prompt Engineering: Evolving LLM Prompts with Genetic Algorithms

Large Language Models (LLMs) have proven to be one of the most innovative and capable techniques in modern Artificial Intelligence (AI). However, the efficacy and consistency of LLMs still rely on the structure and quality of prompts (i.e., their input instructions). In real-world applications, the need for specialized prompt engineering, qualified staff, and domain knowledge often results in suboptimal performance and increased development costs. This paper explores the application of Genetic Algorithms (GAs) in generating and optimizing prompts for LLMs, with the aim of enhancing their performance across various tasks tackled by diversely skilled teams. Here, we customize a standard GA implementation to work with textual individuals, which are initialized and manipulated by LLM-defined crossover and mutation operators to evolve candidate prompts. This approach enables the automated discovery of LLM-based solutions across a broad range of problems with minimal human effort. This process allows LLMs to achieve optimized outputs that are comparable to or even superior to those obtained through expert, costly labor. We test our approach on four modern LLMs by OpenAI, Meta, and Mistral, and four domain-specific problems. Our results demonstrate quantitative improvements in both accuracy and efficiency compared to human prompt engineers. Achieving an average accuracy 24% higher than that of manual crafted prompts, our results also prove the viability and potential of autonomous optimization tools to discover high-performing prompt configurations, reducing the need for extensive human intervention and trial-and-error methods.