An LLM-Based Genetic Algorithm for Prompt Engineering

Large Language Models (LLMs) have gained recognition as valuable assets across virtually all industries, yet they rely heavily on manually crafted input prompts. In real-world applications, the dependence on specialized staff, skilled prompt engineering, and domain-specific knowledge often leads to suboptimal performance and increased costs. This paper investigates the use of Genetic Algorithms (GAs) to autonomously generate, evolve, and judge LLM prompts. Specifically, we customized a standard GA implementation to handle textual individuals, which are manipulated by LLM-guided genetic operators that iteratively create and enhance candidate prompts. Additionally, LLMs are employed to assess the correctness of outputs, forming the basis of our GA's fitness function. Our findings suggest that GA-driven prompt engineering can consistently produce solutions that are superior in both accuracy and efficiency compared to those acquired by prompt engineers who possess technical skills but lack domain-specific knowledge and a full understanding of vendor-specific prompting idiosyncrasies. This conclusion is supported by experimental results obtained from four public datasets and three modern LLMs developed by OpenAI, Meta, and Mistral AI. Ultimately, this study highlights the viability and potential of fully automated optimization tools in minimizing human effort in writing high-performance prompts.