Large Language Models as Particle Swarm Optimizers

Recently, several approaches have gained attention by integrating large language models (LLMs) into evolutionary algorithms. Building on this trend, we introduce Language Model Particle Swarm Optimization (LMPSO), a novel method that incorporates an LLM into the swarm intelligence framework of Particle Swarm Optimization (PSO). In LMPSO, the velocity of each particle is defined as a part of the prompt that generates the next candidate solution, leveraging an LLM to produce solutions while respecting the PSO paradigm. This integration enables an LLM-driven search process that adheres to the foundational principles of PSO. We evaluate LMPSO on the Traveling Salesman Problem (TSP) and on a heuristic improvement task for TSP, where solutions are represented as program code. Heuristic improvement aims to enhance existing heuristics by searching for new ones; it is challenging for standard PSO due to the solution representation in the form of program code. Experimental results demonstrate that LMPSO can generate high-quality solutions for small TSP instances and improve existing TSP heuristics while following the PSO search framework. By incorporating LLMs into PSO, LMPSO expands the applicability of swarm intelligence and highlights the potential of LLMs for addressing complex optimization challenges.