Principled Pathways Towards Autonomy and Emergence in Agents

This talk will cover recent advancements in automating drone swarm formations using foundation models. The first work, CLIPSwarm, leverages vision-language foundation models to generate drone formations that match natural language descriptions. CLIPSwarm refines simple robot formations based on CLIP similarity, enabling the generation of 2D shapes from single-word inputs. The second work, Gen-Swarms, introduces a novel framework that combines deep generative models with reactive navigation algorithms to automate the creation of 3D drone shows from text categories. By adapting flow matching models and incorporating motion constraints, Gen-Swarms generates smooth, collision-free trajectories for drones to form desired shapes. Together, these works demonstrate the power of foundation models to automate the artistic design and control of drone swarms, addressing challenges in high-level semantic control, trajectory generation, and collision avoidance.