Choreographing CAV Formations on the Fly with Reflection

Abstract Formation control of connected and automated vehicles (CAVs) has shown great potential for improving traffic efficiency and safety. However, most existing studies focus on tracking predefined formations for fixed tasks, offering limited adaptability to changing task requirements. To address this limitation, we propose GRF-TAFC, a Grounded and Reflective Framework for Task-Adaptive Formation Choreography of CAVs. GRF-TAFC integrates context-grounded formation generation, optimization-based execution, and execution-driven reflection into a closed loop. A formation generator elicits the formation generation capabilities of pretrained MLLMs and grounds their reasoning over unstructured task inputs in real-time tool outputs, domain knowledge, and experience memory, yielding formations tailored to current task requirements while respecting environmental constraints. An execution module realizes the generated formations through polynomial-time CAV-to-target assignment and joint spatio-temporal trajectory optimization, facilitating collision avoidance and safe execution. A reflection module reasons over task context and actual execution outcomes, distilling reusable experience for iterative refinement. Real-world experiments across dynamic tasks and environments demonstrate GRF-TAFC’s effectiveness and practicality, with experimental videos available on the project website: https://dongjh20.github.io/GRF-TAFC.