Simulation under Stress: A Comparative Benchmarking of Large-Scale Traffic Simulators

Modern urban mobility systems are increasingly dependent on precise and scalable simulation platforms to facilitate the design, assessment, and optimization of intelligent transportation systems (ITS). This paper sets forth a thorough benchmarking investigation of three extensively utilized traffic simulation platforms: SUMO (Simulation of Urban MObility), CityFlow, and MATSim, with an emphasis on their computational efficacy within large-scale synthetic traffic scenarios. We conduct an evaluation of these platforms across vehicle volumes ranging from 10 to 10 million vehicles, assessing total runtime, CPU and GPU utilization, and memory consumption on high-performance computing infrastructures. Our results reveal notable architectural trade-offs: SUMO exhibits predictable linear scaling but becomes constrained by CPU limitations at elevated vehicle counts, CityFlow encounters memory limitations beyond 10,000 vehicles, whereas MATSim necessitates meticulous JVM (Java Virtual Machine) optimization to effectively manage large-scale scenarios. This investigation provides essential guidance for researchers and practitioners in the selection of suitable simulation tools for urban-scale traffic modeling and identifies critical computational challenges associated with the scaling of simulations for smart city initiatives.