TransitSim-LLM: a closed-loop framework for GTFS-driven transit service evaluation using open-source LLMs and SUMO microsimulation

Introduction Resource-constrained transit agencies face challenges in translating service data into actionable interventions and evaluating their operational consequences. Methods This study develops TransitSim-LLM, an integrated framework combining General Transit Feed Specification (GTFS) diagnostics, large language model (LLM) reasoning, and Simulation of Urban Mobility (SUMO) microsimulation. The framework integrates seven service diagnostics, issue prioritization, validated intervention generation, comparative simulation, and planning interpretation. It was evaluated using the MyBas network in Johor Bahru, Malaysia. Results The analysis covered 21 routes and 271 stops, identifying four principal service concerns. Two intervention scenarios satisfied five predefined technical validation criteria. The frequency-enhancement scenario increased completed vehicle trips by 5.2%, reduced the reported waiting-time indicator by 6.2%, and increased vehicle kilometers by 5.8%. Benchmarking four openly available LLMs yielded diagnostic quality scores of 85/100 for three Llama models. LLM-generated scenarios achieved higher rubric-based quality scores (80-86) than rule-based (43) and random (25) alternatives. Discussion TransitSim-LLM demonstrates the potential of integrating structured transit diagnostics, LLM-assisted intervention generation, and microsimulation for evidence-informed transit planning. However, observed passenger-demand data and operational feasibility assessments remain necessary before real-world implementation.