TransMoE: Multimodal traffic prediction with large language model and mixture of experts

Traffic prediction plays a crucial role in intelligent transportation systems, especially given the challenges posed by complex spatiotemporal dynamics and the integration of multimodal data. To address this, our work introduces TransMoE, an innovative framework that leverages Large Language Models (LLMs) in combination with a Mixture-of-Experts (MoE) architecture to forecast traffic from heterogeneous data sources. Specifically, we propose a novel adapter, MixLoHa, designed for parameter-efficient fine-tuning and capable of modeling spatially heterogeneous contextual patterns and temporal dynamics across multiple modalities for grid-wise traffic forecasting. Using real-world datasets that incorporate social media streams, real estate information, and points of interest, TransMoE demonstrates significantly improved accuracy and robustness compared to existing methods, achieving a substantial reduction in Mean Absolute Percentage Error. This research underscores the potential of multimodal fusion technologies in advancing traffic prediction, offering new possibilities for real-time urban traffic management.