A Multimodal Learning Approach for Protecting the Metro System of Medellin Colombia Against Corrupted User Traffic Data

A critical task in infrastructure security is to model user traffic in transportation systems to alert whenever anomalous behavior is observed. Discerning those abnormal samples is possible by auditing the available data, which then enables proper policy making to guarantee fair tariffs and the design of strategies to tackle problems such as passenger congestion. In this paper, we present an offline cybersecurity approach for the multimodal modeling of user traffic for the Colombian metro. To identify the anomalies, we design custom Deep Autoencoders based on the embeddings produced by the Self-Supervised Learning TabNet architecture. Additionally, we provide explainability through a SHAP-based component and the analysis of external image data using LLaVA as the selected Large Multimodal Model. The results indicate that most problems that occur on one metro line also affect the other, demonstrating the interconnectivity of the metro system, a crucial aspect that motivates the coordinated emergency response to improve the passenger travel experience. Although the detected problems might already have been identified and reported on social media, the transparency provided helps create confidence when an abnormality is observed, and in case there is no backup information on our official external data sources, it represents an alert to examine it more deeply, becoming an intelligent assessment tool for the metro. This article also sheds light on the potential of the publicly available dataset used and the importance of expanding its existing variables and information.