Change Detection in Partially Observed Large-Scale Traffic Network Data

Intelligent Transportation Systems generate an unprecedented amount of high-dimensional traffic data. The proper analysis of such data can transform traffic monitoring mechanisms. However, existing monitoring methods for detecting abrupt changes in traffic patterns have two limitations. First, they do not capture the spatiotemporal characteristics of traffic data and are not equipped with a built-in mechanism to handle missing observations. To address these limitations, this study proposes a dynamic, robust tensor completion method to monitor and detect changes in partially observed traffic data streams. The proposed method simultaneously completes and decomposes the partially observed data into a sum of a low-rank tensor that captures the spatiotemporal patterns and a sparse tensor that captures anomalies. Subsequently, the proposed method defines a statistic monitored by an exponentially weighted moving average control chart to detect abrupt temporal changes. The performance of the proposed method is evaluated by simulation and case studies. The simulation results indicate the proposed method outperforms all benchmarks. It can also detect changes more than twice as fast as other benchmarks in terms of average run length in most scenarios. The proposed method is also applied to the traffic data in New York City to evaluate its performance in detecting unusual traffic patterns when Hurricane Sandy hit the city. The experimental results demonstrated the superiority of the proposed method in quickly detecting unusual changes at both network and road segment levels. Particularly, the proposed method detects changes in traffic patterns approximately twelve hours earlier than the next best alternative benchmark method.