Dynamic Clustering and Anomaly Detection of Train Delays in Stream Data: An Incremental Dirichlet Process Approach

Effective delay management is essential for enhancing high-speed railway (HSR) operational efficiency. Clustering methods, a core technique for identifying delay patterns, categorize trains with similar operational characteristics. However, existing studies rarely consider anomalies, which are significant deviations from typical train operational characteristics and often indicate faults or infrastructure weaknesses. Detecting and analyzing these anomalous delays is therefore crucial. As delay data continuously accumulate, traditional clustering methods require repeated re-clustering of dynamically changing datasets, making them time-consuming and labor-intensive. To efficiently cluster and detect anomalies in train delay data, this paper proposes an Incremental Dirichlet Process Mixture Model (IDPMM)-based anomaly detection method. This approach integrates posterior probability-based anomaly detection with density-based checks to achieve real-time identification of anomalous delays. Subsequently, adaptive clustering is performed on each batch of non-anomalous data, merging similar clusters to incrementally refine the clustering model. Meanwhile, the clustering structure is dynamically adjusted to enhance model robustness. Experimental validation on multiple public datasets and real-world train operation data demonstrates that, compared to the traditional Dirichlet Process Mixture Model (DPMM), the proposed IDPMM method reduces processing time by 65.3%-87.7% and memory consumption by 80%-95%. Moreover, among evaluated incremental clustering methods, IDPMM achieves the highest anomaly detection performance. In real-world applications, IDPMM facilitates real-time anomaly detection, enabling early identification of anomalous trains at the onset of delay propagation. This capability supports timely dispatch interventions, mitigating cascading delays and improving train operation stability and scheduling efficiency.