A Novel Concept Drift Detection Model for Handling Evolving Patterns in Multivariate Time Series

Concept drift detection in Multivariate Time Series (MTS) data is a critical challenge in domains such as finance, IoT sensor monitoring, and cybersecurity, where data distributions evolve dynamically. The paper proposes a Hybrid Autoencoder-LSTM-ADWIN framework that integrates unsupervised learning, deep learning, and adaptive drift detection to improve forecasting accuracy in non-stationary environments. The framework begins with Z-score normalization and a sliding window approach to preprocess MTS data, ensuring robust feature extraction. An Autoencoder compresses spatial dependencies into a low-dimensional latent space, while a Long Short-Term Memory (LSTM) network captures temporal dependencies for predictive modeling. The ADWIN (Adaptive Windowing) algorithm continuously monitors residual errors between LSTM predictions and actual values to detect concept drift dynamically. When drift is identified, the model is incrementally updated, allowing adaptation to evolving patterns while reducing unnecessary retraining overhead. Experimental results demonstrate that the proposed approach outperforms baseline methods in terms of detection accuracy, computational efficiency, and adaptability to both sudden and gradual drifts. This hybrid methodology provides a scalable and effective solution for real-time drift detection and forecasting in dynamic MTS environments.