Literature Review on Concept Drift in High-Dimensional Data Streams: Challenges and Detection Methods

Concept drift, the phenomenon where the underlying data distribution changes over time, presents significant challenges for machine learning models deployed in realtime applications. This issue is particularly pronounced in high-dimensional data streams, where the complexity of monitoring and adapting to these changes can lead to decreased model performance and reliability. This literature review examines existing research on concept drift in high-dimensional data environments, exploring the causes, detection methods and techniques for adaptation. We delve into classical statistical approaches, machine learning and deep learning-based methods, and discuss the inherent challenges posed by highdimensionality. Additionally, in this paper highlight evaluation metrics, benchmark datasets, and a comparative analysis of the strengths and weaknesses of current techniques. This paper concludes with potential future research directions, emphasizing the importance of scalable, adaptive and hybrid approaches to tackle concept drift effectively in high-dimensional data streams.