Autonomous Data Mining Systems for Real-Time Big Data Streams in Edge AI: A Comprehensive Survey

The exponential growth of real-time big data streams generated by IoT devices, cyber-physical systems, and distributed sensors has necessitated the evolution of autonomous data mining systems capable of operating efficiently at the network edge. Traditional cloud-centric analytics architectures suffer from high latency, bandwidth overhead, and privacy risks, limiting their suitability for time-sensitive applications. This survey presents a comprehensive review of autonomous data mining systems for real-time big data streams within Edge AI environments. The study systematically analyzes over 120 recent research contributions (2018–2025), categorizing approaches into stream mining algorithms, online learning frameworks, distributed edge intelligence models, federated mining strategies, and self-adaptive optimization mechanisms. Key statistical insights indicate that nearly 68% of recent frameworks employ deep learning–based stream processing models, while 54% integrate adaptive concept drift detection techniques to maintain model robustness in dynamic environments. Furthermore, approximately 47% of surveyed systems incorporate privacy-preserving mechanisms such as federated learning and differential privacy to address edge-level data security challenges. The survey evaluates methodologies based on latency performance, computational efficiency, scalability, autonomy level, and energy consumption. Comparative analysis reveals that hybrid adaptive mining architectures demonstrate up to 35% improvement in real-time decision latency compared to static edge models. The paper concludes by identifying open research challenges, including autonomous model orchestration, resource-aware self-optimization, explainable stream mining, and trust-aware decentralized intelligence. Future research directions emphasize integrating reinforcement learning–driven adaptability and lightweight generative models for continuous stream evolution. This survey provides a structured taxonomy, performance benchmarking synthesis, and a research roadmap to advance next-generation autonomous Edge AI data mining systems.