Enhancing Kubernetes security with machine learning: а proactive approach to anomaly detection

Kubernetes has become a cornerstone of modern software development enabling scalable and efficient deployment of microservices. However, this scalability comes with significant security challenges, particularly in detecting specific attack types within dynamic and ephemeral environments. This study presents a focused application of Machine Learning (ML) techniques to enhance security in Kubernetes by detecting Denial of Service (DoS) attacks and differentiating between DoS attacks, resource overload caused by attacks, and natural resource overloads. We developed a custom monitoring agent that collects telemetry data from various sources, including real-world workloads, actual attack scenarios, simulated hacking attempts, and induced overloading on containers and pods, ensuring comprehensive coverage. The dataset comprising these diverse sources was meticulously labeled and preprocessed, including normalization and temporal analysis. We employed and evaluated various ML classifiers, with Random Forest and AdaBoost emerging as the top performers, achieving F1 macro scores of 0.9990 ± 0.0006 and 0.9990 ± 0.0003, respectively. The novelty of our approach lies in its ability to accurately distinguish between different types of resource overloads and provide robust detection of DoS attacks within Kubernetes environments. These models demonstrated a high degree of accuracy in detecting security incidents, significantly reducing false positives and false negatives. Our findings highlight the potential of ML models to provide a targeted, proactive security framework for Kubernetes, offering robust protection against specific attack vectors while maintaining system reliability.