<b>Anomaly Detection in Smart Grids Using Machine Learning:</b> <b>A Real-Time Streaming Approach Using Apache Kafka and Kubernetes</b>

Smart grids face escalating cyber-physical threats, demanding robust and real-time anomaly detection systems. This paper presents a machine learning (ML) framework for detecting anomalies in smart grid data streams, integrating Apache Kafka and Kubernetes for high-throughput data ingestion and Python-based ML models. We evaluate five algorithms—Random Forest, Neural Network, Support Vector Machine (SVM), Naive Bayes, and K-Nearest Neighbors (KNN)—on a dataset of 100,000 samples with 15 electrical and operational features. The Random Forest model achieves exceptional performance (F1-score: 0.9982, precision: 0.9987, recall: 0.9978), outperforming other models. Kafka integrated in Kubernetes enables real-time data streaming, ensuring timely threat detection. This work bridges scalable data processing with ML-driven security, offering a deployable solution for grid resilience.

Publication Details

Journal
Interdisciplinary Journal of Papua New Guinea University of Technology
Published
2025-10-08
DOI
https://doi.org/10.63900/j960ab54
Primary Topic
Acute Kidney Injury Research
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Anomaly Detection in Smart Grids Using Machine Learning: A Real-Time Streaming Approach Using Apache Kafka and Kubernetes

Interdisciplinary Journal of Papua New Guinea University of Technology
Acute Kidney Injury Research
article

Anomaly Detection in Smart Grids Using Machine Learning: A Real-Time Streaming Approach Using Apache Kafka and Kubernetes

article en

Abstract

Smart grids face escalating cyber-physical threats, demanding robust and real-time anomaly detection systems. This paper presents a machine learning (ML) framework for detecting anomalies in smart grid data streams, integrating Apache Kafka and Kubernetes for high-throughput data ingestion and Python-based ML models. We evaluate five algorithms—Random Forest, Neural Network, Support Vector Machine (SVM), Naive Bayes, and K-Nearest Neighbors (KNN)—on a dataset of 100,000 samples with 15 electrical and operational features. The Random Forest model achieves exceptional performance (F1-score: 0.9982, precision: 0.9987, recall: 0.9978), outperforming other models. Kafka integrated in Kubernetes enables real-time data streaming, ensuring timely threat detection. This work bridges scalable data processing with ML-driven security, offering a deployable solution for grid resilience.

Interdisciplinary Journal of Papua New Guinea University of TechnologyVol. 2(2)
Openalex Percentile: Top 40%
Acute Kidney Injury Research
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

<b>Anomaly Detection in Smart Grids Using Machine Learning:</b> <b>A Real-Time Streaming Approach Using Apache Kafka and Kubernetes</b> · Interdisciplinary Journal of Papua New Guinea University of Technology (2025) | TGRS Research Map | TGRS