Design and Experimental Validation of a Reproducible Big Data-Driven Machine-Learning Framework for Network-Anomaly Analytics

The increasing heterogeneity of modern network environments creates a need for reproducible and automated approaches to network-anomaly analytics. This study presents a modular machine-learning workflow for network-anomaly detection that integrates controlled data generation, training-only preprocessing, model development, validation-based threshold calibration, held-out evaluation, and model persistence. A synthetic dataset containing 20,000 network records, comprising 18,000 normal and 2000 anomalous observations, was generated using predefined traffic-behaviour rules. The models Random Forest, XGBoost, Isolation Forest and Autoencoder were tested with a common experimental pipeline consisting of 60:20:20 training, validation and testing set partitioning, and three experimental seeds. Random Forest had the highest overall accuracy and F1 score in all three experimental runs, both at 1.0000. The mean accuracy for XGBoost was 0.9998 and 0.9992 for the mean F1 score, while the mean accuracy for the Autoencoder was 0.9997 and 0.9983 for the mean F1 score. The mean F1-score, ROC-AUC and PR-AUC of Isolation Forest were 0.9094, 0.9965 and 0.9747 respectively. All the results were summarised by the mean and the sample standard deviation for three runs. Five automated regression tests also passed, verifying reproducible generation, training-only preprocessing, threshold calibration, data-split controls, and model reload consistency. The near-perfect supervised-model results should be interpreted as recovery of strong synthetic generator rules rather than evidence of real-world intrusion-detection generalisation. The study therefore validates a reproducible local batch analytics workflow while identifying distributed processing, real-time streaming, and independent real-world dataset validation as areas for future investigation.