AWS Cloud Security Detection: A Comparative Evaluation of Rule-Based and Machine Learning Methods

Purpose: This study evaluates whether feature-engineered machine-learning detectors identify AWS cloud-security risks more reliably than rule-based methods across IAM misconfiguration, API misuse, and CloudTrail anomaly detection, and whether observed gains generalize beyond synthetic test conditions.Methods: We compared rule-based baselines with Random Forest models for IAM and API misuse and an Isolation Forest for CloudTrail anomalies. Experiments 2 and 3 used structurally realistic synthetic data. Experiment 1 additionally evaluated the detectors on 19 real AWS-managed IAM policies. Experiment 2 included a held-out low-and-slow adversarial evaluation.Results: Random Forest achieved 98.3% accuracy on synthetic IAM detection versus 75.0% for the rule-based method. On real policies, the original model fell to 42.1% accuracy versus 84.2% for the rule-based method; generalizing one overly narrow feature restored accuracy to 84.2% and recall to 87.5%. API-misuse recall fell from 100% to 3.0% against low-and-slow sessions and recovered to 100% after adversarial training. Isolation Forest achieved 97.4% recall on the syntheticCloudTrail anomalies.Conclusion: Machine-learning gains were strongly dependent on the data distribution and threat behavior represented during development. External and adversarial evaluation exposed failures that were hidden by standard synthetic testing.