Machine Learning-Assisted Cryptographic Security: A Novel ECC-ANN Framework for MQTT-Based IoT Device Communication

The Internet of Things (IoT) has surfaced as a revolutionary technology, enabling ubiquitous connectivity between devices and revolutionizing traditional lifestyles through smart automation. As IoT systems proliferate, securing device-to-device communication and server–client data exchange has become crucial. This paper presents a novel security framework that integrates elliptic curve cryptography (ECC) with artificial neural networks (ANNs) to enhance the Message Queuing Telemetry Transport (MQTT) protocol. Our study evaluated multiple machine learning algorithms, with ANN demonstrating superior performance in anomaly detection and classification. The hybrid approach not only encrypts communications but also employs the optimized ANN model to detect and classify anomalous traffic patterns. The proposed model demonstrates robust security features, successfully identifying and categorizing various attack types with 90.38% accuracy while maintaining message confidentiality through ECC encryption. Notably, this framework retains the lightweight characteristics essential for IoT devices, making it especially relevant for environments where resources are constrained. To our knowledge, this represents the first implementation of an integrated ECC-ANN approach for securing MQTT-based IoT communications, offering a promising solution for next-generation IoT security requirements.