Optimizing IoT Protocol Coexistence and Security using Software Defined Network and Intelligent Machine Learning Detection

The rapid growth of heterogeneous IoT environments has made seamless communication across protocols like MQTT and CoAP increasingly difficult, leading to interoperability gaps, latency issues, and security vulnerabilities. This paper proposes a Software-Defined Networking (SDN)-based architecture that integrates MQTT and CoAP through a bidirectional translation layer, while embedding machine learning (ML) intelligence for real-time flag status monitoring and Denial-of-Service (DoS) attack detection. The system leverages classifiers such as SVM, DT, NB, RF, and KNN within the SDN controller to dynamically predict operational states and mitigate malicious traffic. To evaluate performance, a Mininet-based IoT testbed with 50 heterogeneous nodes was deployed. Simulation results demonstrate that the proposed system achieves up to 95% message delivery success, reduces average latency by 18% compared to baseline translation methods, and saves 12–15% residual energy when using SVM-based classification. While the system improves interoperability and security, it also introduces computational overheads at the SDN controller due to ML inference, which may impact CPU and memory utilization in resourceconstrained environments. The proposed solution is highly relevant for smart city, industrial IoT, and healthcare applications, where interoperability and real-time resilience against attacks are critical. By unifying heterogeneous devices and enhancing security, this approach provides a scalable and practical pathway for next-generation IoT networks.