Reducing E2E Delay in Wireless Multi-Hop Networks: A Multi-Agent DRL-Based Approach

Recently, bio-inspired algorithms have gained popularity among researchers due to their capability to handle complex optimization problems in a distributed manner by imitating natural processes of the living organisms on earth.In previous studies, two bio-inspired algorithms called desynchronization (DESYNC) and multi-hop DESYNC (MH-DESYNC) have been proposed to improve the throughput in wireless networks in a distributed way.These decentralized algorithms have shown promise for scalability and adaptability, making them suitable for real-world applications.Although the MH-DESYNC has effectively improved the throughput of the wireless multi-hop networks, it still faces the problem of end-to-end (E2E) delay.In this work, we aim to fill the gap of our previous study (MH-DESYNC algorithm) by using machine learning (ML).We propose a design of decentralized multi-agent deep reinforcement learning (DRL), a type of ML algorithm, incorporated with MH-DESYNC for improving not only the throughput but also the E2E delay in wireless multi-hop networks.To evaluate our proposed method, we compare it with the multi-agent DRL (MA-DRL), the MH-DESYNC, and expected E2E delay (EED) by performing under the simulation of multiple routing paths in wireless multi-hop networks.The training result shows that our proposed method converges faster with low fluctuation compared to the MA-DRL method.Additionally, our proposed method outperforms the MA-DRL, the MH-DESYNC, and EED methods for both E2E delay and throughput performances.