Network Routing Optimization Using an AWAC-Enhanced PPO and GAT Architecture

This paper proposes a novel deep reinforcement learning (DRL) framework for network routing optimization based on the integration of Proximal Policy Optimization (PPO) with Advantage-Weighted Actor-Critic (AWAC) style enhancements, and Graph Attention Networks (GAT). Traditional routing algorithms often struggle to adapt to dynamic network conditions. In contrast, prior DRL-based methods like those using Deep Q-Networks (DQN), Graph Neural Networks (GNN), or basic Q-learning algorithms exhibit limitations in scalability and decision-making precision. This model employs a two-layer GAT to extract attention-weighted node embeddings that capture the topological structure of the network. These embeddings are shared by both the actor and critic heads to guide routing decisions and value estimation. An advantage-weighting mechanism, adapted from AWAC, is applied to prioritize actions that yield higher advantages, improving learning efficiency and policy performance. This strategy enhances convergence speed and promotes the discovery of more efficient routing paths. Extensive simulations conducted on a custom network environment demonstrate that the proposed AWAC-Augmented PPO and GAT approach significantly outperforms conventional DQN-GNN-based routing models regarding transmission success rate, path efficiency ratio, and hop count. The results highlight the effectiveness of advantage-weighted policy optimization in achieving robust and adaptive routing in dynamic and highdimensional network environments.