Heterogeneous-Agent PPO RL for xApps Coordination in Digital Twin Enabled O-RAN

This paper proposes a novel application of Heterogeneous-Agent Proximal Policy Optimization (HAPPO) for resource allocation in Open Radio Access Networks (ORAN), enhanced by Digital Twin (DT) technology. The proposed framework enables the coordination of heterogeneous xApps, where one xApp handles power control with a continuous action space, and another manages resource block (RB) allocation with a discrete action space. By leveraging a synchronized and communication-free DT environment, the framework eliminates the need for real-time inter-xApp communication and synchronization typically required in live O-RAN systems. The HAPPO algorithm employs a clipped surrogate loss to ensure stable training and mitigate policy oscillations. It allows each xApp to maintain an independent actor network while sharing a centralized critic for advantage estimation, enabling both agent-level flexibility and coordinated optimization. Experimental results demonstrate that the proposed approach achieves superior performance, reaching a QoS score of $93.2 \%$, and maintains low inference latency of 1.5 ms in dynamic O-RAN environments with fluctuating user demands and diverse QoS requirements.