Enhanced exploration in reinforcement learning using graph neural network based intrinsic reward mechanism

We propose a Graph Neural Network-based Intrinsic Reward Learning (GNN-IRL) framework to address the exploration-exploitation trade-off in Reinforcement Learning (RL). This approach leverages the structural modeling capabilities of Graph Neural Networks (GNNs) to represent the transitions and relationships between states in the environment. Intrinsic rewards are computed based on centrality measures and inverse degree analysis within the state graph, enabling the agent to identify and explore novel or under-visited states. The effectiveness of GNN-IRL was validated in four benchmark environments with discrete action spaces: CartPole-v1, MountainCar-v0, Taxi-v3, and LunarLander-v3. Continuous state variables were discretized to construct state graphs, which facilitates the implementation of GNN-IRL but may limit scalability to very high-dimensional continuous spaces. The experimental results show that GNN-IRL outperforms state-of-the-art extrinsic and intrinsic exploration strategies in terms of convergence rate, cumulative reward, exploration efficiency, and state coverage. These findings demonstrate that GNN-IRL effectively balances exploration and exploitation, thereby improving sample efficiency and accelerating policy learning in discretized RL domains.