Enhanced Multi-Critic Deep Reinforcement Learning for Channel Estimation in 6G N2V or I2V Communications

The need for self-adaptive systems is increasing with the growth in automation to improve accuracy and performance. Such self-adaptive systems can be built using Reinforcement Learning (RL) models with the help of simulated and recorded data. Deep Reinforcement Learning (DRL) is proposed by infusing NN into RL models and can be used to improve performance in use cases with sparse reward distributions. Traditional Actor-Critic DRL models act with a single critic and single actor network in case of on-policy execution. In contrast, the performance of the agent which makes decisions in the environment improves when considering the information exchange from multiple critics. In this work, the proposed Teacher Tutor Trainee a version of Multi Critic architecture provides the impact of information exchange from multiple critics for a channel estimation use case. Channel Estimation is a method of calculating the impact of the propagation medium and other external factors that deteriorate a signal and its optimization. The performance improvement can be observed from the increase in the explained variance. On implementation, it is observed that the Proximal Policy Optimization model has shown better performance than the Advantage Actor-Critic model.