Reinforcement Learning Assisted CRDSA Random Access Scheme Based on Enhanced Energy Harvesting for 6G umMTC Networks

In this paper, a multi-replicas independent Q-learning algorithm (MRIQL) assisted enhanced-energy-harvesting-based contention resolution diversity slotted ALOHA (EEH-CRDSA) random access scheme with transmission energy diversity is proposed for sixth-generation (6 G) ultra-massive machine type communications (umMTC) networks, which the RA scheme can be termed as MRIQL-EEH-CRDSA. The case is considered in which every MTC device (MTD) has a finite-sized battery that can be recharged from the Hybrid Access Point (HAP) in the charging subframe and with harvested energy from the circumambient surroundings probabilistically in the transmission subframe. Firstly, the proposed scheme takes full advantage of the difference in the harvested energy level of different replicas from a MTD to result in inter-slot received energy diversity. Subsequently, to overcome the RA congestion issue and to further improve system throughput, a joint optimization problem of the slot and energy levels pairing indexes of replicas is formulated and solved by using the MRIQL algorithm. Numerical simulations demonstrate that the MRIQL-EEH-CRDSA scheme achieves a throughput of 2.0 packets/slot, representing a 284.6 % improvement over conventional CRDSA.