Millimeter-Wave SDR Testbed Enabling Deep Q-Learning for Angle-of-Arrival Detection

Millimeter-wave (mmWave) signals experience severe environmental path loss. To address this issue highly directional antennas with beamforming capabilities are used to align the transmit and receive antenna beams, allowing the signal to travel longer distances. However, advanced algorithms are needed to detect the angle-of-arrival (AoA) for better alignment of the antenna beams. Combining software-defined radios (SDR) with mmWave radio frequency (RF) antenna systems can enable researchers to develop advanced algorithms for real-world scenarios. Using low-cost mmWave RF front-end components and an SDR, we develop a testbed that uses open-source tools and high-level programming languages for beamforming algorithm development. In the testbed, we prototype a deep Q-network (DQN) algorithm for mmWave AoA detection. We evaluate the performance of the algorithm by fine tuning the reinforcement learning (RL) hyperparameters.