An RFSoC Prototype for Third-Party Camera Aided mmWave Communications

Leveraging cameras and LiDARs has been proved as an effective way to achieve beam management without training overhead for millimeter wave (mmWave) communications. However, existing methods place sensors at base station (BS) and/or mobile station (MS), which may induce privacy concerns and augment communications system expenses. In this paper, we propose a novel third-party camera aided mmWave beam management framework and self-build an RFSoC prototype for validation. Specifically, we design third-party camera aided beam alignment and blockage prediction algorithms that could work with random third-party perspective. Then, we leverage the proposed beam alignment to design initial access and beam recovery, and utilize the proposed blockage prediction to realize failure prediction. Hand-off from mmWave to sub-6G communications is conducted once the blockage is predicted. To validate the proposed framework, we develop an RFSoC prototype from scratch independently. The real-world real-time experimental results show that the beam alignment achieves over 98% in top-5 accuracy and reduces the time consumption to below <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1/50$ </tex-math></inline-formula> of that incurred by exhaustive beam sweeping. Meanwhile, the prototype maintains 410 MHz dynamic mmWave communications, and can seamlessly switch to sub-6G communications before a mmWave blockage happens.