RAILS: Radar Range-Azimuth Map Estimation from Image, LiDAR and Semantic Descriptions

Synthetic generation of radar range-azimuth (RA) maps remains challenging due to limited automotive radar datasets. While camera images are abundant across diverse driving environments, translating visual information directly to radar representations requires sophisticated multi-modal fusion techniques. In this paper, we introduce RAILS, a novel convolutional autoencoder architecture that synthesizes realistic RA maps by integrating RGB camera images, LiDAR depth information, and semantic scene segmentation. By leveraging a U-Net inspired architecture with convolutional block attention mechanisms, our approach transforms multi-modal inputs into accurate radar representations. We demonstrate the model's effectiveness across various driving scenarios using the RADIal dataset, showing superior performance in target localization and scene reconstruction compared to existing methods. Experimental results highlight the potential of using auxiliary depth and semantic information to address the scarcity of radar training data, offering a promising approach for enhancing machine learning-based radar perception in autonomous driving systems.