Deep Learning-Based Multi-Modal Sensor Fusion for CSI Compression in Wireless Networks

Accurate channel state information (CSI) compression is a fundamental challenge in wireless communications, especially in highly dynamic environments, such as in vehicular communications. This work explores multi-modal sensor fusion, leveraging radar, camera, and GPS data, to improve CSI compression in wireless communications. More specifically, a scenario is considered in which a vehicle communicates with a static base station (BS), acquiring a CSI estimate via a pilot signal and transmitting the compressed CSI over a rate-limited feedback control channel. The BS, with access to multi-modal side information (MSI) from an MSI server in the form of camera, radar, and GPS sensors. The BS uses this information to refine the CSI estimate using a distributed deep-learning network. This network consists of an encoder at the vehicle, a sensor fusion network at the MSI server, and a decoder at the BS, jointly optimized to minimize reconstruction distortion under limited feedback conditions. The proposed solution is evaluated using the DeepVerse 6G dataset, specifically developed for this study. Several implementations of CSI compression, both static and sequential, are analyzed, showing the adaptability of the neural network fusion architecture to different subsets of sensing modalities. Extensive empirical results demonstrate that the approach significantly improves the quality of CSI estimates, with GPS data alone enhancing accuracy by up to 69% in dynamic scenarios. The empirical evaluations provide useful insights regarding the usefulness of various sensing modalities in CSI estimation. For instance, the results show that in the absence of a control channel, both camera and GPS data contribute significant accuracy gains in CSI estimation, while radar data yields tangible improvements only when supplemented with additional side information transmitted through the control channel. These insights will inform the design of robust CSI compression schemes for next-generation wireless systems.<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>