Leveraging Neural Radiance Field and Semantic Communication for Robust 3D Reconstruction

By leveraging multi-view inputs to synthesize novel-view images, Neural Radiance Fields (NeRF) have emerged as a prominent technique in the realm of 3D object reconstruction. However, the input images of NeRF transmitted through high-interference wireless environment (HIWE) leads to negatively impact the accuracy of 3D reconstruction, thereby limiting its scalability. Fortunately, semantic communication has been proved a effective method to solve the above problem. In this paper, we propose a novel NeRF based 3D semantic communication (NeRF-3DSC) system, which offers robust 3D reconstruction in HIWE. Within NeRF-3DSC, semantic encoder and decoder are employed to extract and recover core semantic features of task-specific specific object (TSO) from the collected images, channel encoder and decoder ensure robust transmission of compressed semantic information in HIWE, lightweight renderer based on NeRF is employed for fast and efficient 3D reconstruction. Moreover, a semantic control unit (SCU) is introduced to guide above components, thereby enhancing the efficiency of reconstruction. Demonstrative experiments demonstrate that the proposed NeRF-3DSC enables robust object reconstruction in HIWE, surpassing the performance of state-of-the-art (SOTA) methods in terms of reconstruction quality.