Three-Dimensional Reconstruction of Satellites Using Dual-Branch NeRF with ISAR and Optical Fusion

The three-dimensional (3D) reconstruction of space targets is much helpful to space situational awareness. An improved Neural Radiance Fields(NeRF)-based dualbranch 3D reconstruction framework capable of integrating low-resolution ISAR and highresolution optical sensor data is proposed. By processing radar and optical images obtained from the same view angle, the dual-branch network outputs the volumetric density, amplitude and phase, and RGB values of the target, which are then rendered into ISAR and optical images. A combined loss of dual-branch between the rendered and input images supervises network training. Once trained, the network generates ISAR and optical images from arbitrary viewpoints and reconstructs the 3D structure of the target. Experimental results on synthetic datasets demonstrate that the accuracy of rendered images from novel viewpoints using the dual-branch network outperforms those obtained by the single-branch networks without fusion. The proposed 3D reconstruction framework also successfully reconstructs the complete structure of the target.