Event-Aided Progressive Neural Radiance Fields Reconstruction Under Challenging Illumination

Scene reconstruction and novel view synthesis are extensively utilized in domains such as autonomous driving, facilitating the creation of more comprehensive datasets and accelerating algorithm development. Current techniques have achieved promising results in image-based synthesis. Nonethe-less, they encounter difficulties including limited dynamic range, motion blur, and suboptimal performance under challenging illumination. Furthermore, these techniques depend significantly on precise camera poses, which are difficult to acquire in practical situations. Event cameras, characterized by their high dynamic range and temporal resolution, demonstrate advantages in challenging illumination and rapid motion scenarios and have been utilized in several autonomous driving applications. This study is the first to combine image and event camera for neural radiance field (NeRF) reconstruction in driving scenarios. The proposed method leverages the unique properties of event camera and adopts a progressive reconstruction strategy to jointly optimize camera poses during training, reducing reliance on pose precision and enabling more robust and accurate scene reconstruction under challenging illumination. Experimental results demonstrate that the proposed approach attains enhanced performance in novel view synthesis, exhibiting a 3.5% increase in PSNR and a 2% increase in SSIM on the DSEC dataset compared to baseline method.