Fast, small and robust hyperspectral 3DGS based on spectral compression

Abstract Three-dimensional reconstruction based on hyperspectral data can be applied in numerous fields. In recent years, the 3D Gaussian Splatting (3DGS) method has garnered widespread interest in RGB images due to its fast training and rendering speeds. Directly extending 3DGS to hyperspectral images faces challenges such as reduced training and rendering speeds and increased demand for computational resources, due to numerous channels in hyperspectral images. This paper proposes a faster, smaller, and noise-robust hyperspectral 3DGS method based on feature dimension compression. The method leverages the advantages of decoupling of spatial and spectral features using the point cloud representation in 3DGS. By employing a two-stage training approach and incorporating a high-frequency regions refinement, this method significantly reduces the training time on hyperspectral data, improves rendering speed, reduces the required storage space and achieves higher robustness to noise, while achieving comparable rendering results compared with the original 3DGS method. The PSNR of rendered image is 33.9 and training time is only 0.55 hour. This method can better perform tasks such as three-dimensional reconstruction and novel view synthesis of hyperspectral data in situations with limited computational resources.