Quant-NeRF: Efficient End-to-End Quantization of Neural Radiance Fields with Low-Precision 3D Gaussian Representation

Neural Radiance Field (NeRF) has been widely investigated for high-quality 3D object rendering based on captured 2D images. Previous research works have continuously improved the rendering quality with various sample representation and encoding strategies. However, a common bottleneck of NeRF is the extreme computational cost and the lack of compatibility with resource-constrained hardware. Despite the high fidelity of the rendered object, the extensive processing time of the pre-trained NeRF model largely degrades the feasibility of energy-efficient NeRF, especially for resource-constrained edge devices such as augmented/virtual reality (AR/VR) headsets. Most prior works focused on efficient hash table representation or simplified tensorial radiance fields with high-precision representation. However, the efficient, low precision, and hardware deployable NeRF with Gaussian-based modeling remains largely under-explored. Motivated by that, this paper proposes Quant-NeRF, a novel hardware-aware algorithm that performs 3D rendering with end-to-end low-precision representation and hardware deployable computation. Quant-NeRF achieves 60× acceleration compared to prior works on GPU, while maintaining high rendering quality as the full-precision baseline. The proposed algorithm achieves peak performance of 250 FPS.