IRIS: An Energy-Efficient Spatial Computing SoC for Real-Time Interactive Rendering and Modeling With Surface-Aware 3-D Gaussian Splatting

The 3-D Gaussian splatting (3DGS), based on a machine learning-driven radiance field technique, is rapidly emerging as a next-generation solution in 3-D graphics. Owing to its short modeling time, computational simplicity, and high rendering quality, it is expected to replace traditional 3-D graphics on edge devices. However, its substantial memory access requirements and the challenges of real-time modification present major barriers to deployment on mobile augmented/virtual reality (AR/VR) platforms. We present a spatial computing system-on-chip (SoC), IRIS, to enable real-time, spatially interactive rendering with surface-aware modelling (SAM). The architecture of IRIS introduces three key hardware features. First, the surface perception unit (SPU) reduces external memory access (EMA) and minimizes Gaussian sorting latency by clustering Gaussians based on surface alignment. Second, the reconfigurable ApproxLP (ALP)-based multiply-accumulator array (RAMA) achieves high computational efficiency and throughput during neural Gaussian feature generation and mesh deformation. Finally, the error direction cache (EDC) enables fast and efficient modeling by eliminating redundant backpropagation (BP) operations. As a result, IRIS is fabricated using a 28-nm process and occupies a die area of 20.25 mm<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup>. When evaluated on the SyntheticNeRF dataset, IRIS achieves <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$34.3{\times }$</tex-math> </inline-formula> faster interactive rendering and consumes <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$300.0{\times }$</tex-math> </inline-formula> less energy per frame compared to an edge GPU, despite having <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$21.3{\times }$</tex-math> </inline-formula> lower external memory bandwidth.