DPoint-GS: Control Point Gaussian Splatting for Dynamic Scenes

Implicit neural representations have enabled novel methods to dynamic scene reconstruction and rendering. However, many advanced neural rendering techniques that rely on these implicit representations encounter two key limitations: difficulty in capturing fine details of objects and a lack of real-time performance in more general dynamic settings. To handle these difficulties, we offer a strategy that segregates the motion and visual elements of dynamic scenes into sparse control points and numerous Gaussian points. By exploiting the significantly smaller number of control points, we learn a compact six-degree-of-freedom transformation basis through interpolation weights, yielding the motion fields for the dense Gaussian points. Next, a deformation Multi-Layer Perceptron (MLP) is utilized to predict transformations that vary over time for each control point. This reduces complexity and boosts learning capacity. It also helps maintain spatiotemporally coherent motion patterns. Additionally, we integrate a point transformer module to better reconstruct details. Our method jointly learns 3D Gaussian distributions, canonical control point position features, and the deformation MLP. This unifies appearance, geometry, and dynamics in one framework. Experiments show that our approach outperforms existing methods in terms of rendering quality.