Gaussian or Plane? Both: Semantic-Driven Voxel Representation for LiDAR–Inertial Odometry

Accurate LiDAR-inertial odometry (LIO) highly depends on the geometric fidelity of the underlying environment representation. We explore the new and interesting research direction of integrating semantic segmentation models into metric odometry algorithms to enrich their representational capacity. Specifically, this letter proposes a semantic-driven hybrid voxel representation in which an off-the-shelf 3D segmentation network assigns every voxel to either a planar or nonplanar class, using planar and Gaussian representations, respectively. Consequently, a hybrid scan matching strategy is presented using class-specific residual models that are tailored to the distinct error statistics of each surface category. The scan matcher is embedded within an Iterated Extended Kalman Filter (IEKF) for odometry and mapping. We evaluate our method on diverse platforms and environments, and show improved localization accuracy across various indoor and outdoor scenarios, while maintaining real time performance.