MVFormer: UNet-Like Transformer With Mix-Voxel Attention for Camera-Based 3D Semantic Scene Completion

3D Semantic Scene Completion (3D SSC), aiming to infer complete 3D scene geometry and semantics from partial observations, is crucial for autonomous driving and robotics. Recent progress in SSC has leveraged the Transformer architecture, while UNet has shown strong multi-scale feature aggregation capabilities. Building on these, we propose MVFormer, integrating the strengths of both architectures to predict the dense geometry and semantics of a scene from 2D images. MVFormer utilizes depth information to separate 2D image features, improving object representation at varying distances, and providing rich geometric and semantic information for voxel initialization. We also introduce a UNet-like decoder with a novel Mix-Voxel attention mechanism, seamlessly integrating Transformer into the UNet structure. This decoder uses multi-scale information to guide voxel feature updates, enhancing multi-scale feature capture. Experiments on SemanticKITTI, SSCBench-KITTI-360 and NYUv2 show that MVFormer achieves state-of-the-art performance, with mIoU scores of 15.81%, 18.94% and 32.96% respectively, while significantly reducing the computational complexity of the model. The code will be available at https://github.com/Ricardovvvv/MVFormer.