DMPG-BEV: Diffusion Model-Based Point Clouds Features Generation for Efficient Camera-Based BEV Perception

Bird’s Eye View (BEV) perception requires swift and accurate representation of real-world 3D information, and BEV-based 3D object detection and semantic segmentation are foundational for downstream autonomous tasks such as prediction and planning. Current BEV perception methods are divided into Light Detection and Ranging(LiDAR)-based and camera-based methods depending on the input data modalities. LiDAR-based methods offer precise 3D information, but are limited by high sensor costs. Conversely, camera-based methods are economical and widely researched. However, 2D image features transform into BEV space introducing feature distortion due to inaccurate 3D information, particularly in depth estimation which results in noisy projections. Denoising Diffusion Probabilistic Models (DDPM) are used to mitigate noise in BEV features. However, DDPM increases computational costs, relies on noisy images BEV features, and limits effectiveness. To address these issues, we propose a lightweight BEV perception framework (DMPG-BEV) to enhance the performance of camera-based methods. This framework employs Denoising Diffusion Implicit Models (DDIM) to generate point cloud features with 3D information, while reducing computational demands by fewer denoising steps. We also introduce a lightweight feature fusion module which integrates point cloud BEV features with image BEV features to incorporate 3D information and reduce noise. The refined BEV features are also well-suited for multi-task networks involving detection and segmentation tasks. Additionally, a lightweight view transformation module is included to further reduce computational costs. Extensive experiments on the nuScenes benchmark demonstrate better performance of our proposed model with reduced computation, highlighting its value in BEV perception.