Enhancing Efficiency in 3D Object Detection with Knowledge Distillation

3D object detection is a critical task in various applications today, such as autonomous driving and robotics. Recently, camera-based approaches have gained popularity over LiDAR-based ones due to their cost-effectiveness and more flexible capabilities. Although generally lighter than their LiDAR-based counterparts, almost all state of the art works heavily depend on computationally intensive deep learning methods, which are still quite demanding. We introduce a method for knowledge distillation tailored to the regression problem, aiming to train a significantly lighter student network by considering multiple loss functions for this purpose. Regarding the teacher model, we utilize a bird's-eye-view (BEV) based architecture and propose suggestions for its improvement. We evaluate the proposed methods on the large-scale dataset nuScenes and manage to outperform state of the art works in terms of the trade off between accuracy and computing time.