Multi-modal Guidance for 3D Driver Pose Estimation with Diffusion

Recognition of driver behavior in autonomous driving environments not only captures risky behaviors such as driver distraction in a timely manner, but also has great potential to improve human-vehicle communication and increase personalized comfort in driving. Diffusion modeling has revolutionized several application areas, but 3D pose estimation for drivers is still involved. In this paper, we propose a novel 3D driver action estimation method, named DiffDrive, which achieves more robust and accurate pose estimation in driving environments in the face of dark light and occlusion by effectively incorporating a multi-modal conditional prior in the diffusion process to guide the global structures. We evaluate the effectiveness of DiffDrive and its superiority over existing models using the Drive&Act dataset.