Event-driven multimodal fusion for image motion deblurring

• Utilizing event data to assist in removing motion blur from images. • Encoding the degree of image blur using event density. • Performing modality fusion by frequency separation. Traditional frame-based cameras are susceptible to non-uniform blurring in real-world scenarios due to their inherent “frame” imaging method. In contrast, bio-inspired event cameras can capture changes in pixel brightness similar to the human eye, recording motion changes in the scene with extremely high temporal resolution, effectively addressing the issue of motion blur. In this paper, we present an Event-Driven Multimodal Fusion (EDMF) deblurring network, which utilizes event data to remove blur and achieve clear, high-quality image restoration. To facilitate the fusion of the two modalities, we first design a Deep Spatio-temporal Event (DSE) voxel grid specifically for deblurring with event cameras, effectively leveraging event information. Our deblurring network initially extracts high-frequency information from images through frequency separation. It then employs a specially designed Event-Image multimodal High-frequency Enhancement (EIHE) module to integrate this information with event data, restoring sharp and clear edges in the images. Compared to other state-of-the-art methods, the proposed EDMF demonstrates outstanding performance, including exemplary visual quality and excellent preservation of fine texture details. The source code and dataset are openly accessible at https://github.com/ice-cream567/EDMF .