Deep Learning Accelerator Design for Lens Flare Attenuation using High-Level Synthesis

Abstract Lens flare artifacts, caused by stray light, significantly degrade image quality, posing challenges in sectors like automotive and surveillance. Recent advances in artificial intelligence, especially deep learning, offer promising solutions. This study introduces FlareNet, a compact, lightweight U-Net-based model for lens flare reduction. Using an iterative training process, including evaluating transfer learning, FlareNet was developed using a synthetic dataset of flare-containing images. Despite having fewer than 150,000 parameters and only 0.5% of the weights of state-of-the-art models, FlareNet significantly enhances image quality, effectively reducing flare artifacts in both synthetic and real-world scenarios. Furthermore, our work shows the Structural Similarity Index (SSIM) to be a more suitable loss metric compared to the Mean Absolute Error (MAE) or Mean Squared Error (MSE) for resource constrained networks. FlareNet flare reduction has the potential to improve the performance of computer vision algorithms, such as edge detectors and object detectors (YOLOv3), when using flare-attenuated images. The model is implemented in C++ using Vitis HLS to validate its feasibility as a hardware accelerator, evaluating various optimization directives and image sizes for resource utilization and latency, providing a detailed exploration of each layer’s contribution to overall performance while analyzing potential optimization improvements. The tests reveal a resource utilization of less than 20% on a Zeus Zynq UltraScale FPGA. Although further optimization is needed for real-time deployment, this work provides a detailed guide to designing, training, and implementing the model, with the relevant code available as open source.