Predictive pixel-wise optical encoding: towards single-shot high dynamic range moving object recognition

Conventional low dynamic range (LDR) imaging devices fail to preserve much information for further vision tasks because of the saturation effect. Thus, high dynamic range (HDR) imaging is an important imaging technology in extreme illuminance conditions, which enables a wide range of applications, including photography, autonomous driving, and robotics. Mainstream approaches require multi-shot methods because the conventional camera can only control the exposure globally. Although they perform well on static HDR imaging, they face a challenge with real-time HDR imaging for motional scenes because of the artifact and time latency caused by multi-shot and post-processing. To this end, we propose a framework, termed POE-VP, which achieves single-shot HDR imaging via a pixel-wise optical encoder driven by video prediction. We use highlighted motional license plate recognition as a downstream vision task to demonstrate the performance of POE-VP. From the results of simulation and real scene experiments, we validate that POE-VP outperforms conventional LDR cameras by more than 5 times in recognition accuracy and by more than 200% in information entropy. The dynamic range could reach 120 dB, and the captured data size is verified to be lower than mainstream multi-shot methods by 67%. The running time of POE-VP is also validated to satisfy the needs of high-speed HDR imaging.