Evaluation of real-time motion area detection for scene-adaptive imaging

Scene-adaptive imaging has been introduced as a new shooting architecture that facilitates dynamic control of the shooting conditions on the basis of local subject characteristics. This imaging architecture efficiently captures wide field-of-view videos that contain various subjects exhibiting diverse textures and movements. The shooting conditions, such as resolution and frame rate, are changed frame-by-frame for each local area of the image sensor. To achieve higher frame rates when shooting moving subjects, moving areas need to be predicted accurately in real time inside the camera. The block matching methods used in video coding are difficult to apply to motion detection owing to their long signal processing time. We propose three motion area determination methods for scene-adaptive imaging with relatively lightweight signal processing: a) frame subtraction, b) optical flow, and c) event-based vision sensor as a sub-sensor. We present the experimental validation of the effectiveness of each method in terms of accuracy and processing speed. The frame subtraction method can approximate the motion area with dilation process and has a short processing time. The method for predicting the direction of movement using the optical flow effectively reduces false detections. The event-based vision sensor reliably detects movements within a short processing time. The proposed evaluation method based on the binary classification index can effectively compare the performance of each method. The findings afford insights related to motion area determination techniques to enable quick and accurate identification of moving areas that are necessary for scene-adaptive imaging technology.