Pixel-Based Change Detection in Moving-Camera Videos Using Twin Convolutional Features on a Data-Constrained Scenario

This work proposes the Pixel-Based Change Detection using a Moving Camera (PBCD-MC) algorithm for detecting anomalies in a cluttered industrial site. The surveillance of industrial environments may be challenging for an anomaly detection problem, as the frequent presence of pipes, cables, and heavy machinery in the scene can create occlusion, undesirable shadows, metallic reflections, etc. A moving camera, surveilling the environment based on reference videos, may also face parallax differences, tracking misalignments, and illumination changes throughout the day. To address these moving camera surveillance problems, the PBCD-MC method was developed based on a hybrid ensemble feature extractor, combining deep learning methods, responsible for generating high-level features, and tree-based algorithms, responsible for selecting and combining the deep features. In addition, in order to deal with the challenges of training with high-dimensional data, a complexity-reducing training strategy that eliminates the need for large training sets while maintaining a robust detection performance was proposed. The PBCD-MC is assessed using a newly developed pixel-level annotated dataset (VDAO-AS) and is shown to outperform state-of-the-art models, achieving overall <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MCC</i> = 0.877 and overall <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DIS</i> = 0.092 on a pixel-level detection, results that are significantly better than those of its competitors. Also, a series of ablation studies proves the importance of the adopted modules for the achieved performance. Software, datasets, results, and the pretrained weights of the proposed architecture are available at https://github.com/lgtavares/VDAO_Pixel.