Combining Deep Networks with Model-Based Scene Segmentation for Reliably Detecting Rocks on Railway Tracks

Railway transportation is crucial to global mobility, serving as a sustainable conduit for transporting goods and passengers; unfortunately, it is vulnerable to different threats, like the presence of rocks on railway tracks. Accidents caused by rocks can disrupt transport schedules, damage infrastructure, and even lead to injuries or fatalities. Addressing these safety challenges is very important for ensuring the operation of rail networks, which in turn enhances overall mobility and minimizes delays, but also for protecting lives. In this paper, we propose a novel approach to detect rocks on the railway tracks using a camera mounted on the train's front-end. In particular, we exploit domain-specific knowledge regarding intrinsic characteristics of the camera, its positioning with respect to the rails and rail-track characteristics, together with the output of a deep neural network for semantic segmentation, to identify the railway track in the image. Once the track area is identified, it is partitioned into patches and a convolutional neural network is applied to analyze these patches and to detect the presence of rocks; furthermore, in order to enhance the robustness of the detection process, we employ a majority voting mechanism across a sequence of video frames. The experimentation, conducted over six videos augmented with the presence of rocks, confirm the effectiveness of the proposed approach.