Dual-Stream Spatio-Temporal Accident Anticipation and Detection

This paper proposes a novel dual-stream spatio-temporal modeling method for spatio-temporal accident anticipation and detection in dashcam videos, a task that requires not only accident anticipation/detection but also localization of the accident participants. Previous methods have been developed based on temporal modeling with global visual embeddings and object visual embeddings, resulting in an insufficient ability to capture spatio-temporal correlations. Moreover, the object embeddings do not comprehensively incorporate visual, motion, and semantic information of the objects, which is all essential to the traffic understanding. To this end, our method uses a dual-stream framework to model the spatio-temporal correlations from both region and object perspectives. The region stream focuses on local visual contents, while the object stream traces the visual changes and motions of objects using a comprehensive object representation. In contrast to previous works, our method has been evaluated for both accident anticipation and detection on four benchmarks. The experimental results show that our method significantly outperforms previous methods on all benchmarks with a lead up to 10.1 in AUC and 6.1 in AP.