Crosswalk: A Traffic Monitoring Dataset for Vehicle Non-Yielding Violations Detection and Interactive Scenarios Understanding

Interactive scenario understanding in traffic has been widely studied for autonomous driving and accident analysis. However, existing studies focus on the interactive behaviors of pedestrians and vehicles from an independent perspective, overlooking the intricate dynamics of their interactions, particularly in the context of violations where vehicles fail to yield to pedestrians. To fill this gap, we introduce <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Crosswalk</i>, a novel dataset with over 7.7k annotated events from 10 hours of surveillance video, providing an integrated view of pedestrian-vehicle interactions in both violation and non-violation scenarios. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Crosswalk</i> highlights the interdependencies between traffic participants and offers event-level evaluation metrics for scene understanding. Besides, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Crosswalk</i> poses two significant challenges in the interactive scenario understanding task, which are capturing subtle patterns of violations and comprehending intricate contextual interactions. To this end, we also propose a multiscale aggregation network that detects violation patterns by integrating temporal and spatial representations while employing hierarchical spatiotemporal fusion. Moreover, we explore state-of-the-art video understanding methods with an event awareness framework as a baseline. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Crosswalk</i> aims to advance the comprehensive understanding of interactive scenarios and specific events in real-world settings. The dataset and code are available at: https://github.com/Nanasaki-Ai/Crosswalk