WatchoutPed: A dataset and model for Vulnerable Pedestrian Anticipation in surveillance videos

This paper addresses the crucial challenge of Vulnerable Pedestrian Anticipation (VPA) in urban environments, utilizing surveillance video data to enhance pedestrian safety. VPA is crucial for identifying pedestrians in potentially dangerous situations, such as walking alongside roads or crossing crosswalks, where the risk of vehicular collisions is elevated. To advance research in this field, we introduce two primary components: the WatchoutPed dataset and the Vulnerable Pedestrian Anticipation Network (VPANet), a baseline network especially designed for VPA. The WatchoutPed dataset has been meticulously enriched with extensive annotations through an innovative auto-labeling technique that integrates ground region analysis with pedestrian state estimation, thus providing a solid foundation for VPA research. Complementing this, the VPANet is engineered to process visual and non-visual inputs extracted from past frames in surveillance footage, enabling it to predict the future state of pedestrians as either safe or unsafe. Tested on the WatchoutPed dataset, VPANet achieves an impressive 89% accuracy, outperforming current methods. Furthermore, we demonstrate the effectiveness of our auto-labeling approach. Notably, the accuracy of VPANet, when trained with the auto-generated annotations from the WatchoutPed, closely parallels that achieved with human-verified annotations, with a negligible variance of less than 1%. The broader implications of our work are significant for the development of smart urban safety infrastructures. Integrating these insights into intelligent crosswalk systems could greatly enhance the monitoring of pedestrian activity near crosswalks, enabling the timely alerting of drivers to the presence of vulnerable pedestrians, and thereby proactively preventing potential vehicular accidents.