Really, Pedestrian Trajectories: How Realistic are the Datasets?

The accurate prediction of pedestrian trajectories is a crucial ability for systems to safely navigate in real-world traffic scenarios. But pedestrian trajectory prediction is challenging due to the social, environmental, and intra-personal context, as well as the multimodal nature of future paths. The effectiveness of path prediction models in accounting for these factors is highly dependent on the dataset on which they are trained. Moreover, in order for such models to be utilized in real-world traffic scenarios the datasets need to reflect their characteristics, such as various infrastructure elements or pedestrian interactions with a range of road user classes. This work examines how realistic the four common pedestrian trajectory datasets are, namely the BIWI Walking Pedestrians (ETH) dataset, the Crowds UCY/Zara (UCY) dataset, the Intersection Drone (inD) dataset and the Stan-ford Drone Dataset (SDD). To this end, a complexity classification scheme is defined that categorizes pedestrian trajectories based on a newly developed Social Attention (SA) factor, trajectory non-linearity, and the number of starts and stops in a trajectory. The datasets are evaluated based on these factors, the complexity classification, and the environmental context, and then compared to one another. This work provides a source of information about strengths and limitations of the datasets, the complexity of individual pedestrian paths, the behaviors that may be learnable, and the application areas for which they may be best suited.