Towards Complete Input Representations for Vehicle Trajectory Prediction Models

Predicting accurate trajectories for traffic participants is a crucial task in the autonomous driving field.Especially in complex scenarios like urban intersections we need to anticipate intentions and positions ofsurrounding agents in order to plan a safe and smooth maneuver for an ego-vehicle. Trajectory predictionremains a challenging task due to a diverse set of influencing factors like the topology of the street, trafficrules and physical dynamics but also stochastic intentions of the agents and their mutual influences. Theperformance of prediction models depends on both the input representation, i.e. the encoding of the scene,and the prediction, i.e. the decoding of trajectories. Recent works have mainly focused on the decodingpart. Conversely, in this work we investigate the input representations of trajectory prediction models.They can roughly be divided into raster- and vector-based methods. Raster-based methods representthe traffic scene as bird’s-eye-view images whereas vector-based methods represent all elements as setsof points. We explain the pros and cons of both methods and carry out preliminary experiments toenhance the representations by 1) enriching the over-simplistic vector representation and 2) combiningboth representations. Our finding results on the popular NuScenes benchmark indicate that very simplerepresentations are actually sufficient for state-of-the-art results. This is in contrast to real-world drivingwhere more information influences driver decisions. Consequently, it might be the call for more diverseand representative datasets and/or more expressive models.