One-Shot Team Recognition and 3D Pose Estimation of Cyclists for Augmented Reality Visualization

Applying advanced computer vision and machine learning tech- nologies transforms how we experience sports events. This research focuses on enhancing the viewing experience of cycling races by identifying and classifying teams from helicopter footage, address- ing challenges posed by fast movements and often similar team uniforms. State-of-the-art object detection and one-shot team recog- nition via siamese neural networks are implemented to provide effi- cient team classification with minimal labeling. A range of advanced computer vision models has been tested for their effectiveness in accurately recognizing teams, with the siamese neural networks achieving a classification accuracy of 95%. Furthermore, tempo- ral tracking and post-processing techniques have been applied to strengthen classification performance. These methods improve the quality of metadata during broadcasts by adding detailed team clas- sification, team visibility, and positioning, thereby facilitating a more informative viewing experience. The developed software also facilitates post-race analyses with visualizations that offer insights into team performances. The research further explores using aug- mented reality (AR) and 3D pose estimation to enhance the visual presentation of live broadcasts. This involves integrating real-time data such as the riders' names or speeds, enriching the broadcast's informational value. The combination of augmented reality and advanced computer vision opens up new possibilities for enhancing live sports broadcasts.