Temporal knowledge graph construction from football match videos

Abstract In the current landscape of sports data analysis, particularly in football, the automatic extraction of information from match videos is a key challenge. Traditional video annotation techniques rely on the generation of natural language descriptions from videos or detection analysis of static frames. The first approach has limitations in the level of detail of the information extracted, while the second does not analyze the information at a high semantic level. This work presents an efficient framework for the automatic construction of Temporal Knowledge Graphs from unlabeled football match videos, using computer vision techniques and multimodal large-language models. The constructed graph refers to the entire video of the match and contains information on the positional values of the players on the field, jersey number identification, team recognition, interactions, and ball possession. The framework implements a solution for handling failures without keyframe loss, as well as support for parallel processing. In addition, we propose two metrics to assess the quality of the resulting graph in terms of overall consistency and missing information.