Offline Reinforcement Learning For Evaluating Football Players

This thesis investigates the use of offline reinforcement learning (RL) to evaluate football player actions using spatio-temporal event data from Wyscout. Centered on the Implicit Q-Learning (IQL) algorithm, the work presents a complete offline RL pipeline, including state-action-reward extraction, model training, and evaluation based on indirect signals such as league standings and player transfers. The thesis also explores how role-specific context can be incorporated by encoding player positions into the state and applying light reward shaping tailored to tactical responsibilities. Defensive behavior is addressed by isolating out-of-possession events and adding possession indicators to the state, enabling the model to associate defensive actions with longer-term outcomes. Although challenges remain due to limited contextual data and the counterfactual nature of defense, results show partial alignment with real-world performance. To handle football’s hybrid action space, combining discrete decisions with continuous spatial parameters, the thesis introduces Hybrid Implicit Q-Learning (H-IQL), which uses a hierarchical, factorized policy and a shared loss function. H-IQL demonstrates effective learning in both a football environment and a modified benchmark task, suggesting its broader potential for offline RL in structured, hybrid action domains.