Real-Time Player Tracking Framework on MOBA Game Video Through Object Detection

The multiplayer online battle arena (MOBA) genre boasts the largest audience in esports, leading to extensive research in esports analysis targeting MOBA games. However, due to the limited availability of openly accessible data or application programming interface (API), most research has been focused on <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Dota 2</i> and cannot be easily extended to other MOBA games. In this article, we present a novel framework that revolutionizes real-time player trajectory extraction directly from the game screen of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">League of Legends</i> (<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LoL</i>) through object detection. To mitigate reliance on APIs, the proposed framework includes a process that generates synthetic images as training data for object detection, detects the positions of the game characters from the minimap, and considers temporal relationships to ensure stable trajectory acquisition against occlusion. For evaluation purposes, we generate ground truth data from <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LoL</i> replays and introduce the concept of occlusion tolerance. Our proposed framework undergoes evaluation and analysis in terms of trajectory extraction accuracy with occlusion tolerance, the significance of synthetic image elements, class-by-class detection accuracy, and processing time. Our framework opens new avenues for esports analysis. We envision its potential extension to other games lacking APIs, provided that they feature a minimap displaying game characters.