Development and Evaluation of an AR Shooting Training System Using Hololens 2 and Reinforcement Learning

With the introduction of Defense Innovation 4.0, South Korea is actively integrating advanced technologies into military training. The current shooting training methods used by the South Korean military rely on projector-based virtual reality (VR) systems or non video see-through (NVST) head-mounted displays (HMDs), which require large spaces and lack real-world interaction. To address these limitations, this study proposes an augmented reality (AR)-based shooting training system, implementing precise tracking of firearm positions and body movements using Hololens 2 and VIVE trackers. Additionally, a reinforcement learning-based AI assistant utilizing the Proximal Policy Optimization (PPO) algorithm is introduced to analyze shooting accuracy and precision, providing real-time feedback to improve user posture. Furthermore, by adopting TCP and UDP-based socket communication, the system achieves an average latency of 2.30ms, validating its feasibility for real-time training.