Aiding Novice Chess Moves with YOLO Pose Detection and Augmented Reality

This paper presents the development of a system combining deep learning, computer vision, and augmented reality to aid learning chess beginner-level players. The proposed approach utilizes a deep learning YOLO model to detect the chessboard and pieces. A series of computer vision algorithms are applied to segment the chessboard into a grid and position detected pieces within the squares. The system provides interactive assistance to the user overlaying helpful information on the augmented reality view. This is achieved by highlighting possible moves for each piece. The custom-trained YOLO model achieved an overall precision of 95% in chessboard and pieces detection. The chessboard keypoints boundary detection reached 99% accuracy. Our chessboard segmentation algorithm was able to correctly identify 98% of the chessboards in the validation dataset. An error rate of 1.45% per chessboard square was achieved while positioning the pieces within the grid. The whole processing pipeline demands an average of 454 ms per image. Future works may explore end-to-end deep learning approaches to improve board and piece localization detection. Additionally, user studies are proposed to evaluate the system’s effectiveness in aiding beginner chess players to improve their skills.