VisionPlay: An Interactive Ball Game Using Computer Vision and Augmented-reality Overlay

This paper looks at how an integrated of technologies can be used for an enhancement of interactive gaming experience in digital sports. A prototype interactive ball game called ‘VisionPlay’ developed in our study offers a natural interactive experience with tennis balls using computer vision and augmented-reality (AR) overlay. We aim to provide a lightweight and cost-effective solution to achieve high tracking accuracy for the ball and precision for the real-time interaction with virtual game content overlaid in the physical environment. We describe some of the design decisions and technical challenges overcome in the development of our interactive game. This paper employs the Intersection-over-Union (IoU) method as the evaluation metric to validate the detection and tracking accuracy of our system, by following up with the qualitative analysis with questionnaires to evaluate the system’s performance and user perception. The IoU analysis demonstrates high accuracy and stability of object detection under different environmental conditions. The user study reveals significant improvement in participants’ exercise motivation and entertainment experience. The innovative form of interaction, enabled by real-time capturing of physical movements through the camera, provides an unparalleled degree of immersion and involvement.