Hands-On Gaming: Real-Time Gesture Detection in Rock-Paper-Scissors Using Mediapipe and CNN

Real-time hand gesture detection enables intuitive non-verbal communication between humans and devices. In gaming, using hand gestures as a control mechanism enhances user engagement and provides a more immersive experience. This research implements a real-time hand gesture detection system for the Rock-Paper-Scissors game by combining Mediapipe for feature extraction and a Convolutional Neural Network (CNN) for gesture classification. Mediapipe detects and extracts key hand landmarks in real-time, while the CNN classifies the gestures into three categories: rock, paper, and scissors. The proposed system demonstrates high performance, achieving an impressive classification accuracy of 98.12% after 500 training epochs. Furthermore, it ensures adequate detection speed for seamless real-time interaction in gameplay. Future improvements to this system include utilizing a larger and more diverse dataset to enhance the model's ability to recognize a broader range of hand gesture patterns. This approach could improve robustness across different lighting conditions, hand sizes, and variations in gesture execution. Additionally, refining the CNN model architecture could lead to further performance gains in both accuracy and speed. The findings of this research highlight the potential for hand gesture detection to enhance interactivity in gaming and beyond. Applications such as augmented reality (AR), virtual reality (VR), and other interactive systems could benefit from this technology. This study demonstrates that integrating Mediapipe and CNNs can provide an accurate, responsive, and engaging user experience, paving the way for further advancements in gesture-based interaction technologies.