VIGOR: Reviving Cloud Gaming Sessions

Cloud gaming is a multi-billion dollar industry. A client in cloud gaming sends its movement to the game server on the Internet, which renders and transmits the resulting video back. In order to provide a good gaming experience, a latency below 80 ms is required. This means that video rendering, encoding, transmission, decoding, and displaying have to finish within that time frame, which is especially challenging to achieve due to server overload, network congestion, and losses. In this paper, we propose VIGOR, a new method to revive cloud gaming sessions by recovering lost or corrupted video frames. Unlike traditional video frame recovery, VIGOR uses game states to enhance recovery accuracy significantly and utilizes partially decoded frames to recover lost portions. We develop a holistic system that consists of (i) efficiently extracting game states, (ii) modifying H.264 video decoder to generate a mask to indicate which portions of video frames need recovery, and (iii) designing a novel neural network to recover either complete or partial video frames. VIGOR is extensively evaluated using iPhone 12 and laptop implementations, and we demonstrate the utility of game states in the game video recovery and the effectiveness of our overall design.