SynCoDel: Network-Assisted Synchronization of Video and Haptic Streams for Teleoperations

Teleoperation has become mainstream in several applications, such as telesurgery, manufacturing, and construction. The human operator relies on video, audio, and haptic feedback data from the remote site to adjust operations, ensuring safety and collision avoidance. These applications require real-time perception and synchronization of modalities to maintain a high quality of experience, which presents a significant challenge. Human sensitivities are different among modalities, while communication networks rely on statistical multiplexing to maximize bandwidth utilization, often neglecting the specific latency requirements of each modality. We introduce SynCoDel to minimize asynchrony between video and haptic streams by considering event pairs, which are synchronized events. By prioritizing video packets within these pairs, we effectively utilize bandwidth resources initially reserved for high-priority haptic streams. Evaluation on a practical testbed with the programmable data plane and synthetic data shows that SynCoDel reduced asynchrony over half while maintaining low latency for haptic and video streams in congestion scenarios.