Transformer-Based Latency Compensation for Cooperative Perception

The perception coverage for autonomous vehicles and robots is limited by the physical properties of the sensing devices and safety regulations. Cooperative perception uses the sensing data shared among multiple vehicles and roadside units to prolong the sense distance and increase safety. However, communication and computation latency become inevitable and critical issues. This work designs and develops a transformer-based model named TransComNet using historical point cloud data to predict data in the future to alleviate the impact of latency. Using the self-attention mechanism of transformer architecture, the model learns the long-term relation among the point cloud data’s temporal and spatial features, thereby generating accurate predictions. The results show that TransComNet can effectively compensate for the latency, thereby its AP outperforms the state-of-the-art method and the baseline up to 6.38% and 9.24%. Specifically, its AP increases significantly up to 34.96 for occluded moving vehicles. Last but not least, its run-time inference time remains constant when the number of connected agents increases.