Latency Robust Cooperative Perception Using Asynchronous Feature Fusion

Recent advancements in cooperative perception have showcased substantial improvements compared to single-agent perception. Nonetheless, the inherent latency present in such systems can dramatically impair their effectiveness. In this paper, we propose a Latency Robust Cooperative Perception framework, named LRCP, to compensate for the effect of temporal asynchrony. The intuition of LRCP is to directly fuse asynchronous bird's-eye view (BEV) features instead of estimating aligned features. To achieve this, we first propose a novel flow prediction module that uses cached past BEV features to predict the flow with a non-discrete time delay at the BEV feature level. Then, the predicted flow is employed to guide the spatial sam-pling location of interests. Our approach substantially en-hances the robustness of temporal asynchronous cooper-ative perception. Specifically, we achieved robust performance across a range of latencies up to 500 ms, with a per-formance degradation of only 1 percent point for [email protected] metric and 4 percent points for [email protected] metric at 500ms on two public datasets (V2X-Sim and Dair- V2x). Code to reproduce our results is available at https://github.com/JesseWong333/LRCP.