Exploiting implicit knowledge for streaming perception object detection

Stream perception is a more challenging task than offline perception. Existing methods perform stream perception object detection by endowing real-time detectors with the ability to predict the future. The difficulty of such methods mainly lies in perceiving complex and changing video background environments, as well as varying object speeds. In this context, we propose a real-time object detection model that utilizes implicit knowledge to enhance features. First, we use a channel implicit knowledge module to perform early fine-tuning on Argoverse-High Definition (Argoverse-HD). This allows the model to perceive the background environment and obtain rich positional features. Then, we use a spatial implicit knowledge module to refine the movement speed features of objects. These refined features are integrated with position features for final fine-tuning. In the final fine-tuning stage, we further weight the original dynamic top- k label assignment strategy to measure the importance of positive samples. Through this weighting, we aim to obtain finer-grained object localization. Our model achieves 37.8% streaming Average Precision (sAP) on Argoverse-HD ( + 0 . 9 % over baseline) with merely 0.01G additional Floating Point Operations (FLOPs) and a latency increase of less than 3 millisecond (ms). Code is available on https://github.com/GjtZ/ISYOLO.git .