Penetrating Spatiotemporal Correlations in Event-stream Data via Tensor Factorization

Event cameras show unique advantages in dynamic scene perception due to their high temporal resolution and low latency. However, the spatial sparsity and time density of the event-stream data make it difficult to be expressed by frame, which brings challenges to the application based on traditional vision technologies. Considering that the event point carries the motion information of the target object, and has a clear spatiotemporal correlation, if this spatiotemporal correlation can be effectively understood, the quality of the event frame can be effectively improved. Note that tensor factorization is an excellent spatiotemporal data representation method, therefore, we proposed an event frame enhancement method based on tensor factorization. In this method, the event-stream data is first transformed into a tensor, then the spatiotemporal characteristic matrix of the event-stream data is obtained based on CP factorization, and ultimately the spatiotemporal characteristic matrix is used to enhance the event frame. In order to verify the effectiveness of the method, this paper proposes a verification framework based on repeated action counting. The experimental results show that this method can effectively capture the spatiotemporal correlation between event points and can enhance the event frame.