Multivideo Stitching Method Based on Temporal Compressive Sensing

In this paper, we proposed a real-time high frame rate video stitching method based on temporal compressed sensing technology, which stitched the video captured by multiple cameras into a panoramic video with a large field of view (FOV). First, the same sample mask with repetitive mode is used to process the multi-camera input video stream, which reduced the need for high spatiotemporal resolution images and simplified data processing. Then, the compressed video stream was spliced, and the improved scale-invariant feature transformation (RootSIFT) was used for key point detection and description. The key point matching was performed by a hybrid matcher combining Brute Force (BF) and Fast Linear Approximated Nearest Neighbours (FLANN), and the homology matrix was estimated based on Random Sample Consistency (RANSAC) after geometric verification and outlier elimination. The linear masking method and the group stitching strategy were used to eliminate the parallax. Finally, based on the temporal video compressive sensing (CS) reconstruction architecture of the fully connected neural network, the spliced panoramic compressed frames were processed to generate a panoramic video stream. Experimental results showed that the proposed method not only optimized the computing and storage resources, but also significantly improved the stitching effect, which provided a new solution for high-speed and high-frame-rate video processing and stitching.