Seam-Driven Video Stitching Algorithm Based on Pedestrian Detection

With the increasing demand for video surveillance, people have higher requirements for the visual field of videos. Video stitching technology is a mainstream approach to acquire panoramic video at a low cost by stitching videos with overlapping areas captured by multiple imaging devices. However, the current methods for video stitching often lead to visually unrealistic results such as misalignment and ghosting artefacts in overlapping areas, particularly when moving targets are present. Herein, a pedestrian detection-based video stitching algorithm is proposed to address the misalignment and ghosting artefacts in overlapping areas caused by moving targets to create panoramic videos. Precisely, we extract a series of images from the video frame by frame and initially align the images by employing SIFT and RANSAC. In the stage of image fusion, we determine the rectangular boxes containing pedestrians based on HOG features, then we add a pedestrian term into the seamline energy function and calculate the path with the lowest energy to get a visually optimal seamline that avoids pedestrians. The individually stitched frames are compiled into a complete panoramic video. The improved algorithm in this paper is compared with some of the state-of-the-art methods. Several nonreference image evaluation indicators and stitching speed data were used to demonstrate the effectiveness of the video stitching algorithm proposed in this paper, which can avoid moving pedestrians and achieve better stitching results.