A High-Speed Omnidirectional Video-Shooting System for Automatic Multiobject Tracking

In observation using omnidirectional video-shooting system, the targets may be incompletely captured as they may move out of the field of view of the fixed virtual cameras. To solve this problem, we implemented viewpoint pitch and yaw control and developed an autotracking control for virtual cameras. Viewpoint control was achieved by applying segment-specific offsets to modify the pan–tilt mirror trajectory. Additionally, the autotracking approach can discern the viewpoint-control signals on the basis of the relative positions of the object-detection boxes within the viewpoint images. This configuration can enable an omnidirectional video-shooting system to function as an array of virtual pan–tilt cameras, capable of centering observed targets within their viewpoints. Results of verification experiments with 10 fps omnidirectional video-shooting indicate that viewpoints of the virtual pan–tilt camera can be moved to the destination positions determined by manual input signals. Moreover, through autotracking in 2° control steps, a moving target could be centered on an arbitrary single viewpoint, with maximal control angles ±8° in both horizontal and vertical directions. Finally, autotracking observations with multiple targets demonstrated that virtual pan–tilt cameras containing the desired targets can be synchronously controlled to track and center the observed targets in their viewpoint images.