Improvement of Sound Quality in Visual Microphone by Manipulation of Focused Area

This paper presents methods for improving the sound quality of visual microphone by emphasizing the focused area in a captured video. When sound reaches an object, it causes vibrations of the object’s surface. Thus, the visual microphone can extract sound by measuring the displacement of the object captured by the camera. In the captured video, there may be area that arise blurred (out-of-focused) due to the depth of field. In out-of-focused area of the captured video, it is difficult to accurately measure the displacement of the object being vibrated by sound, which may deteriorate the quality of the extracted sound. Here, the out-of-focused area is not taken into account in conventional sound extraction methods. In this paper, we propose three methods to extract sound by focusing on the focused area, where displacement can be measured more accurately than in the out-of-focused area. The proposed methods utilize out-of-focused area removal, weighted phase variation, and both processing to emphasize the measured displacement in the focused area of the captured video by using the focal rate that represents the degree of focus. Experimental results show that the proposed methods improve the quality of the extracted sound compared to the conventional method.