Sound Quality Improvement in Visual Microphone by Emphasizing Focused Area Based on Focal Rate

This paper presents a method of improving the sound quality of a visual microphone by emphasizing the in-focus area in a captured video. A visual microphone extracts sound by measuring the displacement of a vibrating object from the video of an object vibrating due to sound. In the captured video, there may arise blurred (out-of-focus) areas may arise due to depth of field. In these out-of-focus areas of the captured video, it is difficult to accurately measure the displacement of the vibrating object, which may reduce the sound quality of the extracted sound. However, out-of-focus areas are not taken into account in conventional sound extraction methods. We propose a method to improve the sound quality of the extracted sound in the visual microphone by emphasizing the in-focus area in the captured video. The proposed method emphasizes the measured displacement in the in-focus area of the captured video by using the focal rate that represents the degree of focus. Experimental results show that the proposed method improves the quality of the extracted sound compared to the conventional methods.