VDFSR Knowledge based Variable-Density Photographic Film to Sound Generation Method

Image domain film to sound generation (IDFSG) is a research hotspot in film industry and cross-modal learning. Existing researches neglect sound generation need for variable density film soundtrack. To solve the problem of constructing sound generation processes during variable density film soundtrack reproduction (VDFSR), we propose a PF2SG method based on the VDFSR knowledge, which can effectively achieve high-quality sound generation performance. Specifically, from the perspective of mining the intrinsic consistency of different types of signal capability transformations, we embed variable density film soundtrack reproduction knowledge into the corresponding sound generation process in stages. Moreover, to further enhance the stability of our IDFSG method, we design a sound reproduction module for variable density film soundtrack that integrates the photometric sampling interval based on the mechanical structure of the film projector. Our method effectively explores the intrinsic consistency between the image domain information and the audio domain information during VD film sound generation. Experiments have shown that our method can effectively generate higher quality audio compared to existing methods and commercial devices.