Two-stage instrument timbre transfer method using RAVE

Recently, the real-time audio variational autoencoder (RAVE) method was developed for high-quality audio waveform synthesis. The RAVE method is based on a variational autoencoder and employs a two-stage training strategy. However, the RAVE model still has limitations in timbre transformation, especially when converting between instruments with significantly different timbres. Issues such as pitch instability, inaccurate timbre reproduction, and severe degradation in sound quality can arise. To enhance timbre transfer performance, we propose a two-stage timbre transformation method using RAVE, which involves applying two timbre transfer models to perform a dual transformation on the original input audio. To evaluate the proposed method, we trained the model and tested its performance using audio generated from MIDI and SoundFont2 sound sources. The results demonstrate that the proposed method improves timbre transfer compared to the single-stage RAVE model.