An Investigation Of Rotary Position Embedding For Speech Enhancement

Transformer architecture has driven significant advancements speech enhancement (SE). Due to the position-agnostic characteristic of Transformers, positional embedding has become a crucial component to make Transformers to distinguish the order of elements among the input sequence. Rotary Position Embedding (RoPE) demonstrates exceptional performance on language models, particularly in processing the variable-length input sequences. However, the impact of RoPE on SE has not been thoroughly investigated. To address this, this paper introduces RoPE into SE and proposes the Transformer-RoP architecture, then comprehensively evaluates its effective-ness. Experiments were conducted with a standard Transformer architecture as the baseline and PSM as the training target. The results illustrate that Transformer-RoP achieves substantial improvements across all metrics.