BinauralLM: spatial prompted text-to-binaural audio generation with large language model

Abstract Binaural audio is a practical spatial audio format for headphone playback and is widely used in immersive media such as games, VR/AR, and interactive multimedia. Currently, binaural audio is still largely obtained either by specialized recordings or by spatializing pre-recorded monaural signals, which limits flexibility for controllable content creation. While recent text-to-audio models can generate the required audio, most of them target monaural output or unconstrained stereo. They provide limited control over spatial attributes such as direction and motion. To address these challenges, we propose BinauralLM, a spatial prompted text-to-binaural audio generation framework with large language model (LLM). BinauralLM adopts a two-stage pipeline. First, we use a Qwen2.5 LLM to autoregressively generate coarse binaural token sequences conditioned on text and spatial prompts. To effectively couple semantic content with spatial cues, we introduce a spatial prompt encoder and an attention fusion module to produce a unified conditioning representation for binaural token generation. Second, we employ a flow matching model that takes the coarse binaural tokens as conditions and reconstructs high-quality binaural waveforms in the continuous domain. Experimental results on the BEWO-1M dataset demonstrate that BinauralLM achieves superior performance on standard text-to-audio metrics and binaural spatial metrics, and achieves higher MOS scores in perceptual evaluation, especially on spatialization quality. These results indicate that our proposed method offers an effective route to controllable and high-quality text-to-binaural audio generation.