FIM-Stereo: Mono-to-Stereo Audio Upmixing via Neural-Codec Token Prediction with a Text-Conditioned Language Model

FIM-Stereo is a two-stage system for mono-to-stereo audio upmixing based on next-token prediction over a neural audio codec. The pipeline decomposes stereo into orthogonal Mid and Side channels, tokenizes both with EnCodec at 24 kHz / 6.0 kbps, and trains two components: a LoRA-adapted Qwen3-0.6B autoregressive language model that predicts the semantic (RVQ layer 0) Side tokens from the Mid channel and an optional transcript, and a non-autoregressive Transformer (approximately 31 million parameters) that fills the remaining seven Side layers in parallel given the full eight-layer Mid context. The system is trained on 28,539 LibriSpeech utterances with synthetic ILD/ITD stereo in the azimuth range [-80°, +80°]. On a 30-utterance in-domain test set, the autoregressive model reaches 40.47% top-1 token accuracy on the Side layer (413x chance), and the full pipeline achieves a Pearson correlation of r = 0.712 between predicted and ground-truth stereo width. On a 30-utterance out-of-domain LJSpeech evaluation, the pipeline produces perceptible stereo on 67% of samples without fine-tuning. A comparison between a 5,000-utterance proof-of-concept and the full 28,539-utterance run shows that the benefit of text conditioning attenuates with data scale (from +12.72 percentage points with p < 0.001 at POC to a non-significant +3.70 percentage points at full scale), which the author interprets as a scale-dependent confirmation of the semantic-necessity hypothesis raised in the author's prior project FIM-AudioRestore. To the best of the author's knowledge, this is the first application of neural-codec token prediction with a language model to mono-to-stereo upmixing.