Addressing tonal weaknesses in neural audio codecs through synthetic data augmentation and balanced training

Abstract Neural audio codecs (NACs) achieve strong perceptual quality at low bitrates but consistently struggle with harmonic-rich, sustained tonal content such as bells, glockenspiel, and piano. Prior work has used synthetic tonal augmentation mainly to improve embedding quality for evaluation; we instead treat it as a first-class training methodology that directly targets codec reconstruction fidelity. This paper presents Project Chimera, a training framework for tonal-robust NACs, instantiated as a 48 kHz codec with 32 codebooks at 30 kbps. The framework introduces three coordinated components: (1) a physically-inspired tonal synthesis generator modelling inharmonic partial structures and resonant decay beyond simple sinusoids; (2) a difficulty-aware curriculum sampling strategy that progressively increases tonal complexity during training; and (3) Exponential Moving Average (EMA) codebook stabilisation with quantisation-capacity analysis that characterises and prevents Residual Vector Quantisation (RVQ) collapse under tonal training. In limited-scale experiments on a 720-hour dataset, Project Chimera achieves 0.51 mel distance and 4.42 ViSQOL at 30 kbps on these objective metrics, outperforming the Descript Audio Codec (DAC) at comparable bitrates. A listening test with $$n=22$$ trained listeners shows subjective gains of approximately $$+0.47$$ MOS overall and $$+0.9$$ MOS on tonal material; the 95% confidence intervals for the overall, tonal, and general-music comparisons separate from DAC, while speech and environmental scores remain comparable. Per-phase checkpoints of the curriculum run show tonal fidelity, codebook entropy, and subjective quality improving monotonically as tonal complexity is staged in. The objective results come from a limited-scale training run; large-scale objective replication remains future work. Code and model weights will be released upon acceptance.