HarmonicBERT: Music Production Language Modeling

This paper introduces HarmonicBERT, a masked language model customized for the field of music production through fine-tuning of the DistilBERT architecture. Utilizing a corpus extracted from college-level textbooks on music production, HarmonicBERT exhibits significant improvements in evaluation loss compared to both BERT and DistilBERT. Practical evaluations demonstrate HarmonicBERT's effectiveness in word unmasking, analogical reasoning, and bias assessment, highlighting its potential for advancing language understanding in music production contexts.