Investigation on Transformer-Based Translation From Mandarin to Hakka

This study explores Mandarin to Hakka translation using Transformer models, specifically addressing the challenges of dialectal variations and limited data. We developed specialized models tailored to Hakka's linguistic characteristics and expanded the Hakka corpus. Using the mT5-small model, we fine-tuned it with layer normalization adjustments and gradient clipping to better handle Hakka's unique linguistic features.