Transformer-Based Models for Low-Resource Language Processing in Specialized Technical Domains

Natural language processing encounters major difficulties when processing low-resource languages because technical fields require specialized terminology and lack sufficient annotated data. The performance of transformer-based multilingual models depends on their ability to operate in different domains and their available resources for training. The researchers developed a transformer-based framework that uses XLM-RoBERTa (XLM-R) to process languages that have limited resources for their needs. The proposed methodology includes systematic data preprocessing methods that create low-resource data through data augmentation and implement a two-stage domain adaptation approach that begins with pretraining on unlabeled technical text before moving to task-specific classification fine-tuning. The FLORES-200 multilingual dataset provides a structured benchmark assessment that tests low-resource conditions through its multiple testing methods. Model performance assessment uses standard evaluation metrics, which include accuracy and precision and recall and [Formula: see text]1-score, together with confusion matrix analysis and evaluation curve measurements. The experimental results show that the proposed approach achieved 0.98889 accuracy, 0.98925 precision, 0.98889 recall, and 0.98889 [Formula: see text]1-score, which demonstrates strong accurate classification abilities. The study shows that multilingual transformer models, which use domain adaptation and data augmentation methods, deliver effective solutions to data scarcity challenges while improving performance in technical fields.