Semantic Role Labeling for Code-Switched Data

In natural language processing, pre-trained language models have shown significant promise in various linguistic tasks. This study investigates the efficacy of various pre-trained models, namely XLM-RoBERTa, M-BERT, and HING-MBERT, within an extensive classification framework encompassing semantic role labeling and part-of-speech tagging. The dataset was meticulously pre-processed to include tokenization, semantic role and POS tagging, and language ID encoding. The token vectors were then standardized in length through padding or truncation and fed into the models using tensor representations. Model training was executed with K-Fold cross-validation, leveraging an AdamW optimizer to fine-tune the weights over 5 epochs. The evaluation metrics-precision, recall, accuracy, and F1 score-were employed to assess model efficacy. Among the models assessed, HING-MBERT demonstrated superior performance across all metrics, achieving the highest precision (0.7843), recall (0.7168), accuracy (0.9239), and F1 score (0.7985). This suggests its enhanced capability in semantic understanding and classification tasks compared to XLM-RoBERTa and M-BERT.