Challenges in Syntax-semantics Investigation for Language Models: A Case Study of Korean Control Dependencies

This study evaluates the ability of language models (LMs) to process subject-verb dependencies in Korean control constructions, involving a covert subject (PRO), using three models: KLUE-RoBERTa, KR-BERT, and KoELECTRA. We focused on LMs’ capacity to detect honorific mismatches and manage complex grammatical structures. Unlike previous findings in Indo-European languages, KLUE-RoBERTa and KR- BERT showed greater sensitivity to subject NPs, whereas KoELECTRA was more responsive to object NPs and better at detecting honorific mismatches. Additionally, increasing the distance between subject and object NPs generally improved model performance.