Too Polite to be Human: Evaluating LLM Empathy in Korean Conversations via a DCT-Based Framework

As LLMs are increasingly used in global conversational settings, concerns remain about their ability to handle complex sociocultural contexts.This study evaluates LLMs' empathetic understanding in Korean-a high-context language-using a pragmatics-based Discourse Completion Task (DCT) focused on interpretive judgment rather than generation.Our dataset systematically varies in relational hierarchy, intimacy, and emotional valence, enabling fine-grained comparisons between proprietary/open-source LLMs and native Korean speakers.Most LLMs showed over-empathizing tendencies and struggled with ambiguous relational cues.Neither model size nor Korean fine-tuning significantly improved performance.Additionally, humans exhibit a nuanced understanding of social context and relational nuances, whereas LLMs rely on surface-level heuristics.These findings highlight the limitations of LLMs in sociopragmatic reasoning and introduce a scalable, culturally flexible framework for evaluating socially aware AI.