Evaluating Large Language Models on Chinese Zero Anaphora: A Symmetric Winograd-Style Minimal-Pair Benchmark

This study investigates how large language models (LLMs) handle Chinese zero anaphora under symmetric minimal-pair conditions designed to neutralize shallow syntactic cues. We construct a Winograd-style benchmark of carefully controlled sentence pairs that require semantic interpretation, pragmatic inference, discourse tracking, and commonsense reasoning rather than structural heuristics. Using GPT-4, ChatGLM-4, and LLaMA-3 under zero-shot, one-shot, and few-shot prompting, we assess both accuracy and the reasoning traces generated through a standardized Chain-of-Thought diagnostic. Results show that all models perform consistently on items solvable through local cues but display systematic asymmetric errors on 19 universally misinterpreted sentences that demand deeper discourse reasoning. Analysis of these failures reveals weaknesses in semantic role differentiation, topic-chain maintenance, logical-relation interpretation, pragmatic inference, and long-distance dependency tracking. These findings suggest that while LLMs perform well on simpler tasks, they still face challenges in interpreting contextually omitted arguments in Chinese. The study provides a new controlled evaluation resource, an interpretable error analysis framework, and evidence of differences in symmetric versus asymmetric reasoning behaviors in LLMs. Future research could expand the current benchmark to longer discourse contexts, incorporate multi-modal or knowledge-grounded cues, and explore fine-tuning LLMs on discourse data, helping clarify whether asymmetric patterns stem from deeper reasoning challenges or from interactions between models and the evaluation format.