Structure dependence and the syntactic expertise of ChatGPT

Abstract The paper contributes to a discussion of the syntactic expertise of ChatGPT. It focuses on its handling of the range of ‘syntactic puzzles’, such as ‘onion sentences’, ‘garden paths’, grammatical illusions and structural ambiguity. The parsing of such cases by humans dwells on the property of structure-dependence, which is not shared by the AI language generator – ChatGPT. The latter is believed to process sentences in a sequential, though not strictly linear fashion, not using either recursion or hierarchy, the two pillars of human structure dependence. The assessment of ChatGPT’s syntactic expertise is based on two experiments in which the language model is asked to interpret and judge acceptability of English examples displaying such peculiar syntax. This is preceded by ‘a training session’ in which the model’s understanding of the nuances of such cases is assessed. The outcome is then confronted with the results of two parallel questionnaire studies, with the same set of linguistic tasks, administered to a group of human respondents, untrained in linguistics, with a native-like proficiency in English.