The Syntax-Semantics Gap: Assessing GPT Models' Telicity Recognition in Directional Prepositions

This study examines the capacity of three GPT models (GPT-3.5 Turbo, GPT-4, and GPT-4o) to recognize telicity distinctions in directional prepositions, comparing their performance to humans across four telicity diagnostics. Results reveal that, while the models exhibit some differentiation in specific tests, they show substantially lower sensitivity to telicity than humans. This underscores a notable gap between syntax and semantics in GPT models. In line with Mahowald et al. (2024), these findings highlight a contrast between formal competence, where models excel in syntactic tasks, and functional competence, where they continue to struggle with intricate semantic challenges like telicity.