POLKE: A system for comprehensively annotating pedagogically-oriented grammatical structure use in language production

Analysing a speaker’s language, in particular their usage of certain vocabulary and grammatical structures, can give insights into their linguistic abilities and cognitive competence. Traditional approaches to characterising language production typically incorporate linguistic complexity metrics, which, while useful for various tasks such as learner proficiency evaluation and text readability assessment, have limited pedagogical relevance. There have been calls for more fine-grained annotations of linguistic structures in recent research, particularly in the field of second language acquisition (SLA), where grammar is a central part of second language (L2) learning. However, there is a lack of comprehensive tools for systematically analysing detailed grammatical structures. To address this gap, we present POLKE (Pedagogically-Oriented Language Knowledge Extractor), a tool for automatically annotating English grammar use based on the English Grammar Profile (EGP). In this paper, we introduce the POLKE system and outline the results of its evaluation, which yielded satisfactory results.