Lemmas Matter, But Not Like That: Predictors of Lemma-Based Generalization in Morphological Inflection

Recent work has shown that overlap -whether a given lemma or feature set is attested independently in train -drives model performance on morphological inflection tasks.The impact of lemma overlap, however, is debated, with accuracy drops from 0% to 30% reported between seen and unseen test lemmas.In this paper, we introduce a novel splitting algorithm designed to investigate predictors of accuracy on seen and unseen lemmas.We find only an 11% average drop from seen to unseen test lemmas but show that the number of lemmas in train has a much stronger effect on accuracy on unseen than seen lemmas.We also show that the previously reported 30% drop is inflated due to the introduction of a near-30% drop in the number of training lemmas from the original splits to the novel splits.These results help us better understand the factors affecting morphological generalization by neural models.