Neural pronominal anaphora resolution for Amharic text using multiple embeddings

We describe a deep neural model for pronominal anaphora resolution in Amharic, a Semitic language of Ethiopia. In Amharic, pronouns may appear as standalone words or implicit ones, i.e. morphemes of other words. Both carry morphological features that can be used to determine their potential antecedents among all the mentions of entities in a text. To achieve this, the model computes span embeddings that combine context-dependent boundary representations with a head-finding attention mechanism and prunes the space of possible pronoun-antecedent pairs in a coarse-to-fine and heuristic approach. To facilitate a proper treatment of implicit pronouns the model combines character-based morpheme embeddings with pre-trained word embeddings by means of a BiLSTM recurrent neural network that captures a given word's morphological and semantic features. These representations are contextualized by applying another BiLSTM neural network. We also include an attention mechanism over the words in each span to indicate its headword. The model achieves F1 scores of 50.27% and 50.03% on the test data when using a heuristic or the coarse-to-fine approach, respectively.