Mixed Information Bottleneck for Location Metonymy Resolution Using Pre-trained Language Models

Metonymy resolution (MR) is a crucial challenge in natural language understanding and information retrieval. Recent large-scale pre-trained language models have shown promising results in various natural language processing (NLP) tasks, including MR. Despite these achievements, current models still struggle in many real-world scenarios. Since these models rely heavily on contextual information and ignore entity information, they are prone to extract irrelevant features and overfit when fine-tuned with less training data. In this article, we propose a mixed information bottleneck framework to address the above issues, which learns optimal data representations based on the principle of minimal sufficiency. Our model can effectively mitigate irrelevant features in context and entity by using different types of information bottlenecks for entity and context information separately while reducing the dimensionality of latent representations. We show that our approach achieves state-of-the-art performance on three benchmark datasets for location MR, outperforming previous Bert-based methods by a large margin. Ablation studies and qualitative analysis show the effectiveness of our models in reducing dimensionality while extracting more relevant features.