LIP-NER: Literal Patterns Benefit LLM-Based NER

Large Language Models (LLMs) can enhance the performance of Named Entity Recognition (NER) tasks by leveraging external knowledge through in-context learning.When it comes to entity-type-related external knowledge, existing methods mainly provide LLMs with semantic information such as the definition and annotation guidelines of an entity type, leaving the effect of orthographic or morphological information on LLM-based NER unexplored.Besides, it is non-trivial to obtain literal patterns written in natural language to serve LLMs.In this work, we propose LiP-NER, an LLM-based NER framework that utilizes Literal Patterns (LiP), the entity-type-related knowledge that directly describes the orthographic and morphological features of entities.We also propose an LLM-based method to automatically acquire literal patterns, which requires only several sample entities rather than any annotation example, thus further reducing human labor.Our extensive experiments suggest that literal patterns can enhance the performance of LLMs in NER tasks.In further analysis, we found that entity types with relatively standardized naming conventions but limited world knowledge in LLMs, as well as entity types with broad and ambiguous names or definitions yet low internal variation among entities, benefit most from our approach.We found that the most effective written literal patterns are (1) detailed in classification, (2) focused on majority cases rather than minorities, and (3) explicit about obvious literal features.