Performance Comparison of Rule Matching, spaCy and BERT for Location Extraction in Unstructured Texts

Extracting location information from tweets and other social software statements is essential for governments and organisations to make timely decisions. However, challenges include extracting valid locations from massive posts and handling latest events in a few posts, which require models to process data quickly and extract the correct locations from small data. This paper constructs a comparative experimental framework for location entity recognition based on three techniques, namely rule matching, spaCy, and BERT. This study selects about 19,000 labelled social media posts. Results show that rule matching has a high precision rate (>90%), but suffers a low recall rate (~35%) and low efficiency; the spaCy (en_core_web_sm) model balances performance under full data training (F1: 0.8778, recall: 100%), and its small sample version (only 100 data) still maintains excellent performance (F1: 0.9040), demonstrating a strong small sample adaptability; the BERT model performs outstandingly in the semantic understanding task (F1: 0.8418), but its small-sample version suffers from a significant overfitting problem.