Finding the Sweet Spot: An Empirical Study on Dataset Size, Performance, and Efficiency in Relation Extraction

Relation extraction (RE) is a critical task in natural language processing that identifies relationships between entities within text. While increasing the size of annotated datasets generally enhances model performance, the cost and effort associated with human annotation can be prohibitive. This study investigates the impact of dataset size on RE performance using the Semantic Evaluation 2010 Task 8 dataset. Four deep neural models-Attention-Based Convolutional Neural Network, Attention-Based Bidirectional Long Short-Term Memory Network, Relation Bidirectional Encoder Representations from Transformers, and Matching the Blanks-were evaluated across eight subsets of the dataset to simulate size reduction. Performance metrics such as accuracy, precision, recall, and F1-score were analyzed. The results indicate that although larger datasets improve model performance, the benefits diminish beyond a certain size, illustrating the law of diminishing returns. These findings emphasize the importance of balancing dataset size and quality for cost-effective training and suggest strategies such as data augmentation and transfer learning to optimize performance with limited data.