Evaluating LLM-Based Temporal Subject Assignment in Fiction: The Impact of Evaluation Frameworks on Metadata Interpretation

This study examines how automatically generated temporal subject metadata for fiction can be evaluated when time is represented as year intervals. Using 45 Swedish novels with manually assigned chronological headings as the basis for comparison with model-generated intervals, the study contrasts two evaluation perspectives: overlap-based classification (precision, recall, F1) and boundary-based deviation (RMSE). The analysis shows how different evaluation frameworks highlight different aspects of temporal alignment between predicted and manually assigned intervals. By combining these perspectives, the study makes visible different types of temporal discrepancies and shows how evaluation design influences what is counted as alignment in interval-based subject metadata.