Beyond Rule-Based Processing: LLM-Assisted TEI Encoding of Editorial Interventions in Historical Correspondence

This long paper presentation discusses an LLM-assisted approach for automatically converting editorial interventions from (print) editions into proper TEI XML encoding, focusing on the correspondence corpus of Joseph von Hammer-Purgstall. The study addresses the challenge that rule-based processing cannot handle the ambiguous and contextually-dependent nature of editorial notations across different intervention types (abbreviations, errors, supplied text, unclear readings, omissions). The research develops a pipeline that first extracts bracketed sequences and applies LLM processing with tailored prompts. Currently, the results are still being evaluated against a manually annotated reference corpus. However, the pipeline is already prepared for future deployment: once reliable results are achieved, it will automatically replace the original bracketed sequences with TEI-encoded editorial interventions. Testing multiple commercial and local models on a small dataset containing square brackets as editorial interventions, they found GPT-5 performed best overall, though the local Qwen model provided a viable alternative to commercial solutions. The evaluation framework measures intervention categorization accuracy, element application correctness (including TEI validity, element accuracy and adherence to project rules), and content preservation, revealing that while LLMs show promise for this complex disambiguation task, challenges remain regarding original editorial guidelines reconstruction, TEI encoding flexibility, and the need for more granular evaluation metrics specifically designed for editorial intervention assessment.