Automatic information extraction from ego-documents: a comparative analysis of the effectiveness of large language models based on the example of K.A. Berezkin's diary.

The subject of the study is a comparative analysis of the performance, analytical strategies, and limitations of four large language models – Gemini-2.5-Pro, o3, Grok3, and Deepseek-v3 – in the task of extracting structured information from a historical ego-document. The analysis aims to determine the models' ability to work with complex narratives characterized by a high degree of subjectivity, an abundance of indirect evidence, multi-layered meanings, and emotional coloration. The key limitations of the models – over-interpretation, missing indirect evidence, and the trade-off between completeness and accuracy – are considered part of their analytical strategies. The material used was the diary of the Vologda gymnasium student K.A. Berezkin for the year 1849. The work addresses a complex task of developing and testing an approach that allows for the transformation of unstructured source text into a dataset suitable for solving a specific historiographical task – analyzing the perception of the European revolutions of 1848-1849 in the Russian province. The methodology is based on the automatic extraction of structured information using large language models. A comprehensive toolkit has been developed, including a domain-specific ontology, prompts, and a detailed JSON schema for data capture. The performance of the models was evaluated based on quantitative (completeness, accuracy, F1-score) and qualitative indicators (granularity, adherence to the ontology, understanding of historical context, typical errors). The scientific novelty lies in the first systematic testing and comparative analysis of the performance of leading language models in working with a historical ego-document in domestic historiography. It was established that the models implement various data extraction strategies: from exhaustive, but "noisy" coverage (Gemini-2.5-Pro) to highly accurate, but selective (Deepseek-v3), which directly determines the suitability of the resulting dataset for different research scenarios: from exploratory analysis to the creation of verified databases. The key conclusion of the study is that automated extraction is not merely a technical operation, but a form of digital hermeneutics. Accordingly, the final dataset is not objective data passively "discovered" in the source, but capta – a set of information selected for a specific task. The study shows that the application of artificial intelligence raises historian's requirements for critical expertise, shifting their role from information retrieval to verification and interpretation of machine results.