Fine-tuning LLAMA models for historical databases: methods, challenges, and long-term implications

Abstract This article examines the potential of fine-tuned open-source language models—specifically Meta’s Llama family—for constructing large-scale historical databases, using a biographical database as an example. Historians confront an ever-growing volume of documents, yet the capacity to read and interpret sources remains a major bottleneck. Drawing on a project that aimed to transform Danish and Norwegian biographical entries into structured data, this study demonstrates how fine-tuned language models can perform at the level of a skilled research assistant while also producing superhuman output in terms of quantity. The article addresses critical methodological and technological obstacles, including context-window limits, logic errors, language translation inconsistencies, and the shifting availability of commercial models. It then argues that open-source models provide the stability and reproducibility crucial to long-term historical research, as they are less prone to sudden updates or deprecations. The article includes reflections on the extensive post-processing required to transform raw model outputs into curated historical databases, as well as a discussion of how workflows and fine-tuning strategies can be optimized for marginal languages such as seventeenth-century Danish. In conclusion, it offers practical lessons on leveraging large language models for structured historical data, emphasizing the importance of model control, sustainable research workflows, and realistic expectations regarding speed and error rates.