Instruction-tuning pretrained causal language models to restore ancient Greek papyri and inscriptions

Abstract This article presents an experiment in fine-tuning Meta’s Llama 3.1 8B Instruct model to restore missing characters in ancient Greek inscriptions and documentary papyri. For papyri, the restoration model achieved a character error rate of 14.9 per cent, top-1 accuracy of 73.5 per cent, and top-20 accuracy of 86.0 per cent for sequences up to 10 characters. Geographic attribution reached 66.4 per cent accuracy (top-3: 79.9 per cent); chronological attribution showed a mean deviation of 21.7 years from actual dates (median: 0 years). For inscriptions, restoration achieved 20.5 per cent CER, 63.7 per cent top-1 accuracy, and 83.0 per cent top-20 accuracy. Geographic attribution reached 75.0 per cent accuracy (top-3: 83.7 per cent); dating showed a mean deviation of 37.1 years (median: 3 years). When benchmarked against Ithaca on a shared test set as well as recently edited inscriptions, the instruction-tuned models excelled in text restoration while naturally handling scriptio continua. However, geographic and chronological attribution performance was lower than Ithaca’s. When retrained with Ithaca’s 80/10/10 data split, the model still outperformed Ithaca in restoration, even on the test set with recently edited inscriptions. Although comparisons to previous models should be treated with caution—since, due to the pretraining of Llama, they were not trained on identical datasets—the results suggest that fine-tuning large pretrained language models using instruction templates holds significant promise for ancient text restoration.