An Assessment of NLP Approaches for the Comparison of Glosses

This paper describes the novel application of two large language models, fine-tuned on Latin text, to the novel task of comparing glosses drawn from different medieval manuscripts. The aim is to measure how effectively the relationship between glosses, and the manuscripts containing them, can be determined by automatic means. The performance of these two sentence similarity models was compared against that of more rudimentary models which compare the character content of glosses. The development of a gold standard dataset, utilised in testing models for this experiment, is also described. Finally, all models were applied to a real-world test set, and their output manually analysed by an expert annotator. The results of these experiments demonstrate that gloss relations can be determined with a relatively high degree of accuracy using the models described in this article, and that the application of such models to gloss content can reveal valuable information about the interactions between manuscript sources.