A Classifier of Word-Level Variants in Witnesses of Biblical Hebrew Manuscripts

The current project is inscribed within the field of stemmatology or the study and/or reconstruction of textual transmission based on the relationship between the available witnesses of given texts.In particular, the variants (differences) at the word-level in manuscripts written in Biblical Hebrew are addressed.A dataset based on the Book of Ben Sira is manually annotated for the following variant categories: 'plus/minus', 'inversion', 'morphological', 'lexical' or 'unclassifiable'.A strong classifier (F1 value of 0.80) is then trained to predict these categories in collated (aligned) pairs of witnesses.The classifier is non-neural and makes use of the two words themselves as well as part-of-speech (POS) tags, hand-crafted rules per category, and additional synthetically derived data.Other models experimented with include neural ones based on the state-of-the-art model for Modern Hebrew, DictaBERT.Other features whose relevance is tested are different types of morphological information pertaining to the word pairs and the Levenshtein distance between the words within a pair.The strongest classifier as well as the used data are made publicly available.Coincidentally, the corelation between two sets of morphological labels is investigated: professionally established as per the Qumran-Digital online library and automatically derived with the sub-model DictaBERT-morph.