OBI-BERT: A Pre-trained Language Representation Model for Ancient Script Character Prediction

This paper presents a computer-aided method for interpreting Oracle Bone Inscriptions (OBI) utilizing a domain-specific BERT pre-training model. To tackle critical challenges in current OBI studies, such as incomplete character recognition and ambiguous character identification, we introduced OBI-BERT, a tailored pre-trained language model. This model employs a deep-shallow feature fusion strategy, significantly improving its understanding of OBI texts by initializing the embedding layer with pre-trained OBI CBOW and optimizing learning rates differentially. During training, we implemented a dynamic random masking strategy (with a mask rate of 20%) and utilized a six-layer encoder structure to achieve advanced semantic modeling of OBI texts. The experimental results indicate that our method attains an accuracy of 66.26% in OBI character prediction tasks, showcasing not only precise recognition of common OBI characters but also robust performance in managing incomplete and ambiguous characters. This approach offers a novel technical pathway for OBI research, potentially overcoming the limitations of traditional manual interpretation and expediting the compilation and study of historical documents from the Shang Dynasty.