Using ChatGPT and Other AI Engines to Vocalize Medieval Hebrew

Hebrew is usually written without vowel points, making it challenging for some readers to decipher. This is especially true of medieval Hebrew, which can have nonstandard grammar and orthography. This paper tested four artificial intelligence (AI) tools by asking them to add vowel points to an unpublished medieval Hebrew translation of the Lord’s Prayer. The vocalization tools tested were OpenAI’s ChatGPT-3.5 and ChatGPT-4, Pellaworks’ DoItInHebrew, and Dicta’s Nakdan. ChatGPT-3.5 freely changed the text, even rewriting some phrases and adding an entire sentence. ChatGPT-3.5 also provided erroneous vowels in its rewritten Hebrew text. ChatGPT-4 did a moderately good job with only a few errors, but also modified the orthography. One of ChatGPT-4’s errors was not trivial, resulting in the invention of a word. When challenged, ChatGPT-4 corrected this confabulation by inventing another word, which it claimed was a “rare form” for which it provided a fictitious derivation. When challenged on this second made-up word, ChatGPT-4 replaced the word from the input text with a word based on an entirely different root. DoItInHebrew inserted vowels that produced a gibberish text. In contrast, Dicta’s Nakdan provided near perfect vocalization, with only one genuine error, but like ChatGPT-4 it modified the orthography. ChatGPT-3.5, ChatGPT-4, and DoItInHebrew exhibited serious “hallucinations,” of both the “factual” and the “untruthful” varieties, typical of other AIs, making them counterproductive for vocalizing historic Hebrew texts. Nakdan can be a powerful tool but still requires someone with expertise in Hebrew grammar to verify and correct the vocalization. Nakdan’s interface simplified correcting the vocalization, although it required its user to have advanced knowledge of Hebrew.