The Lies Characters Tell: Utilizing Large Language Models to Normalize Adversarial Unicode Perturbations

Homoglyphs, Unicode characters that are visually homogeneous to Latin letters, are widely used to mask offensive content.Dynamic strategies are needed to combat homoglyphs as the Unicode library is ever-expanding and new substitution possibilities for Latin letters continuously emerge.The present study investigated two novel mitigation approaches that do not rely on strict mappings but instead harness the power of large language models to neutralize both known and unknown homoglyphs:(1) indirectly normalizing homoglyphs by replacing non-Latin characters with a delimiter and prompting large language models to "fill in the blanks" and (2) directly normalizing homoglyphs by using large language models to determine which characters should be replaced with Latin letters.We found that GPT-4o-mini constructed normalized text with an average cosine similarity score of 0.91 to the original tweets when applying our indirect method and 0.96 to the original tweets when applying our direct method.This study indicates that large language model-based normalization techniques can effectively unmask offensive content concealed by homoglyphs.