ViNoise-CSE: Contrastive Learning with Tone-Dropping Augmentation for Robust Vietnamese Text Classification

Vietnamese is a highly tonal language where the presence or absence of diacritics significantly alters word meanings. On social media, users often omit diacritics—either for typing convenience or to evade content moderation filters—which degrades the performance of standard NLP models. In this paper, we propose ViNoise-CSE, a contrastive learning framework that integrates a novel ToneDropper data augmentation strategy. By jointly training with tone-dropped positive pairs, ViNoise-CSE learns sentence representations that are resilient to diacritic omission. We evaluate our method on two downstream tasks: Hate Speech Detection (ViHSD) and Sentiment Analysis (VSFC). Experimental results show that the pure tone-dropping configuration of ViNoise-CSE significantly outperforms the strong PhoBERT baseline under extreme synthetic diacritic-loss conditions (+12.28% Macro-F1 at 100% tone dropping). Furthermore, the learned representations demonstrate exceptional stability, maintaining nearly flat performance across all noise levels (dropping only 1.49% F1 from 0% to 100% noise). Our semi-supervised variant (SupCon) substantially improves few-shot performance, yielding a +20.88% Macro-F1 gain over the baseline with only 1% labeled data. Finally, we provide a critical empirical evaluation showing that while contrastive pre-training excels in few-shot settings, simple supervised data augmentation outperforms it when full labeled datasets are available.