RMixDA: A Random Compound Data Augmentation Operation Framework for Neural Machine Translation

In Neural Machine Translation (NMT), data augmentation is an effective method for improving model robustness by generating diverse augmented data from existing datasets. Typically, the quality of augmented data is evaluated based on its similarity and diversity to training data. Current approaches often rely on a single type of operation, limiting their ability to fully capture the range of possible variations in real-world data. This limitation will affect the robustness and generalization of model. To address this, we propose RMixDA: a Random Mixed Algorithm Framework For Data Augmentation, which dynamically integrates multiple NMT-oriented data augmentation methods. RMixDA defines an extensible collection of multi-augmentation operations using the Backus-Naur Form (BNF) framework, which categorizes data augmentation methods into pre-embedding (<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pre</i>_<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">embed</i>_op), embedding (<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">embed</i>_<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">op</i>) and post-embedding (<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">post</i>_<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">embed</i>_<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">op</i>) levels. These operations are combined using a random walk algorithm to generate diverse augmented data, for maintaining both similarity and diversity. We further incorporate contrastive learning, identifying augmented data as positive or negative samples to evaluate similarity more effectively. Additionally, a novel evaluation method for augmented data quality is introduced, which is regularized by the NMT model’s loss function to improve the positive correlation between augmented data quality and model training. Extensive experimental studies demonstrate the effectiveness of RMixDA and its scalability for integrating diverse data augmentation operations.