CCKA: Continual Cross-Domain Knowledge Adaptation for Multi-Domain Machine Translation

Multi-domain machine translation (MDMT) has demonstrated remarkable performance in handling multiple domains within a single model and enhancing cross-domain knowledge. The major challenge lies in continually updating the original MDMT models as new domain data arrives. Previous works primarily focus on two separate aspects: mitigating catastrophic forgetting in the original model and acquiring incremental knowledge from new domains. However, addressing both issues simultaneously would better meet practical needs. Additionally, the cross-domain knowledge between the original and new domains remains underutilized. To this end, we propose a <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</b>ontinual <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</b>ross-domain <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</b>nowledge <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</b>daptation for multi-domain neural machine translation framework (<bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CCKA</b>), our approach contains two learning stages that encourage original models to acquire cross-domain knowledge from new domain data, and preserves the model architecture without introducing parameters. Experimental results on the UM-Corpus and OPUS multi-domain datasets show superior performance of our proposed model compared to representative baselines.