Boosting LLM’s Continual Sentiment Understanding for Low-Resource Languages

The emergence of Large Language Models has demonstrated remarkable performance in various sentiment understanding tasks. However, their sentiment understanding performance degrades sharply for low-resource languages (e.g. Acehnese) due to the primary training on high-resource languages (e.g. English). To this end, this paper proposes a new task, namely LLM's Continual Sentiment understanding for Low-resource Language (CS4L), seeking to boost the sentiment continual learning ability of LLMs across various low-resource languages. In particular, this paper poses two key challenges in CS4L: 1) Efficient sentiment retention for intra-language, aiming to mitigate the sentiment forgetting for each language efficiently. 2) Efficient sentiment alignment for inter-language, aiming to align the sentiment information across various languages efficiently. In this way, this paper proposes a new Efficient Sentiment Neuron Modifying (ESNM) approach to CS4L. Specifically, ESNM designs two novel blocks to address the two challenges above respectively: 1) Masked Retentive Updating block via merely one shared LoRA. 2) Precise Aligned Editing block via the efficient neuron editing. Comprehensive experimental evaluations on our constructed multiple CS4L datasets demonstrate the great advantages of our proposed ESNM approach. Compared to the the state-of-the-art baseline (i.e., O-LoRA), the accuracy of sentiment classification improved by an average of 2.18% and the forgetting rate decreased by 5.36%. This justifies the importance of the proposed CS4L task and the effectiveness of our ESNM approach in enhancing continual sentiment understanding of LLMs for low-resource languages.