A study on extraction QA-model based on disturbing word embedding and adversarial self-attention mechanism

Question-and-answer model research has recently been continuously explored in the NLP field, such as the development of human-machine interaction. But the extent to which these models truly understand language remains questionable, and in the face of antagonistic examples, relying too much on some of the word characteristics, makes the model unable to accomplish the tasks that require language understanding and reasoning ability, and the deficiencies of the model are magnified. Therefore, this paper proposes a method (PEC) based on disturbing word embedding and antagonizing self-attention mechanism, which is to obtain semantically retaining adversarial samples by disturbing verb embedding and reduce the noise effect of word embedding process on model training. Through contextual global constraint countermeasure training to enhance the model's deep mining of semantic information, which adds anti-self-attention mechanism to reduce the model's dependence on false features to improve the model's prediction of answers using simple matching strategies. The effectiveness and superiority of PEC method in enhancing the accuracy and robustness of question and answer model are verified by numerous experiments on multiple datasets.