Keywords extraction algorithm based on attention mechanism of BERT model

Text keyword extraction refers to finding the words or phrases from an article that best represent its topic or content. Text keyword extraction is not only helpful to quickly understand the content of the text, but also can be used in NLP tasks such as information retrieval, automatic summarization, text classification, etc. In order to improve the performance of keyword extraction algorithm, this paper proposes a keyword extraction model AttentionRank based on the self-attention mechanism of BERT model by using BERT pre-trained language model. The pre-trained BERT language model can recognize keywords through self-attention and cross-attention, and enhance the ability of keyword extraction algorithm to understand context. The experimental results show that AttentionRank has obvious advantages in keyword extraction compared with LDA and LSA algorithms.