Improving code completion efficiency through grouped attention

Abstract Although the code completion model based on the Transformer architecture has achieved remarkable results, the computational complexity of its multi-head self-attention grows quadratically with the increase of sequence length, resulting in low efficiency. To this end, we propose a code completion model based on grouped attention, referred to as GACC. This method groups the attention heads of a representation in the query, key, or value so that the attention heads in the same group share the single attention head of other representations, thereby reducing the number of parameters and computational complexity. We conducted experiments on public datasets of Python and JavaScript, and the experimental results show that compared to models based on multi-head self-attention, GACC effectively reduces the time taken for the model to suggest the next code token while achieving comparable performance on Top-k and MRR metrics.