Exploring Contextualized Embeddings for C++ Code Search and Algorithms Classification

Contextualized embeddings of words (such as BLOOM and BERT) outperform static embeddings (such as Word2vec, Fasttext, and GloVe) for many natural language processing tasks. In this article, an exploration is conducted of the use of contextualized embeddings for the purposes of searching for C++ code and classifying the algorithms that are utilized. Furthermore, a novel approach is proposed for reducing the dimensionality of the resulting embeddings, leveraging a multilayer perceptron trained with Multi Similarity Loss. The experiments performed for the methods of multilabel classification of algorithms and for the code similarity search show that the proposed method not only effectively reduces the dimension of embeddings, but also increases their representativeness.