VuL-MCBERT: A Vulnerability Detection Method Based on Self-Supervised Contrastive Learning

Software code defects are extremely harmful to society. Therefore, automated code vulnerability detection has become an increasingly critical technology in the field of software engineering. In recent studies, pre-trained models are widely used in software engineering tasks. We aim to enhance the understanding of vulnerable code through pre-training techniques. Most of the existing pre-trained models focus on the analysis of code text and perform vulnerability detection by fine-tuning. However, they do not highlight the specialized learning of the vulnerability code, and are not able to detect some vulnerabilities caused by subtle differences. To address these challenges, we propose VuL-MCBERT, a new pre-trained model. We augment the original code sample data with a designed heuristic method, and train the model in a dynamically updated way, so that it can learn vulnerable code patterns in the pre-training phase. Experimental results show that VuL-MCBERT achieves the best performance on the CodeXGLUE benchmark dataset and outperforms the best baseline by 2.78% on Acc, demonstrating the effectiveness of our approach.