EFVD: A Framework of Source Code Vulnerability Detection via Fusion of Enhanced Graph Representation Learning and Pre-trained Transformer-Based Model

With the increasing number of publicly disclosed security vulnerabilities, the threat of software vulnerabilities to national security and network security can no longer be ignored. The rise of open-source software has provided a wealth of code resources, facilitating the application of deep learning in the domain of source code vulnerability detection. Research on vulnerability detection focuses on how to effectively represent the syntactic and structural information of the code and how to make the deep learning model learn the relevant features of the vulnerable code effectively. Existing methods often hard to balance the local dependencies and global information of code when performing code representation, and it is easy to learn features unrelated to vulnerabilities when faced with unbalanced datasets. To solve these problems, we propose a vulnerability detection model EFVD, which combines sequence-based and graph-based methods. EFVD consists of three main parts: (1) a sequence-based module that uses pre-trained Codebert to learn the global information of the code; (2) Graph-based module, we propose a GGNN with edge attention mechanism (EA-GGNN). EA-GGNN can dynamically assign weights to different edge types, fuse heterogeneous edge information into node representation, and effectively capture local and remote dependencies between nodes. (3) Vulnerability classification module, we used focal loss to replace the cross-entropy loss to enhance the model's focus on minority classes. Compared to the state-of-the art methods, EFVD achieves a maximum absolute improvement of 35.63% in accuracy and 289.32% in F1 scores over the three publicly available benchmark datasets.