Design of automatic code defect detection model based on Transformer

In order to improve the accuracy of defect detection in complex codes, an automated recognition model based on Transformer is designed to study the feature extraction, semantic encoding and defect modeling mechanism. The CodeXGLUE dataset is used as an example to construct a multi-label classification task with 21,854 samples and a maximum Token length of 512. Experimental results show that the model achieves 0.78 on multi-class F1-score, which is significantly better than that of BiLSTM (0.61) and CNN (0.65), and maintains a high recognition ability on the exception call and variable undefined classes. The ablation experiments verify the significant contribution of positional coding and Focal Loss mechanism to performance stability and class separation.