Soybean genomic phenotype prediction method based on improving the transformer model with batch normalization and cosine annealing algorithm

In breeding, single nucleotide polymorphisms (SNPs) are often used to predict phenotypes to assist breeding and improve breeding efficiency. In this paper, a GPformer network prediction model combining batch normalization and cosine annealing optimization algorithm is proposed. The model first uses the multi-head self-attention mechanism in Transformer to extract features from gene sequences, then uses the batch normalization layer to improve training stability and accelerate convergence, and finally uses the full connectivity layer to output regression results. In addition, in the process of network training, cosine annealing adjustment strategy is used to prevent the model from falling into the local optimal solution and failing to jump out of the local solution. The model is compared with four common models in predicting the soybean phenotypes of SdWgt, Oil and Hgt, and the best results are obtained in different prediction tasks. We also validated the need for each component of the model through ablation studies.