Transformer-Guided Genetic Programming for Symbolic Regression

Genetic programming (GP) algorithms are widely used for solving symbolic regression (SR) problems. However, traditional GP struggles to effectively leverage knowledge gained from previous tasks when tackling new ones, leading to inefficient search processes. At the same time, the Transformer architecture has demonstrated superior performance through extensive experimental validation. To address this issue, we propose a novel approach called Transformer-GP. The core idea behind this method is to train a Transformer-Tree model to learn the distribution characteristics of node and edge components in symbolic trees. The model's predicted outputs are subsequently utilized as prior knowledge to guide genetic operations, including initialization and mutation, within the GP population. We conducted experiments on 12 benchmark problems, and the results demonstrate the effectiveness of the proposed method.