A novel LLM-based GPT-neo transformer with machine learning architecture for promoters prediction in Agrobacterium

Promoters are vital for the initiation of gene transcription, and their accurate uncovering in bacterial genomes is crucial for the progression of genetic and biotechnological research. This research introduces an innovative approach for detecting promoters in Agrobacterium tumefaciens, integrating a large language model (LLM)-based GPT-Neo transformer with Random Forest (RF) feature engineering and traditional machine learning methodologies. The GPT-Neo model was initially used to tokenize nucleotide sequences and produce high-dimensional embeddings that include the structural and interactional characteristics of promoter regions. The embeddings function as input for the RF classifier, which conducts probabilistic feature extraction to provide augmented feature vectors, therefore enhancing the model’s prediction capabilities. The integration of LLM-based feature extraction with RF feature engineering results in substantial performance enhancements, with the model attaining an accuracy of 97% and an AUC of 0.99, surpassing conventional methods. Furthermore, the use of the Synthetic Minority Over-Sampling Technique (SMOTE) rectifies class imbalances, hence providing equitable sensitivity and specificity. This methodology signifies a considerable progress in promoter prediction, providing a solid foundation for prospective applications in bacterial genomics and bioinformatics.