An Efficient Transformer Model Enhanced by S-Transform and Transfer Learning for Predicting Gas Distribution in Deeply Buried Reservoirs

In predicting gas-bearing potential in deeply buried reservoirs, traditional time-frequency analysis struggles with weak seismic responses. Inspired by Transformers in text translation, this study develops an efficient Transformer model that adaptively captures gas-sensitive frequency features through multi-head self-attention mechanisms. The framework operates on 1D instantaneous amplitude spectra with binary gas-bearing/gas-free labels, generating probabilistic gas presence predictions. Methodologically, the approach combines S-transform (ST) for high-resolution time-frequency decomposition with transfer learning for overfitting mitigation. Implementation proceeds through three stages: First, ST decomposes seismic time data to analyze frequency energy distributions, extracting main frequency intervals for dataset construction. Second, an efficient Transformer model is built, where the seismic amplitude spectrum is segmented using a Segment Embedding layer and transformed into high-dimensional vectors. These vectors, combined with classification tokens and learnable positional encoding, are processed by Efficient Multi-head Self-Attention (EMSA) to enhance inter-head information flow. The classification output is derived from the final hidden state of the classification token via linear classification and softmax layers. Third, transfer learning involves synthetic noisy data pretraining, interpolation-based model adaptation, and low-learning-rate fine-tuning. Experimentally, the pre-trained model demonstrated robust noise tolerance, achieving 95% accuracy on test dataset containing noisy (-15dB~5dB SNR) and clean samples. After fine-tuning, the final model achieved a testing accuracy exceeding 85% of seismic data from the Sichuan Basin in China. The predicted gas-bearing reservoir distribution closely matched the well-logging data, demonstrating robust generalization and transferability across diverse geological domains.