Propagating the Prior From Shallow to Deep With a Pre‐Trained Velocity‐Model Generative Transformer Network

Abstract Building subsurface velocity models is essential to our goals in utilizing seismic data for Earth discovery and exploration, as well as monitoring. With the dawn of machine learning, these velocity models (or, more precisely, their distribution) can be stored accurately and efficiently in a generative model. These stored velocity model distributions have the potential to be utilized in future applications, such as in regularization and uncertainty quantification in inverse problems. However, most generators, like normalizing flows or diffusion models, treat the image (velocity model) uniformly, not explicitly exploiting the spatial dependencies and resolution changes. To address this weakness, we introduce VelocityGPT, a novel implementation that utilizes Transformer decoders trained autoregressively to generate a velocity model from shallow subsurface to deep. Owing to the fact that seismic data are often recorded on the Earth's surface, a top‐down generator can utilize the inverted information in the shallow as guidance (prior) to generating the deep. Specifically, the distribution of velocity information in the shallow layers guides the predictions deeper. To facilitate the implementation, we use an additional network to compress the velocity model. We also inject prior information, like well or structure (represented by a migration image) to generate the velocity model. Using synthetic data, we demonstrate the effectiveness of VelocityGPT as a promising approach in generative model applications for seismic velocity model building.