Critical current density prediction in high temperature superconducting tapes using transformer-based deep learning

Abstract This paper presents a deep learning approach that employs large-scale pre-trained transformer models to predict the critical current density ( J c ) in high-temperature superconducting (HTS) materials under various temperatures and magnetic fields. The current widely adopted J c prediction methods, particularly Kim-like models, exhibit non-negligible errors at high magnetic fields, with the mean absolute percentage error (MAPE) reaching as high as 127.4% in the 5–8 T range on our validation dataset. Moreover, these models require parameter fitting at each temperature, with prediction accuracy reduced significantly when experimental data is sparse. To address these limitations, three encoder-only transformer based models (10 M, 85 M and 446 M parameters) were trained on a public dataset containing 260 823 samples from 20 different HTS tapes. These models can predict J c from the operating temperature, externally applied magnetic field, and tape type. Compared to Kim-like models, the proposed transformer models demonstrate substantial improvements across a wide range of magnetic fields and temperatures. In the 5–8 T range, the transformer models achieve mean absolute error (MAE) reductions of 82%–86%. On the Robinson test dataset (26,083 samples), all transformer models achieved MAE < 20 A cm −1 and MAPE < 10%, maintaining R 2 > 0.99 across all test conditions. Subsequently, three validation cases with extended experimental conditions (4.2–10 K and up to 18 T) were evaluated to test the versatility of the proposed models, with comparison against four established machine learning methods. Validation results indicate that large-scale transformer models maintain high predictive accuracy under extreme extrapolation—in the Shanghai Superconductor case containing 168 experimental samples, yielded an R 2 of 0.9916 and a MAPE of 4.36%, while even the 446M parameters model completed each prediction in about 45 ms, enabling near real-time J c prediction for practical applications. These results verify that the developed models successfully captured the current-carrying capability of HTS materials, enabling accurate predictions far beyond their training boundaries, providing valuable guidance for HTS device design and optimisation. The developed J c prediction tool is available at https://uoesupermachine.github.io/SupermachineGroup/Jcpre .