Optimizing Transformer-based Models for Radiation Estimation at High Energy Accelerator Facility

At high energy accelerator facilities, it is important to control radiation exposure to ensure the safety of the staff and public. In this paper, we optimized various Transformer-based deep learning models to efficiently estimate radiation exposure at the Thomas Jefferson National Accelerator Facility (JLab). We explored the effects of utilizing various model embedding types and training with either channel dependence or independence. The models were trained and optimized on radiation, energy, current, and beam loss accounting data collected from sensors at JLab. Our results demonstrated that the Transformer-based model utilizing temporal embedding and channel dependence achieved the best performance with an average R<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> score of 0.771. To the best of our knowledge, this work represents the first attempt to explore transformer deep learning models for radiation estimation at JLab and showed potential to further improve radiation estimation performance.