Using transformers to learn system models

Abstract Models play a critical role in supporting Verification and Validation activities. However, they are often unavailable in practice, such as for legacy systems or third-party components. Model learning addresses this gap through two main approaches: passive learning, which infers models from existing execution traces, and active learning, which interacts with the system under learning (SUL) to generate more general models, but at higher computational cost. In this work, we propose a novel application of Transformer architectures to implicitly learn generic system models from execution traces alone, combining the efficiency of passive learning with the generality of active learning. We design and evaluate two Transformer-based architectures: one focused on exploitation, achieving over $$\varvec{95\%}$$ valid trace generation; and another focused on exploration, generating approximately $$\varvec{60\%}$$ novel, previously unseen traces. Our methods outperform state-of-the-art model learning techniques, demonstrating that Transformers can achieve results comparable to active learning while requiring significantly fewer resources.