Prompting Bidirectional Model Transformations - The Good, The Bad and The Ugly

This paper investigates the comparative effectiveness of model-to-model transformations generated by an LLM based upon user prompts versus those created with dedicated model transformation languages, using a standard benchmark. The emergence of Generative AI offers a novel approach, allowing developers to specify transformations in natural language rather than learning specialized languages. However, our findings suggest that, in its current state, generative AI does not yet pose a threat to dedicated model transformation languages. While AI-assisted approaches promise to provide flexibility and accessibility, dedicated model transformation languages still offer structured advantages crucial for complex transformations, especially when bidirectionality and incrementality are mandatory requirements. This research contributes to the ongoing discourse on the role of AI in software engineering, highlighting its potential and current limitations in enhancing model transformation processes.