Generative AI is not Magic

Although the capabilities of large language models are astonishing and exceed the expectations most researchers had about the potential of language modeling, these models are not complete black boxes, and the principles of how these systems work are not too difficult to understand. We show what theories, methods and skills can contribute to a basic understanding of large language models and how these can fit into a curriculum for students with and without a strong background in mathematics and computer science.