Neural Networks, Probability, and Information

The 2024 Nobel Prize in Physics was awarded to Hopfield and Hinton, who laid the foundation for advancements in artificial intelligence. This special issue explores the contributions of statistical physics and information theory to machine learning. Machine learning uses neural networks to represent the probability distributions of data. In this process, ‘raw’ probabilities are refined and structured with the help of information theory. Machine learning can be divided into discriminative and generative models. This article focuses on the roles of neural networks, probabilities, and information, as exemplified by prominent generative models such as the Boltzmann machine, variational autoencoders, and diffusion models. Finally, we conclude by examining the implications of machine learning advancements for physics.