Neural Networks in Astrophysics and Plasma Physics: Transformers, PINNs, KANNs, and all that

Recently, new approaches have been developed for the numerical solution of problems in Dynamical systems and boundary value problems for Partial differential equations, which are based on the very general models of the type of Neural networks. Those of them who respect the physical background of the processes are called Physics Informed models (PINNs); their respect to the physics is expressed by means of the properly formulated loss functional which uses explicitly the differential equations and the boundary value conditions in a regularized fashion. Despite not excelling in terms of performance (accuracy and training computing time), PINNs present a compelling alternative for addressing challenges that prove difficult for traditional methods, such as inverse problems or parametric partial differential equations (PDEs).Things have gone much further by trying to mimic in other areas the revolutionary innovation in NLP (especially, in the Large Language models - LLMs) – the so-called Transformers. However, their applicability is still restricted to processes described by Dynamical systems, i.e. where the time is present in the equation.We will give a short overview of the above newest developments in the area of AI, and to some successful applications to Astrophysics and Plasma Physics. - FULL TEXT available in PDF.