Optimality: The Unfathomable Mysteries of Machine Learning

The remarkable advancements in artificial intelligence have transformed industrial productivity and propelled scientific discovery. Its history and the flow of trends parallel those of physics: leveraging insights from procedural algorithms to explain empirical results and then seeking optimality for a principled understanding of the system. In this article, I will recollect some of those patterns in history and envision the progress of future understanding. A good understanding has led the community to new ideas by providing focus and setting trends for future exploration. Through mathematical insights into real-world problems, those with a background in physics can uncover novel and nontrivial knowledge in machine learning.