Adaptive Learning with Gaussian Process Regression: A Comprehensive Review of Methods and Applications

Gaussian processes (GPs) are a popular method in machine learning (ML) to model complex systems. One advantage of GPs over other ML models is their ability to quantify uncertainty in predictions. In the past, many advanced methods for GPs have been developed and published for various applications. Adaptive learning (ADL) is one of these applications, in which the consideration of uncertainty prediction plays a major role. The goal of ADL is to replace costly and time-consuming experiments and simulations of complex systems with surrogate models. This is achieved by strategically minimizing queries to maximize efficiency. In the ML literature, various reviews cover either GP methods or ADL strategies. Their focus is more on specific aspects. A comprehensive overview of different GP methods in various ADL applications was missing. This review categorizes GPs and related advanced methods for the first time in the context of ADL applications. A classification is provided for advanced GP methods, ADL methodologies, and practical application areas of GPs with ADL. This review distinguishes between ADL strategies with single-point and batch-query methods for Bayesian optimization and active learning, and highlights real-world applications such as material and product design, as well as efficient modeling for costly simulations and experiments. By combining these aspects, it offers a comprehensive guide for researchers and practitioners applying ADL with GPs to their specific use cases.