Nonmyopic Bayesian process optimization with a finite budget

Optimization under uncertainty is inherent to many PSE applications ranging from process design to RTO. Reaching process true optima often involves learning from experimentation, but actual experiments involve a cost (economic, resources, time) and a budget limit usually exists. Finding the best trade-off on cumulative process performance and experimental cost over a finite budget is a Partially Observable Markov Decision Process (POMDP), known to be computationally intractable. This paper follows the nonmyopic Bayesian optimization (BO) approximation to POMDPs developed by the machine-learning community, that naturally enables the use of hybrid plant surrogate models formed by fundamental laws and Gaussian processes (GP). Although nonmyopic BO using GPs may look more tractable, evaluating multi-step decision trees to find the best first-stage candidate action to apply is still expensive with evolutionary or NLP optimizers. Hence, we propose modelling the value function of the first-stage decision also with a GP, whose data will correspond to virtual evaluations of second-stage decision trees build upon myopic rollouts. Thus, the nonmyopic initial decision can be efficiently optimized via BO and the virtually learned value function. Effectiveness of the approach is demonstrated in a wide benchmark with synthetically generated functions as well as to optimize small batch production with a chemical reactor.