Optimal Parameter Estimation of Biological Systems through Deep Reinforcement Learning

Recent works in Bayesian sequential optimal experimental design (sOED) for inverse problems have shown how the Bayesian sOED problem can be recast as a partially observable Markov decision process (POMDP) and then optimized for expected information gain with deep reinforcement learning (DRL) methods. We extend these results by showcasing empirical evidence of the effectiveness of these methods in solving for the other objectives of minimizing error between the maximum a posteriori (MAP) estimate and the true parameters and minimizing error in the forward model when using the MAP estimate. Among analyzing two source location inverse problems involving contaminants and acoustics already studied in the literature, we introduce the challenging Lotka-Volterra inverse problem modeled as a POMDP and showcase the effectiveness of DRL in all three studied objectives. The results show that DRL is highly versatile by achieving the best score on the majority of environments and objectives while matching the objective score of the next best method on the remainder.