Online redesign of dynamic experiments for high-throughput bioprocess development using deep reinforcement learning

For efficiency and reproducibility, modern biotech laboratories increasingly rely on robotic platforms to perform complex, dynamic experiments to generate informative data for bioprocess development and knowledge discovery. Furthering the goal of self-driving biolabs imposes the need of automating cognitive demanding tasks such as redesigning online an experiment to maximize information gain in the face of different sources of uncertainty including microorganism behavior, hardware failure, and noisy measurements. In this work, a reinforcement learning (RL) based formulation of the online experimental redesign problem for dynamic experiments is proposed. Simulation-based learning of a redesign policy for parallel cultivations in a high-throughput platform is discussed to provide implementation details regarding the RL agent design (perceptions and actions) and the reward function used. The simulated environment and a training workflow for sequential information control are integrated with the Proximal Policy Optimization (PPO) algorithm to learn how to modify ’on the fly’ an offline design based solely on previous observations and actions in a given experiment. Results obtained demonstrate the feasibility of using deep RL to guarantee the quality of the generated data and increase the level of automation that can be used on high-throughput platforms. • The importance of online redesign of dynamic experiments to generate informative data in the face of uncertainty. • Close-loop redesign as a reinforcement learning control problem. • Simulation-based deep learning of the redesign policy. • Illustration of the proposed approach in a high throughput cultivation platform.