OMG-RL: Offline Model-based Guided Reward Learning for heparin treatment

Accurate medication dosing plays a critical role in the therapeutic process. Recent studies have investigated the application of Reinforcement Learning (RL) to optimize administration strategies. In RL, the definition of a reward function is indispensable; however, reliance on a limited set of explicitly defined rewards fails to capture the heterogeneity of clinical characteristics across patients. Moreover, the wide range of medications used in practice makes it impractical to design a dedicated reward function for each drug. To address this challenge, we propose learning a reward network that reflects clinicians’ therapeutic intentions, moving beyond predefined reward functions. In this study, we introduce Offline Model-based Guided Reward Learning (OMG-RL), an offline Inverse Reinforcement Learning (IRL) method. By incorporating a dynamic model into sample-based Maximum Entropy IRL, the method learns a parametrized reward network from offline data, enhancing the agent’s policy. We validate OMG-RL on the clinically significant heparin dosing task. Our results demonstrate that the agent is positively reinforced not only in terms of the recovered reward but also in activated partial thromboplastin time (aPTT), a key laboratory test for monitoring heparin effects. These findings suggest that the OMG-RL effectively captures clinicians’ therapeutic intentions. More broadly, our approach provides a general framework for RL-based medication dosing.