An Efficient Human-in-the-loop Method for Stochastic Policy Reinforcement Learning

This study introduces Entropy-Q Human-in-the-Loop Reinforcement Learning (EQ-HRL), a novel human-in-the-loop reinforcement learning framework designed for continuous control tasks. To tackle the dual challenges of sample inefficiency and excessive reliance on expert input in interactive RL, we propose a dual-condition advising mechanism that integrates policy entropy analysis with Q-value discrepancy assessment. The framework utilizes dynamic thresholding to enable adaptive guidance triggering and incorporates a dual-experience replay architecture to enhance knowledge assimilation. Extensive experiments conducted across benchmark environments demonstrate that, within a constrained advice budget, the proposed method significantly reduces expert queries while maintaining competitive success rates. These results highlight the framework’s ability to foster effective human-AI collaboration in complex decision-making scenarios by striking a balance between autonomous exploration and strategic expert intervention.