Quality-Aware Sampling and Local Sequence Reward Modeling for Offline Imitation Learning

Offline reinforcement learning has emerged as a key research area in recent years. However, current algorithms face two main challenges in practical deployment. They rely heavily on handcrafted dense reward functions. Also, reward engineering in complex task scenarios needs significant domain knowledge and data labeling resources, leading to high economic costs. Some offline imitation learning methods learn reward functions via expert demonstrations. But the scarcity of expert data makes it hard to assess the quality of unlabeled data accurately, resulting in poor reward allocation. To address this key issue, we propose a quality-aware sampling framework to correct reward learning deviations by identifying near-expert trajectories. Also, we introduce local sequence information into the reward design process to build a new reward mechanism. We evaluated the framework on multiple continuous locomotion tasks within the D4RL benchmark. Results show our method improves policy learning effectiveness and demonstrates strong parameter robustness.