A review of offline training of robotic manipulator control under limited data

Offline reinforcement learning (RL) has emerged as a promising paradigm for robotic skill acquisition, particularly for manipulation tasks where real-world trial-and-error interaction is costly, time-consuming, or unsafe. In contrast to broad offline RL surveys, this paper presents a focused narrative review centered on robotic manipulation under data-limited conditions, where datasets often suffer from narrow state–action coverage, sparse successful demonstrations, noisy trajectories, and limited behavioral diversity. These constraints exacerbate key challenges in offline RL, including distributional shift, unstable value estimation, weak reward propagation under sparsity, and poor policy generalization. This paper adopts a focused narrative review methodology; it does not claim systematic review status and does not follow a PRISMA protocol, but instead provides a structured, evidence-informed synthesis of representative offline RL methods selected for their methodological significance, benchmark relevance, and applicability to data-limited robotic manipulation conditions. The review analyzes major families of offline RL methods, including conservative value-based approaches, behavior-constrained learning, advantage-weighted methods, uncertainty-aware techniques, model-based offline RL, sequence-modeling approaches, and offline-to-online adaptation strategies. The discussion is grounded in benchmark observations from D4RL and NeoRL, alongside evidence from robotic manipulation systems such as QT-Opt and RoboNet. Comparative analysis suggests that conservative and uncertainty-aware methods typically offer stronger robustness and policy stability under constrained robotic datasets, whereas behavior-constrained and advantage-based approaches remain highly sensitive to dataset quality, behavioral diversity, and reward structure. Model-based and sequence-modeling methods can further improve data efficiency and long-horizon capability, but introduce additional challenges related to model reliability, synthetic rollout fidelity, and sequential data requirements. The paper also discusses practical issues in real-world deployment, including safety constraints, contact instability, sensor noise, and safe online adaptation. Finally, it highlights future research directions such as uncertainty-aware learning, physics-informed world models, foundation models, representation learning, and safe offline-to-online adaptation for robotic manipulation. Overall, this review provides a targeted synthesis of offline RL methods for robotic manipulation under realistic data-scarce conditions, while identifying key limitations, benchmark trends, and open research challenges for next-generation robotic learning systems.