Addressing Information Loss in Multi-View Robotic Manipulation Through History Observation

Multi-view end-to-end reinforcement learning as robotic manipulation control methods has already been widely used. However, in real-world scenarios, it is common for visual information from cameras to be incomplete or missing, such as object occlusion or sudden environmental change. Such incidents are difficult to avoid and fatal for manipulation deployment and performance. How to ensure continuous high-quality task performance even when visual information is missing while maintaining low resource consumption is a challenging but highly valuable research field. In this work, we propose a solution to the problem of missing information from views during tasks by acquiring it from history and making trade-offs between historical views and damaged original views to facilitate the manipulation's task completion. Our method has demonstrated state-of-the-art results across an extensive range of environments and various experimental settings, showing a strong grasp of historical information and robustness to prolonged information loss. In extreme cases, the model can complete the designated tasks successfully with just a glimpse of the environment.