Graph Reinforcement Learning-based Reachability Map for Generalized Mobile Manipulation

Mobile manipulators need to determine feasible navigation positions before manipulation tasks. Real-world environments, with varying obstacles and objects, pose significant challenges for computing optimal navigation positions due to their variability. In this work, a novel method named <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</u>raph <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</u>einforcement Learning-based Re<underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a</u>chability <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</u>ap (GRAM) is proposed. First, GRAM uses a graph attention network (GAT) to capture the spatial relationships between objects. Then, it leverages the Q-value from the pre-trained critic network to generate the reachability map. The reachability map is integrated into navigation policies for long-horizon tasks, effectively solving the skill transition problems. Extensive simulation and real-world experiments were conducted on the Fetch mobile robot platform. The results demonstrate the superiority of GRAM, with simulation results showing an average 16.3% performance improvement over the baseline in four flexible environments. In long-horizon tasks, GRAM’s overall task success rate improved by 4.2%. The project is open-source at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/nubotnudt/Grand RM</uri>.