Vision-based reinforcement learning control of soft robot manipulators

Purpose This study aims to tackle control challenges in soft robots by proposing a visually-guided reinforcement learning approach. Precise tip trajectory tracking is achieved for a soft arm manipulator. Design/methodology/approach A closed-loop control strategy uses deep learning-powered perception and model-free reinforcement learning. Visual feedback detects the arm’s tip while efficient policy search is conducted via interactive sample collection. Findings Physical experiments demonstrate a soft arm successfully transporting objects by learning coordinated actuation policies guided by visual observations, without analytical models. Research limitations/implications Constraints potentially include simulator gaps and dynamical variations. Future work will focus on enhancing adaptation capabilities. Practical implications By eliminating assumptions on precise analytical models or instrumentation requirements, the proposed data-driven framework offers a practical solution for real-world control challenges in soft systems. Originality/value This research provides an effective methodology integrating robust machine perception and learning for intelligent autonomous control of soft robots with complex morphologies.