Exploring Edge Inference Feasibility of Small Scale Deep Learning Models for Robotic Manipulation

Large deep-learning models for general-purpose robotic applications rely on powerful cloud computation resulting in high inference latency, computational costs, and network traffic. Despite such requirements, these models exhibit strong anchoring bias to training dataset and limited real-world performance in unseen environments. This paper proposes a lightweight approach for closed-loop robotic arm manipulation using off-the-shelf, open-source, small deep learning models, and a single robotic wrist-mounted camera. We evaluated the method on simple pick and place tasks inferred in real-time on an Apple MacBook M1 Pro, achieving <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{8 2. 5}$</tex> percent repeatability that is comparable to other approaches involving large models like Google RT, Meta R3M, VoxPoser, PerAct, and OK-Robot. Our approach puts the Vision-Language Model in a closed-loop governed by pre-defined actions that are invoked by a Small Language Model. We further explore various Language, VisionLanguage, and Vision-Language-Action models for edge inference and discuss the limitations of small, quantised models for such applications.