Lightweight LSTM-based memory architecture for occlusion-aware robotic manipulation on microcontrollers

This paper presents Tiny-STM, a compact recurrent controller for repetitive robotic manipulation when the target object may be temporarily occluded. The controller uses a standard four-gate LSTM with a 7-dimensional input, a 128-dimensional hidden state, and a linear 3-dimensional displacement output. A deterministic four-slot coordinate memory retains the most recent 3-D position of each object while it is not visible. Camera frames are processed on a host PC/SBC by color-based thresholding, whereas an ESP32 performs recurrent inference, inverse kinematics, and PWM command generation. The deployed recurrent model occupies 68.4 KB and was trained on 30 manually collected demonstrations. One trained INT8 checkpoint completed 17 of 20 physical trials, corresponding to an observed success rate of 85.0% and a two-sided 95% exact binomial confidence interval of 62.1–96.8%. The measured ESP32 computation time was 2.80 ms. Combining that measurement with the nominal 30-FPS frame period, profiled segmentation and communication times, and a datasheet-based actuator-response allowance gives an aggregated end-to-end latency estimate of 92.6 ms, or approximately 10.8 Hz. These results characterize a single deployed checkpoint. They do not measure training-run variability or isolate the contribution of the coordinate memory from that of the recurrent state.