Improved Memristor Control using Device Physics and Deep Reinforcement Learning

The steep non-linearity of memristive switching dynamics, combined with variations and asymmetry, can pose challenges in the accurate control of analog conductance updates. We present an effective control framework, "Probe-then-Program (PtP)", where the underlying physics of memristors is leveraged to fit a statistically accurate model from measured data, aiding the training of a programming agent. During the probe phase, the parameters’ statistical distributions in a physics-based memristor model are inferred using sequential Bayesian inference in measured data. This model then supports the training of a reinforcement neural network (Proximal Policy Optimization, PPO) that generates optimized write pulses. In the programming phase, the optimized pulses are applied to ensembles of devices, achieving significantly shorter tuning sequences for multi-level conductance programming compared to conventional write-and-verify technique. We experimentally demonstrate the full pipeline and show the efficacy by performing conductance mapping on a 4k memristive crossbar array. Improved control over nanoscale memristive devices in crossbar arrays supports many in-memory computing applications, while highlighting broader opportunities to integrate physics-based models and machine learning techniques.