Memory Efficient Reinforcement Learning for Microcontroller Class Edge Devices: Near DQN Control at a Fraction of the Memory in Solar-Powered Street Lighting

Solar-powered street lights and similar smart city nodes must decide, minute by minute, how much to sense, which on-device model to run, and how brightly to light, all from a small battery and a modest panel, on a microcontroller with very little memory. Reinforcement learning (RL) can learn this trade-off, but most reported controllers say little about what the policy itself costs to store. This paper compares three RL controllers, tabular Q-learning, linear function approximation (Linear FA), and a deep Q-network (DQN), against five baseline policies in a Python simulation of a solar street-lighting node with a 100,000 J battery, a 100 mW peak solar panel, and a 0.2 to 1.6 W LED. The control problem is posed as a Markov decision process with a four-dimensional state and a 27-action joint control space covering TinyML model selection, sensing frequency, and LED brightness. Over 20 greedy evaluation episodes, the RL policies earned cumulative rewards of 298.7 to 349.6, against −355.6 to −822.7 for the baselines. DQN reached 94.9% accuracy and kept the node running for the full 1,440 minutes, whereas the Greedy baseline exhausted its battery at minute 741. Linear FA reached 94.7% accuracy with a policy of only 1.3 KB, compared with 14.9 KB for DQN and 63.3 KB for Q-learning, and earned a composite deployment score of 0.926 against 0.958 for DQN and 0.634 for Q-learning. The differences were significant at the 95% confidence level. For nodes of this class, a well-featured linear policy captures most of the benefit of deep RL at a small fraction of the footprint.