Adaptive Linear Control of a Cartpole Using Minimum Spiking Neurons Trained with Prescribed Error Sensitivity

In the search for highly parallel, power-efficient, and scalable chips for control applications, neuromorphic architecture has emerged as a promising candidate. Neuromorphic computing promises to address some of the the von Neumannian computing challenges, and computes using spiking neurons with adjustable synaptic delays that help match the latency of the controller with the system. However, learning in spiking neural networks poses a problem because of the non-differentiability of spikes. Although there are approximations like surrogate gradient decent for supervised learning, learning algorithms customized for spiking neural networks are required. Moreover, in order to design a compact, application-specific controller, the approximate knowledge of the number of neurons needed for a particular task is of importance, because the size of the chip depends on the number of neurons and synapse compartments on the chip. This work combines both of these needs and investigates the control performance on a cartpole with an adaptive controller by varying the number of neurons. The bob mass is varied and friction is added to the cart and the pole. First, the work shows the implementation of the Prescribed Error Sensitivity (PES) learning rule using a single neuron for the supervised learning of some standard functions, within the Neural Engineering Framework. Next, the work investigates the adaptive control of a cartpole against varying parameters, by progressively reducing the neuron number starting from 20 neurons, to arrive at a minimum number of spiking neurons, when trained with the PES rule. This knowledge is then used to run the adaptive controller on an FPGA board.