Trainable movement control using spikes and muscle-twitch dynamics

The biological sensorimotor system is a source of inspiration for the design of neuromorphic ballistic control systems. A large portion of sensorimotor-inspired research focuses on the sensory encoding and information processing stages of the system. However, research on broader task-performance systems, involving actuator control on the output side remains scarce. In this work, we develop and train a neuromuscular-inspired model to perform ballistic control. In the model, a spiking neural network's output spikes are used to generate twitch-like signals. These twitches are the basis for generating a continuous fluctuating output signal that is used to operate an actuator. We refer to the the used model as the Twitch Neural Network (TwNN). As a test case, the model is trained to control the paddle of an adapted version of the game of Pong. An adapted version of the Direct Feedback Alignment learning rule, specifically for integrate-and-fire neurons, is introduced. The new rule avoids the update-locking problem of backpropagation, allowing network weight updates in parallel. The model output consists of one group of agonist-innervating motor neurons, and one group of antagonist-innervating motor neurons. We find that it is possible to teach a neuromuscular-inspired system to control the paddle in the game of Pong with the adapted Direct Feedback Alignment learning rule. The best-performing baseline model achieved a hit rate of 96%. By applying logarithmic scaling to the output activity, a hit rate of 98% could be achieved. Finally, by replacing the neuromorphically unrealistic exact summation steps with leaky integrators in training, the range of good learning parameters became more narrow and clear. The best-performing model reaches a hit rate of 99%. Threshold analysis during training has shown that learning is robust to a variety of neuron thresholds. Noise analysis has shown that the system is robust to membrane potential noise during inference for uniform noise up to values in the order of around 0.1-1% of the neuron threshold value per time step.