Extrapolation of Neural Networks for On-Chip Model Predictive Control: Insights from Nearest Neighbor Filtered Dataset and Disturbance Analysis

Neural Networks (NNs) can transform Model Predictive Controllers (MPCs) by serving as efficient and computation-friendly alternatives for control strategies. However, NNs struggle with extrapolation, where performance is unknown when inputs deviate from the training data distribution. To investigate this, a methodology is developed to analyze the relationship between out-of-distribution input states and MPC-trained NN performance. The strategy employs fast nearest-neighbor search algorithms to generate out-of-distribution NN inputs at increasingly far distances from the training set. By generating out-of-distribution elements at controlled distances from the input domain, we quantify NN prediction errors against the ground truth MPC outputs. Results show degradation in NN performance as perturbation distance increases, highlighting the network’s sensitivity to extrapolations. In addition, the extrapolation capacity is influenced by the number of learnable parameters in the NN architecture. The larger network tends to have higher accuracy in the testing dataset but exhibits overfitting and more out-of-distribution errors. These findings underscore the need for extra safety layers, such as real-time anomaly detection, to handle inputs beyond the MPC operating space. The development of such techniques is crucial in safety-critical systems such as type-1 diabetes management.