Addressing Inference Time of Machine Learning Models in Embedded Systems

Embedded Systems are used for a wide range of specialized computing purposes including surveyal, safety, security, and quality of life. Many areas that embedded systems are used in require the use of machine learning models. Constraints can be placed on embedded systems. Timeliness of execution, user satisfaction, security, power, and resource limitations must be considered when designing for embedded systems. Neural networks excel at complex tasks that are otherwise intractable, but their relatively high computational cost poses a challenge for inclusion in embedded systems. Neural network architectures should be optimized to reduce the total number of operations performed while maintaining comparable accuracy. Fewer operations correspond to faster inference times.A new neural network architecture, MultiskipNet, is proposed, implemented, and examined. MultiskipNet aims to improve upon existing works for decreasing resource requirements of machine learning models for embedded systems. MultiskipNet is designed to greatly decrease the total FLOPs required to perform image recognition with negligible or no loss to classification accuracy compared to existing models. Relevant existing works are described and used as the foundation for building MultiskipNets. The approach is described and evaluated against baseline models. The purpose of this dissertation is to examine existing architectures and their relation to embedded systems, propose modifications and new implementations, and evaluate the performance of neural networks and the systems they execute on. Neural networks are chosen as both a mechanism to improve system performance and as an optimization challenge.