NAMP: A Network-Aware Model Partitioning Framework for Constrained Devices

Despite the growing computing capacity of sensors, enabling them to execute Machine Learning (ML) models without requiring cloud resources is still a challenge given their limited computational power, memory, and energy. Current ML frameworks for developing, deploying and executing ML models typically overlook these limitations and consider monolithic models that run in single-devices. This paper proposes the Network-Aware Model Partitioning (NAMP) framework that allows breaking neural networks into smaller submodels considering the resources available in the system. The submodels are deployed as services on a network of constrained devices to provide distributed inferences. Two emulated scenarios with 15 devices with limited memory show NAMP deploying a distributed LeNet-based model that would be unfeasible if monolithic. Comparative analysis shows the NAMP-derived solution minimises communication overhead, increasing inference efficiency.