Optimizing Memory Scheduling for Efficient Image Patch Inference on Resource-Constrained Devices

In this paper, we propose an optimized memory scheduling technique for efficient image patch inference on resource-constrained devices. Our approach addresses the inefficiencies in the existing MCUNet v2's memory scheduling algorithm, which maintains buffer tensors longer than necessary, leading to suboptimal memory usage. We designed an algorithm to dynamically adjust memory blocks and retain tensors only for the necessary duration during patch inference. This method significantly reduces the memory footprint, particularly for the maximum input/output tensor. The results demonstrate substantial improvements in memory utilization, enabling more complex models to be deployed on resource-constrained devices. Our findings highlight the potential of this scheduling technique to enhance the feasibility and performance of deep learning applications on MCUs.