Data-Driven Analysis of Patch Inference Efficiency on Memory-Constrained Devices

Recent advancements in artificial intelligence have significantly propelled the development of deep learning inference on compact devices. As such, research into optimizing deep learning models for devices with constrained runtime memory resources has become increasingly active. Various optimization efforts have been pursued, including software algorithms like Quantization and Pruning. Moreover, optimizations that consider hardware resource constraints, such as Neural Architecture Search (NAS), have also been explored. Among these, the technique of spatially dividing the Activation Tensor and performing inference in Patch units has been studied. In Patch inference, identifying the appropriate points for Patch division and the optimal size of the Patches is crucial. This paper presents an experimental analysis aimed at exploring the optimal points for Patch inference considering memory and execution time constraints. Additionally, it proposes strategies for selecting optimal Patch parameters to enhance inference performance on resource-limited devices.