Review of Tiny Machine Learning Model Pruning Techniques on Resource-constrained Devices

Lightweight pruning facilitates the deployment of machine learning models on resource-constrained devices. This review systematically examines pruning techniques across different technical paths, along with lightweight strategies that incorporate pruning. Regarding pruning techniques, the review successively delves into the principles, implementation methods, and applicable scenarios of structured pruning, unstructured pruning, and automated pruning. Structured pruning holds significant advantages in hardware implementation; unstructured pruning, on the other hand, demonstrates unique potential in fine-grained optimization, while automatic pruning methods achieve more precise model compression through intelligent search strategies. Subsequently, it centers on the synergy among pruning and other lightweight approaches, presenting the integration of pruning with quantization, the combination of pruning and distillation, as well as the pruning concepts incorporated in lightweight neural network architectures. The review concludes by highlighting some current challenges facing pruning technologies and offering insights into potential future research directions. These integration strategies not only enhance the model's operational efficiency in resource-constrained environments, but also offer innovative ideas for further compression while maintaining high accuracy.