Better Reliability Compression: Model Pruning with Calibrated Uncertainty Estimation for Mobile Deep Learning Applications

Model compression techniques enable developers to deploy powerful Deep Neural Network (DNN) models on mobile devices with smaller model sizes, faster inferences, and lossless accuracy. However, the impact of compression on a model's ability to identify potentially incorrect predictions, called model reliability, has not been well studied. We incorporate structured pruning techniques and uncertainty estimation to enable compressed models to identify potentially incorrect predictions, such as random-guess predictions, which are critical in safety-critical scenarios. Furthermore, we characterized how the reliability of compressed models changes under varying compression ratios. Our evaluations show that the reliability of compressed models degrades much faster than their accuracy as the compression ratio increases. To address this problem, we proposed a framework, Better Reliability Compression (BRC), which finetunes and calibrates the uncertainty estimation in compressed models. In addition, evaluations on out-of-distribution (OOD) data detection and adversarial example detection demonstrated that the BRC framework enhances the reliability of compressed models, outperforming those fine-tuned solely with cross-entropy.