Gradient-Aware Depth Sensitivity Scoring for Optimizing Neural Network Pruning: (PhD Forum Paper)

Pruning is essential for deploying neural networks on edge devices, with unstructured pruning effectively compressing models without significant accuracy loss. Current methods focus mainly on weight magnitude, neglecting important factors like layer depth, activation nonlinearity, and gradient feedback, leading to suboptimal pruning. To overcome these issues, we propose a novel technique that adjusts layerwise sparsity dynamically using multiple contextual factors. Our score-based mechanism incorporates weight magnitude and adapts to modelspecific features such as layer depth, nonlinearity, and gradient information. This method preserves crucial connections, significantly reduces parameters, and consistently outperforms traditional pruning techniques, offering a robust, hyperparameter-free solution.