SPRBM: Adaptive Parameter-Wise Pruning via Task–Complexity Gradient Alignment

Parameter importance in neural-network pruning is commonly estimated from static weight statistics, predefined sparsity rules, or isolated gradient snapshots. Such criteria do not directly control how strongly each parameter is regularized during task optimization. We introduce SPRBM, a sparsification framework that uses task–complexity gradient agreement for parameter-wise adaptive regularization during training and post-training pressure calibration. For every trainable parameter, SPRBM measures the signed agreement between the task gradient and the gradient of a differentiable complexity objective. This signal determines an adaptive regularization coefficient during training. After training, coefficients recomputed on multiple validation mini-batches are averaged and converted into parameter-specific pruning thresholds. The resulting framework does not impose a fixed global sparsity pattern. Instead, pruning pressure emerges from the interaction between task optimization, parameter complexity, and post-training validation-batch calibration. We instantiate SPRBM with \(L_1\) and \(L_2\) complexity functions and evaluate it on autoencoders, restricted Boltzmann machines, vision transformers, and LoRA adapter sparsification for language adaptation. Across these settings, SPRBM produces architecture-dependent sparsification regimes, including approximately 85% sparsity for the RBM, 30–35% sparsity for vision transformers, and 38–53% sparsification of LoRA parameters. Matched-sparsity ablations further show that disrupting the correspondence between the learned alignment signal and parameter locations degrades performance, indicating that the observed behavior cannot be explained by sparsity level alone.