What Does Recovery Recover? An Empirical Study of Prediction Retention after Structured Pruning

To reduce the environmental impact of artificial intelligence, researchers aim to reduce the computation required for model inference while maintaining task performance. A common approach uses structured pruning to remove network components and recovery training to regain performance in the compressed model. Recovery is commonly evaluated through overall accuracy. However, improvements on some examples can compensate for unresolved pruning damage on others, concealing persistent failures that affect model reliability. To identify these hidden failures, we investigate how damage repair, corrections of original errors, and losses of correct predictions contribute to recovery gains, and whether repaired predictions remain correct during training. Our empirical study connects overall performance, group outcomes, and individual prediction histories, with matched continuation of the original unpruned model as a training reference. The analysis shows that small overall accuracy differences can coexist with larger group differences as corrections and losses offset each other. Prediction histories reveal the contributions of damage repair and other prediction changes, while accounting for continuation losses can change the interpretation of where additional losses concentrate. Repaired predictions can also become incorrect again. These findings characterize recovery quality through the composition of gains, the distribution of losses, and the persistence of repairs. Based on this analysis, we define a continuation-referenced retention diagnostic that complements overall and group accuracy and negative flips relative to the original model. Together, these measures provide criteria for evaluating recovery training and comparing compressed models according to computational savings, predictive performance, and prediction retention.