- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
Machine unlearning often uses approximate updates to avoid the cost of retraining a model after data deletion. However, executing these updates in finite-precision arithmetic introduces rounding and quantization errors that can undermine guarantees derived in exact arithmetic. This paper proposes ExecCert, a release-time verification framework that checks whether the actual unlearned model satisfies either the original method’s certificate or a specified accuracy requirement relative to retraining, without explicitly computing the exact retrained model. For sequential deletions, the authors develop Incremental RRV for ridge-regression heads on frozen representations, maintaining certified bounds on accumulated numerical errors and approximate-inverse accuracy with work per deletion, where is the feature dimension. Experiments on four published unlearning implementations and sequential deletion services evaluate how certification affects release decisions, the tightness of error bounds, verification costs, and the feasibility of satisfying requirements under different numerical precisions.
Critical strengths
- As machine unlearning moves toward practical deployment, reliable certification becomes increasingly important. In this direction, this work addresses this need by checking whether numerical errors in the executed updates undermine the intended guarantees.
- For ridge heads on frozen representations, Incremental RRV maintains certified numerical evidence and reduces verification costs when release checks are sufficiently frequent.
Critical weaknesses
-
The paper certifies linear prediction scores on a fixed, predefined set of representations, but does not establish corresponding guarantees for general trainable neural networks or generalization guarantees for unseen inputs. Also, the experiments do not demonstrate that the chosen numerical tolerances correspond to a meaningful level of resistance to recovering deleted information under the practical settings. This restricts the interpretation of the certificates as evidence of successful unlearning.
-
The incremental method targets a frozen feature extractor with a trainable ridge-regression head. The additional gradient-based certificate requires strong convexity which general neural network training does not ordinarily satisfy even with adding a proper regularization term. Extending the analysis and evaluation to trainable feature extractors or nonconvex objectives would substantially strengthen the contribution.
-
Standard certified-removal definitions compare the distributions of models produced by unlearning and retraining, limiting an observer’s ability to distinguish between them. RRV instead bounds the distance between a particular executed candidate and a specified retraining reference. This bound alone does not establish distributional indistinguishability.
-
The incremental method maintains dense matrices with storage and can require rebuilding, where is the feature dimension. Its the FP32 stress experiment shows lower certification coverage, greater total processing time, and higher memory use than fresh verification. These results limit the generality of the efficiency claims. Experiments across larger feature dimensions, longer deletion sequences, and different levels of numerical sensitivity would help establish when incremental verification is beneficial. Demonstrating applicability to larger trainable components would require additional methodological development beyond the current ridge-head implementation.
Questions for authors
- What additional assumptions would be needed to extend RRV to trainable feature extractors and nonconvex neural networks?
- Can RRV be extended to certify approximate indistinguishability between the output distributions of the unlearned and retrained models on unseen inputs? What additional assumptions would this require?