- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies unlearning for materials-property regression. It start from the assumption that similar structures remain after deletion, so a retrained model can still predict the deleted record well. The authors define the “deletion floor” as the retrained model’s expected loss on that record. They attempted to prove that an update close to retraining must have a loss near this floor. They used an exact ridge formula to separate the floor from the prediction change caused by deletion. They also propose a per-request reporting protocol. Experiments mainly use ridge regression on synthetic data.
Critical strengths
There are two highlights about this paper.
First, the paper makes concrete the point: in regression unlearning, high post-deletion accuracy is not a failure in itself.
Second, Near-duplicate structures (e.g., ionic substitutions) are common in materials-property data. This makes the domain a reasonable and under-explored test case for studying redundancy effects in machine unlearning evaluation.
Critical weaknesses
Severn concerns about the paper
First, the theory involved is largely known. For example, Theorem 1 is a standard step in differential privacy (bounding expectations of bounded functions), and its “sharpness” follows from a two-point construction. Beyond restating these known results, the paper does not explain how they help with the core difficulties of unlearning in materials-property prediction.
Second, the referenced baseline seems not to be a true retraining. Feature standardization and the RFF bandwidth are computed with the deleted record and kept fixed. It is suspected that the “retrained” reference therefore still depends on the deleted record through preprocessing.
Also, there is little connection between theory and experiments.
Questions for authors
-
Why is this unlearning study framed around of the materials? So far as the review know, the deletion floor, the bounds and the protocol look fully generic and would apply to any regression task with redundant data.
-
Following question 1, does any part of the method or analysis depend on materials-specific structure?
-
How many retrains does each request need to estimate the floor for a NN model?
-
What decision does the module FloorScore support (given its weak correlation and its use of the deleted label)?